Monitoring Auditor and Audit Program Performance

Monitoring Auditor and Audit Program Performance (ISO/IEC 27001 Lead Auditor)

Introduction
Monitoring auditor and audit program performance is a core responsibility of the person managing an audit program. It is described mainly in ISO 19011:2018, clauses 5.6 (monitoring the audit programme), 5.7 (reviewing and improving the audit programme) and 7.6 (evaluation of auditor competence). Supporting guidance comes from ISO/IEC 17021-1 for certification bodies and ISO/IEC 27006 for ISMS certification bodies.
The underlying idea is simple. An audit program is a management system in its own right, so it must be planned, carried out, checked and improved (Plan-Do-Check-Act). Monitoring is the Check step, and improvement is the Act step.

Why It Is Important
1. Reliability of audit conclusions: Top management, certification decision-makers and interested parties rely on audit results. If auditors are not competent, or audits are badly run, nonconformities may be missed or falsely raised. Both outcomes damage trust.
2. Achieving audit program objectives: The program exists to meet defined objectives. Examples include verifying ISMS conformity, supporting certification or driving improvement. Monitoring shows whether those objectives are actually being met.
3. Maintaining auditor competence: Competence is not permanent. Standards, threats and technologies change, for example the transition to ISO/IEC 27001:2022 and the restructured Annex A. Ongoing evaluation keeps auditors current.
4. Risk management: ISO 19011 requires the program manager to identify and address risks and opportunities. Examples include insufficient resources, wrong team composition, poor communication, or confidentiality and impartiality threats. Monitoring is how these risks are detected.
5. Accreditation and credibility: Certification bodies must show accreditation bodies that they monitor auditor performance. This includes witnessing auditors on site and reviewing their reports.
6. Continual improvement: Lessons learned feed back into future audit planning, training and methods.

What It Is
The topic has two connected parts.

A. Monitoring audit program performance
This means checking whether the audit program as a whole is being carried out as planned and is achieving its objectives. ISO 19011 clause 5.6 says the program manager should evaluate:
• Whether schedules are being met and program objectives achieved.
• The performance of audit team members, including the team leader and technical experts.
• The ability of audit teams to carry out the audit plan.
• Feedback from top management, auditees, auditors and other interested parties.
• Whether audit documentation is sufficient and adequate throughout the audit process.

Clause 5.6 also notes that monitoring can show a need to change the program. Possible triggers include:
• Audit findings.
• Demonstrated effectiveness or maturity of the auditee's management system.
• Effectiveness of the audit program itself.
• Changes to audit scope, criteria or methods.
• Changes to the auditee's management system or to external issues.

Clause 5.7 then covers reviewing the program to judge whether its objectives have been achieved. Review considers:
• Results and trends from monitoring.
• Conformity with audit program processes.
• Changing needs and expectations of interested parties.
• Audit program records.
• Alternative or new auditing methods.
• Alternative ways of evaluating auditors.
• Effectiveness of the actions taken to address program risks and opportunities.
• Confidentiality and information security issues relating to the program.

Results of the review can lead to corrective actions and program improvements. They should be reported to the individual or organization that commissioned the program, usually top management.

B. Monitoring and evaluating auditor performance and competence
ISO 19011 clause 7 describes how auditor competence is determined, evaluated, maintained and improved. The main steps are:
• Determine the competence needed (clause 7.2.3). Competence combines knowledge and skills, both generic and discipline-specific, plus personal behaviour (clause 7.2.2). Personal behaviour includes being ethical, open-minded, diplomatic, observant, perceptive, versatile, tenacious, decisive, self-reliant, able to act with fortitude, open to improvement, culturally sensitive and collaborative.
• Establish evaluation criteria, which may be qualitative or quantitative (clause 7.4). Examples include demonstrated behaviour, education, work experience, auditor training and auditing experience.
• Select evaluation methods (clause 7.5).
• Conduct the evaluation (clause 7.6), comparing collected information against the criteria.
• Maintain and improve competence (clause 7.6) through continual professional development and regular participation in audits.

Evaluation methods (ISO 19011 Table 3)
• Review of records: to verify background, such as education, training, employment, professional credentials and audit logs.
• Feedback: how auditor performance is perceived, through surveys, questionnaires, references, testimonials, complaints and performance evaluations.
• Interview: to assess personal behaviour and communication skills, verify information and test knowledge.
• Observation: to assess behaviour and the ability to apply knowledge and skills, through role play, witnessed audits and on-the-job performance.
• Testing: to assess behaviour, knowledge, skills and how they are applied, through oral and written exams and psychometric tests.
• Post-audit review: to obtain information on performance during audit activities and identify strengths and opportunities for improvement, through review of audit reports and discussions with the team leader, the team and, if appropriate, the auditee.

How It Works in Practice
Step 1: Define performance indicators (KPIs) for the program. Examples include:
• Percentage of audits completed on schedule.
• Percentage of audit reports issued within the agreed timeframe.
• Number of auditee complaints or appeals.
• Auditee satisfaction scores.
• Percentage of auditors meeting CPD requirements.
• Number of findings later overturned on technical review.
• Recurrence of nonconformities at auditees, which indicates corrective action follow-up quality.
• Resource utilization and audit day accuracy.

Step 2: Collect data. Sources include audit reports, audit plans, feedback forms, witness audit records, technical review results, complaint logs, training records and team leader evaluations of team members.

Step 3: Evaluate auditors individually. Typical activities are:
• The team leader evaluates each team member after the audit.
• Auditees provide feedback.
• The program manager or certification body performs periodic witness audits. Under accreditation practice these are often required at least once in each auditor's certification cycle, according to the certification body's procedures.
• Report reviewers assess the quality of evidence, findings and their traceability to requirements.

Step 4: Analyse trends. Patterns can show systemic problems rather than isolated ones. Examples include repeated weak reporting by one auditor, late reports across the program, or recurring complaints about a specific sector.

Step 5: Take action.
• For auditors: targeted training, mentoring, supervised audits, changes to assigned technical areas, or removal from the auditor pool if necessary.
• For the program: revised schedules, more resources, updated procedures, new audit methods such as remote auditing, or adjusted audit criteria.

Step 6: Review and report. The program manager conducts a periodic program review, reports the results to top management or the client, and keeps records as evidence of program implementation (clause 5.5.7).

Step 7: Improve continually. Feed outputs into the next planning cycle. This closes the PDCA loop.

Roles and Responsibilities
• Audit program manager: owns the monitoring and review process, evaluates auditors, manages program risks and reports to top management.
• Audit team leader: evaluates team members' performance, gives feedback to the program manager and ensures audit quality.
• Auditors: maintain their own competence through CPD and self-reflection, and accept feedback.
• Auditee: provides feedback on the audit experience.
• Top management or client: receives review outputs and provides resources.

Records to Retain
• Auditor competence and performance evaluation records.
• Selection of audit teams.
• Audit plans, reports and nonconformity reports.
• Feedback and complaints, with their resolution.
• Program review results and resulting actions.
• Records of maintaining competence, such as CPD logs.

Common Pitfalls
• Treating competence as a one-time qualification instead of something continually maintained.
• Relying only on auditee satisfaction, which may reward lenient auditors.
• Monitoring individual audits but never reviewing the program as a whole.
• Collecting feedback without acting on it.
• Ignoring impartiality and confidentiality as performance factors.

Exam Tips: Answering Questions on Monitoring Auditor and Audit Program Performance
1. Anchor answers in ISO 19011. Cite the relevant clause: 5.6 for monitoring, 5.7 for reviewing and improving, and 7 for auditor competence and evaluation. Examiners reward correct references to standards.
2. Use PDCA language. Describe monitoring as the Check stage and improvement as the Act stage. This shows you understand the system-level logic.
3. Distinguish the two levels. Questions often mix up program performance with auditor performance. State clearly which you are addressing, or address both. Program performance concerns objectives, schedules, resources and risks. Auditor performance concerns competence and behaviour.
4. Name concrete evaluation methods. List the Table 3 methods: records review, feedback, interviews, observation (witness audits), testing and post-audit review. Explain why combining several methods gives a balanced evaluation.
5. Give measurable KPIs. In scenario questions, propose specific indicators such as on-time report delivery or the number of complaints. Avoid vague statements like "check the quality".
6. Recommend proportionate actions. If a scenario describes a poorly performing auditor, choose supportive and corrective steps first: feedback, training, mentoring and a supervised audit. Escalate to withdrawal only if there is no improvement. Always mention keeping records.
7. Think about root causes. If several audits show the same problem, suggest a systemic program issue. Examples include inadequate procedures, poor audit duration calculation or insufficient briefing, rather than blaming individuals.
8. Include feedback from all interested parties. Mention top management, auditees, team leaders and auditors themselves.
9. Remember the reporting line. Program review results go to the individual or organization that commissioned the program, usually top management. Mention this in any answer about program review.
10. Spot the trap answers in multiple-choice questions. Be wary of options suggesting that:
• Competence, once established, needs no further evaluation.
• Only auditee satisfaction matters.
• The auditee is responsible for evaluating the program.
• Monitoring happens only at the end of the certification cycle.
The correct option usually reflects ongoing, multi-source, documented evaluation leading to improvement.
11. Link to risk. Show that monitoring identifies risks to achieving program objectives and checks whether actions on those risks were effective. This matches the clause 5.3 and 5.7 language.
12. Structure essay answers. A reliable structure is: definition, purpose, inputs, methods, outputs and actions, records, then improvement. Examiners appreciate logical, complete answers.
13. Use ISMS-specific context where possible. Mention auditor competence in Annex A controls, risk assessment methods, cloud security, or ISO/IEC 27001:2022 transition knowledge. For certification bodies, mention ISO/IEC 27006 requirements.

Sample Exam Question and Model Answer
Question: As audit program manager, you notice that three auditee complaints in six months concern the same auditor's unclear nonconformity statements. What should you do?
Model answer:
1. Record and investigate the complaints, and review that auditor's recent reports.
2. Discuss the findings with the auditor and the relevant team leaders.
3. Identify the competence gap, here the skill of writing nonconformities that are clear, evidence-based and traceable to requirements (ISO 19011 clause 7.2.3.2).
4. Arrange targeted training or mentoring, followed by a witnessed or supervised audit to verify improvement.
5. Check whether other auditors show similar weaknesses, which could indicate a systemic program issue such as missing report templates or calibration sessions.
6. Keep records of all of the above.
7. Include the case in the periodic program review reported to top management (clause 5.7).
8. Update procedures or training content as part of continual improvement.

Key Takeaway
Monitoring auditor and audit program performance means continually checking that the right people, using the right methods, are achieving the program's objectives, and acting on what you learn. In the exam, show structured, standards-based, evidence-driven and improvement-focused thinking, and you will answer these questions with confidence.