Reliability and Sufficiency of Audit Evidence (ISO 27001 Lead Auditor)
Introduction
Audit evidence is the foundation of every audit conclusion. In ISO/IEC 27001 Lead Auditor training, which is built on ISO 19011 (Guidelines for auditing management systems) and ISO/IEC 27006 / ISO/IEC 17021-1 (requirements for certification bodies), you must understand both how reliable a piece of evidence is and whether you have enough of it. A finding is only as strong as the evidence behind it.
Why It Is Important
1. Credibility of conclusions: The principle of an evidence-based approach (ISO 19011, Clause 4) requires conclusions to be rational, reliable and reproducible. Weak or insufficient evidence leads to wrong conclusions.
2. Fairness to the auditee: A nonconformity raised on hearsay or a single anecdote is unfair and can be challenged or overturned.
3. Certification integrity: Certification bodies and accreditation bodies rely on auditors to make defensible certification decisions. Unreliable evidence undermines the value of the ISO 27001 certificate.
4. Risk management of the audit: Audits are based on sampling. Understanding sufficiency helps manage audit risk, which is the risk that the auditor reaches an incorrect conclusion.
5. Defensibility: Findings must withstand appeals, complaints and review by the certification decision maker.
What It Is
Audit evidence (ISO 19011, 3.9): records, statements of fact or other information that are relevant to the audit criteria and verifiable.
Reliability refers to the quality and trustworthiness of evidence, meaning how far you can depend on it to reflect reality.
Sufficiency refers to the quantity of evidence, meaning whether enough has been gathered to support a confident conclusion about conformity or nonconformity.
A related concept is relevance, meaning the evidence must relate to the audit criteria (for example, a clause of ISO 27001 or an Annex A control). Evidence can be reliable but irrelevant, or relevant but insufficient. Good evidence is relevant, reliable, sufficient and verifiable.
How It Works: Reliability
Several factors generally increase reliability:
- Independent source: Evidence from a third party, such as a penetration test report from an external firm or a supplier SLA report, is usually more reliable than a self-assessment.
- Direct observation by the auditor: Seeing a screen lock activate, or watching a visitor sign in, is more reliable than being told it happens.
- Documented and objective: System-generated logs, records and reports are more reliable than verbal statements.
- Effective internal controls: Evidence produced under a well-controlled process, such as protected audit logs with integrity checks, is more reliable.
- Original rather than copies: Originals or system extracts are generally more reliable than photocopies or summaries.
- Contemporaneous: Records created at the time of the event are better than those reconstructed later.
- Corroboration (triangulation): Evidence confirmed by multiple methods (interview, document review and observation) is much stronger.
A typical hierarchy from strongest to weakest:
1. Auditor's direct observation, re-performance or technical testing
2. Independent third-party evidence and system-generated records
3. Internally generated documented records
4. Documented policies and procedures (these show intent, not implementation)
5. Uncorroborated verbal statements (interviews alone)
Important: Interview information should be verified. ISO 19011 states that only information that is verifiable can be audit evidence. Unverified statements may guide further enquiry, but should not alone support a nonconformity.
How It Works: Sufficiency
Sufficiency depends on:
- Risk and significance: Higher-risk areas, such as access control to critical systems, need more evidence.
- Population size and sampling: ISO 19011 Annex A.6 describes judgement-based and statistical sampling. The sample must be representative.
- Consistency of results: If early samples show conflicting results, expand the sample.
- Reliability of each item: Highly reliable evidence may need a smaller quantity. Weaker evidence needs more corroboration.
- Audit objectives and time available: Time is defined in the audit plan, but sufficiency should not be compromised. If evidence cannot be obtained, this is an obstacle that must be reported.
- Period covered: For certification, evidence must show the ISMS has operated over time, such as management reviews, internal audits and records across months, not just on the audit day.
The Process in Practice
1. Collect information through interviews, observation, and document and record review (ISO 19011, 6.4.7).
2. Verify the information to turn it into audit evidence.
3. Evaluate the evidence against the audit criteria to generate audit findings (conformity or nonconformity).
4. Review findings as a team to reach audit conclusions.
The chain runs: Sources of information, then collecting by sampling, then verification, then audit evidence, then evaluation against criteria, then findings, then review, then conclusions.
Examples
- Weak: The IT manager says all leavers have access removed within 24 hours.
- Stronger: A leavers' procedure exists requiring removal within 24 hours.
- Reliable and sufficient: The auditor takes a sample of 15 leavers from the HR system over 12 months, compares termination dates with Active Directory disable timestamps, and finds 3 accounts that were disabled after 10 or more days. This evidence is objective, system-generated, corroborated, representative and supports a nonconformity against Annex A 5.18 / 8.2 (2022 version) and the organization's own procedure.
Common Pitfalls
- Raising a nonconformity based only on an interview.
- Accepting a policy as proof that a control is implemented.
- Using a sample chosen by the auditee without checking that it is representative.
- Generalising from one instance to the whole organization without adequate sampling.
- Ignoring contradictory evidence.
- Confusing a high volume of weak evidence with sufficiency, since quantity does not compensate for poor quality.
Exam Tips: Answering Questions on Reliability and Sufficiency of Audit Evidence
1. Know the definitions verbatim: Audit evidence is records, statements of fact or other information relevant to the audit criteria and verifiable. Look for the word verifiable in answer options.
2. Rank evidence types: In questions asking which evidence is most reliable, choose auditor observation, re-performance, or independent or system-generated records over interviews or self-declarations.
3. Policy is not proof of implementation: If an option says the documented policy proves the control works, it is usually wrong. Look for records showing operation.
4. Corroboration wins: When asked what to do after an interviewee claims something, the best answer is to verify through records or observation, not to accept it or immediately raise a nonconformity.
5. Sampling logic: The correct answers usually show that samples should be representative, selected by the auditor, risk-based, and expanded when anomalies are found.
6. Insufficient evidence equals no finding yet: If evidence is inconclusive, the right action is to gather more, or record an area for follow-up or an opportunity for improvement. Do not raise a major nonconformity without sufficient evidence.
7. Scenario questions: Ask yourself three things: Is it relevant to a requirement? Is it reliable (source, objectivity, independence)? Is it sufficient (sample size, period, consistency)? State these explicitly in essay or long-answer formats.
8. Writing nonconformity statements: Include the requirement, the objective evidence (specific, traceable records, dates, sample sizes, IDs) and the gap. Examiners reward precise, factual evidence descriptions such as 3 of 15 sampled leavers rather than vague wording like some users.
9. Link to principles: Cite the evidence-based approach and fair presentation principles, and the risk-based approach when justifying sample sizes.
10. Watch for distractors: Answers with words like always, only, or never can be traps, although never conclude on unverified hearsay is a correct idea. Also watch for options suggesting the auditee should choose the sample, or that time constraints justify accepting unverified evidence.
11. Uncertainty and limitations: Remember that audit conclusions carry inherent uncertainty because of sampling. ISO 19011 expects auditors to be aware of this and report limitations, such as an inability to access evidence.
12. Time-based evidence for certification: For Stage 2 or surveillance audits, recognise that sufficiency requires records showing operation over time, such as internal audit and management review records.
Quick Memory Aid: R.R.S.V.
Relevant (linked to criteria), Reliable (trustworthy source and method), Sufficient (enough, representative), Verifiable (can be confirmed).
Summary
Reliability is about quality and sufficiency is about quantity. Together they determine whether an auditor can confidently and fairly conclude on conformity. In the exam, always favour verified, objective, corroborated and representative evidence, and choose actions that gather more evidence when it is weak or incomplete.
