Audit Sampling Methods

Audit Sampling Methods in ISO/IEC 27001 Lead Auditor Practice: A Complete Guide

Introduction
Audit sampling is one of the most practical and most heavily examined topics in the ISO/IEC 27001 Lead Auditor syllabus. An auditor can almost never examine every record, every user account, every change ticket or every incident in an organisation within the time allowed. Sampling lets the auditor reach reliable, defensible conclusions about an Information Security Management System (ISMS) by examining only part of the available evidence.

The main guidance on sampling comes from ISO 19011:2018 (Guidelines for auditing management systems), clause A.6 and related sections. ISO/IEC 27006 and ISO/IEC 27007 add requirements and guidance specific to ISMS certification and auditing. ISO/IEC 17021-1 also matters for certification bodies when they sample across multiple sites.

Why Audit Sampling Is Important
1. Time and resource constraints: Audits are planned in audit days. Checking 100% of evidence is rarely feasible or cost-effective.
2. Reliability of conclusions: A properly designed sample gives confidence that what the auditor saw represents the whole population.
3. Audit risk management: Sampling always carries some risk that the sample does not reflect reality (sampling risk). Understanding the methods helps the auditor reduce and disclose that risk.
4. Objectivity and evidence-based approach: ISO 19011 lists an evidence-based approach as a principle of auditing. Sampling is central to collecting verifiable evidence in a systematic way.
5. Credibility of certification: Certification bodies must justify why they believe an ISMS conforms. Defensible sampling supports that judgement.
6. Multi-site organisations: ISO/IEC 27006 lets certification bodies sample sites under defined conditions, which makes multi-site certification practical.

What Audit Sampling Is
Audit sampling means selecting less than 100% of the items in a population and examining them, so that the auditor can form a conclusion about the entire population.

Key terms:
- Population: The full set of items about which a conclusion is needed, for example all access requests raised in the last 12 months.
- Sample: The subset actually examined.
- Sampling unit: The individual item selected, such as one change ticket or one employee training record.
- Sampling risk: The risk that the auditor's conclusion from the sample differs from the conclusion that would be reached by examining the whole population.
- Non-sampling risk: Risk from other factors, such as the auditor misinterpreting evidence or using the wrong procedure.

ISO 19011 notes that audit evidence is based on samples of available information. There is therefore an element of uncertainty in auditing, and people acting on audit conclusions should be aware of it.

The Two Main Families of Sampling Methods

1. Judgement-Based Sampling (Non-Statistical)
The auditor uses knowledge, skills and experience to choose the sample. ISO 19011 says judgement-based sampling should consider:
- Previous audit experience within the audit scope
- The complexity of requirements, including legal and regulatory requirements
- The complexity and interaction of the organisation's processes and management system elements
- The degree of change in technology, human factors or the management system
- Previously identified significant risks and opportunities for improvement
- Output from monitoring of management systems

Characteristics:
- Fast, flexible and focused on risk.
- Results cannot be projected statistically onto the population with a quantified confidence level.
- The auditor's conclusion is a professional judgement. It is not a statistically valid statement.
- It is the most common method in ISMS audits, especially certification audits with limited time.

Example: An auditor chooses five privileged access accounts on critical servers that were recently changed. The selection is based on risk, not chance.

2. Statistical Sampling
The sample is designed using probability theory. Each item has a known, usually equal, chance of being selected. ISO 19011 notes that statistical sampling design uses selection based on probability, so that results can be evaluated statistically.

Two common types:
- Attribute-based sampling: Used when there are only two possible outcomes for each item, such as correct/incorrect or pass/fail. Most ISMS control testing works this way, for example whether access reviews were signed off.
- Variable-based sampling: Used when the result is on a continuous range, such as the monetary value of an error or the time taken to patch.

Factors in designing a statistical sample:
- Confidence level: How sure the auditor wants to be (for example 95%).
- Tolerable error/deviation rate: The maximum rate of non-conformity the auditor will accept.
- Expected error rate: The anticipated rate of non-conformity.
- Population size and its characteristics.

Characteristics:
- More objective, with results that can be projected and defended mathematically.
- Takes more time and needs specialist knowledge.
- Requires a well-defined, complete and homogeneous population.
- The auditor must document the sampling plan.

Sample Selection Techniques
Whichever family is used, items can be selected in several ways:
- Random selection: Every item has an equal chance, often using random number generators. This is the basis of statistical sampling.
- Systematic (interval) selection: Pick every nth item after a random start, for example every 20th incident ticket.
- Stratified sampling: Divide the population into subgroups (strata) and sample each one. Examples are high-risk versus low-risk assets, or each department. This keeps important subgroups from being missed.
- Cluster sampling: Select groups of items, such as all records from one month, rather than individual items.
- Haphazard selection: The auditor picks items without a structured technique but tries to avoid bias. It is not truly random, so it is not statistical.
- Block selection: Choose a contiguous block, such as all changes in March. This is quick, but representativeness may be weak.
- Risk-based/targeted selection: Focus on high-risk, high-value or unusual items. This is a form of judgement sampling.

How Audit Sampling Works: Step by Step
ISO 19011 describes a sampling process along these lines:
1. Establish the objectives of sampling: What do you want to conclude? For example, whether new joiners receive security awareness training within 30 days.
2. Determine the extent and composition of the population: Identify the complete list, such as all new joiners in the period. Check that it is complete and reliable.
3. Select a sampling method: Judgement-based or statistical, plus the selection technique.
4. Determine the sample size: Base it on risk, population size, confidence required and time available.
5. Conduct the sampling activity: Select and examine the items.
6. Compile, evaluate, report and document results: Record which items were selected, why, and what was found. Evaluate whether deviations are isolated or systemic.

The audit report should note the sampling approach where relevant. It should also state that sampling limits the conclusions that can be drawn, since evidence is based on a sample.

Sampling in ISO/IEC 27001 Certification Contexts
- Multi-site sampling (ISO/IEC 27006): A certification body may sample sites instead of visiting all of them, but only if conditions are met. These include a single ISMS centrally administered and audited, internal audits and management review covering all sites, and similar processes at each site. Some sites must be chosen at random and others based on risk. The sample should reflect differences in size, activities, complexity, risk and previous audit results. The central function is always audited.
- Sampling of controls: Over a certification cycle (initial audit plus surveillance audits), the auditor must cover all applicable Annex A controls in the Statement of Applicability. They do not all have to be covered in every surveillance visit.
- Sampling of records: Within each control, auditors sample records such as access reviews, backup logs, supplier assessments and risk treatment evidence.
- Sampling of people: Auditors interview a cross-section of top management, process owners, IT staff and general employees to test awareness and implementation.

Practical Considerations and Pitfalls
- Population completeness: If the auditee hands over a pre-filtered list, the sample may be biased. Auditors should obtain the population independently or verify it, for example by extracting the list from the system themselves.
- Auditee-selected samples: Never let the auditee choose the sample. Doing so undermines independence and objectivity.
- Sample size adequacy: Larger, riskier or more variable populations generally need larger samples.
- Handling exceptions: If a deviation is found, the auditor may expand the sample to see whether it is isolated or systemic. This informs whether a finding is a major or minor nonconformity.
- Documentation: Record sample selection criteria, items selected and results in the working papers. This supports traceability and verifiability.
- Remote/technology-based audits: ISO/IEC 27007 and ISO 19011 acknowledge using technology for sampling, such as screen sharing and data extraction. Auditors should still make sure the population and sample are authentic.

Comparison Summary
Judgement-based sampling: chosen by auditor expertise; quick; risk-focused; not statistically projectable; most common in ISMS audits.
Statistical sampling: chosen on probability; time-consuming; objective; results statistically projectable with known confidence; needs a defined homogeneous population and a sampling plan.

Worked Example
Scenario: An organisation has 1,200 user access changes in 12 months. The auditor must assess control A.5.18 (Access rights) in ISO/IEC 27001:2022.
- Judgement approach: The auditor selects 15 changes, weighted towards privileged accounts, leavers and changes on critical systems. A nonconformity found here supports a finding, but the auditor cannot claim a precise error rate for the whole population.
- Statistical approach: Using a 95% confidence level and a tolerable deviation rate of 5%, a sampling table might suggest about 59 items selected randomly. If zero deviations are found, the auditor can state with 95% confidence that the deviation rate does not exceed 5%.

Exam Tips: Answering Questions on Audit Sampling Methods

1. Know the two families cold. Exams frequently ask you to distinguish judgement-based from statistical sampling. Remember: statistical = probability-based, projectable, quantified confidence. Judgement = expertise-based, risk-focused, not projectable.

2. Quote ISO 19011 factors. When asked what an auditor should consider for judgement-based sampling, list items such as previous audit experience, complexity of requirements, process complexity and interaction, degree of change, previously identified risks, and monitoring outputs.

3. Recognise the selection technique in scenarios. Spot the clues. 'Every 10th record' means systematic. 'Split by department and sample each' means stratified. 'Picked high-risk items' means judgement or targeted. 'Random number generator' means random. 'All records from one week' means block.

4. Watch for independence traps. If a scenario has the auditee choosing the sample or providing a filtered list, the correct answer usually says the auditor should select the sample and verify that the population is complete.

5. Understand sampling risk. Questions may ask why audit conclusions carry uncertainty. Answer: audit evidence is based on samples, so conclusions carry sampling risk. Report users should be aware of this limitation.

6. Expand the sample when deviations appear. In scenario questions where an auditor finds one deviation, a strong answer often includes expanding the sample to determine whether the problem is systemic. Then classify the nonconformity accordingly (major if systemic or a total breakdown, minor if isolated).

7. Multi-site sampling rules. For ISO/IEC 27006 questions, remember the following. The central function is always audited. The sample includes both random and risk-based selection. Conditions include a single centrally controlled ISMS, internal audits covering all sites, and similar activities across sites. Site sampling is not allowed if these conditions are not met.

8. Coverage across the certification cycle. Not all Annex A controls must be tested in every surveillance audit. Over the three-year cycle, all applicable controls in the Statement of Applicability should be covered.

9. Link to audit principles. Tie sampling to the evidence-based approach and independence principles of ISO 19011. Examiners reward answers that connect techniques to principles.

10. Justify sample size. If asked how big a sample should be, do not give a magic number unless the question supplies statistical parameters. Explain that size depends on risk, population size and variability, required confidence, tolerable error, and time available.

11. Mention documentation. In essay or scenario answers, state that the sampling plan, selection rationale, items examined and results should be recorded in audit working papers. This supports traceability and reproducibility.

12. Use structured answers. For long-form questions, a reliable structure is: Define sampling, then name the method, justify why it suits the scenario, describe how you would select items, explain how you would handle deviations, and finish with the limitations and reporting.

13. Eliminate extreme options in multiple choice. Answers that claim sampling gives 'absolute assurance' or that 'every record must be checked' are almost always wrong. Auditing gives reasonable, not absolute, assurance.

14. Statistical needs homogeneity. If a scenario has a highly diverse or poorly defined population, judgement-based or stratified sampling is often the better answer than simple random statistical sampling.

15. Practise scenario reasoning. Examiners test application more than recall. Practise reading a short case and deciding which sampling approach to use, how many items to choose, and how to treat any findings.

Quick Revision Checklist
- Definition of population, sample, sampling unit and sampling risk
- Judgement-based vs statistical sampling (ISO 19011 A.6)
- Attribute vs variable statistical sampling
- Random, systematic, stratified, cluster, block, haphazard and targeted selection
- The six-step sampling process
- Multi-site sampling conditions (ISO/IEC 27006)
- Never let the auditee choose the sample
- Expand the sample on finding deviations
- Document and report sampling limitations

Conclusion
Audit sampling turns an impossible task, examining everything, into a manageable, risk-focused and defensible process. A competent ISO/IEC 27001 Lead Auditor chooses the right sampling method for the context, protects the independence and integrity of the sample, and interprets results carefully. The auditor also makes clear to the audit client the inherent limitations of conclusions based on samples. Master the definitions, the two families, the selection techniques and the scenario reasoning above, and you will be well prepared for any exam question on audit sampling methods.