Artificial Intelligence and Machine Learning Risks

Artificial Intelligence and Machine Learning Risks in ISO 27001 Lead Auditor: A Complete Guide

Introduction
Artificial Intelligence (AI) and Machine Learning (ML) are now used in fraud detection, customer service chatbots, security monitoring (SIEM/UEBA), code generation and decision support. For an ISO/IEC 27001 Lead Auditor, AI/ML is not a separate management system topic. It is a new source of information security risk that must be identified, assessed and treated within the existing Information Security Management System (ISMS). This guide explains why the topic matters, what the risks are, how they map to ISO/IEC 27001:2022, and how to answer exam questions on it.

1. Why AI and ML Risks Are Important
• New attack surfaces: AI systems bring threats that traditional IT controls may not cover. Examples are data poisoning, model inversion, prompt injection and adversarial inputs.
• Large volumes of sensitive data: ML models are trained on big datasets that often contain personal, confidential or proprietary information. This raises confidentiality and privacy concerns.
• Integrity of decisions: If a model is manipulated or biased, the organisation may make wrong decisions automatically and at scale.
• Availability and dependency: Organisations increasingly rely on third-party AI services (cloud AI APIs, Large Language Models). Outages or vendor changes can disrupt operations.
• Shadow AI: Employees may paste confidential data into public generative AI tools without authorisation. This creates data leakage.
• Regulatory pressure: Laws such as the EU AI Act and GDPR (automated decision-making) affect how AI is governed. This links to Clause 4.2 (interested parties) and control 5.31 (legal, statutory, regulatory and contractual requirements).
• Auditor relevance: An auditor must judge whether the organisation's risk assessment (Clause 6.1.2) has considered emerging technologies. An ISMS that ignores widespread AI use may have an incomplete scope or risk assessment.

2. What AI and ML Risks Are
Artificial Intelligence is the broad capability of systems to perform tasks that normally need human intelligence. Machine Learning is a subset of AI in which systems learn patterns from data instead of following explicitly programmed rules.

Key risk categories, mapped to the CIA triad:

Confidentiality risks
• Training data leakage: Models may memorise and reveal sensitive training data.
• Model inversion and membership inference: Attackers query a model to rebuild training data or learn whether a specific record was used.
• Model theft or extraction: Adversaries copy a proprietary model through repeated queries.
• Data shared with external AI providers: Prompts and uploads to public tools may be stored or used for retraining.

Integrity risks
• Data poisoning: Malicious data inserted into training sets corrupts model behaviour.
• Adversarial examples: Carefully crafted inputs cause misclassification, for example fooling malware detection.
• Prompt injection: Malicious instructions embedded in input override an LLM's intended behaviour.
• Hallucinations: Generative AI produces plausible but false outputs.
• Model drift: Accuracy degrades over time as real-world data changes.
• Bias: Unfair or skewed results caused by unrepresentative data.

Availability risks
• Dependency on third-party AI platforms.
• Resource exhaustion attacks on AI APIs.
• Lack of fallback procedures if the AI system fails.

Governance and accountability risks
• Lack of transparency and explainability ('black box' models).
• Unclear ownership of AI assets (models, datasets, pipelines).
• No human oversight of automated decisions.
• Weak change management for model retraining and deployment.

3. How AI/ML Risks Fit Into the ISMS
ISO/IEC 27001:2022 is technology-neutral. It contains no AI-specific clause, but its risk-based framework fully applies. The auditor looks at how AI is handled through the normal PDCA cycle.

Context and scope (Clauses 4.1 to 4.3)
• Internal and external issues should include AI adoption and AI threats.
• Interested parties may include regulators, customers concerned about AI use, and AI vendors.
• The scope should state whether AI systems and data pipelines are included.

Leadership (Clause 5)
• Top management should support policies on acceptable AI use.
• Roles and responsibilities for AI asset owners should be defined (control 5.2).

Risk assessment and treatment (Clauses 6.1.2, 6.1.3, 8.2, 8.3)
• AI-specific threats and vulnerabilities should be identified.
• Risk owners should be assigned.
• Treatment options are to modify, avoid, share or retain the risk.
• The Statement of Applicability (SoA) should justify the controls selected for AI risks.

Relevant Annex A controls (ISO/IEC 27001:2022)
• 5.1 Policies for information security: AI acceptable use policy.
• 5.7 Threat intelligence: Awareness of emerging AI attack techniques.
• 5.9 Inventory of information and other associated assets: Models, training data and AI tools as assets.
• 5.10 Acceptable use of information: Rules on entering data into generative AI.
• 5.12 Classification of information: Classifying training datasets.
• 5.19 to 5.23 Supplier relationships and cloud services: Contracts with AI providers covering data use, retention and retraining.
• 5.31 Legal, statutory, regulatory and contractual requirements: AI regulations.
• 5.34 Privacy and protection of PII: Personal data in training sets.
• 6.3 Information security awareness, education and training: Staff training on AI risks.
• 8.10 Information deletion, 8.11 Data masking and 8.12 Data leakage prevention: Protecting data used by or sent to AI.
• 8.16 Monitoring activities: Monitoring AI behaviour and misuse.
• 8.25 to 8.29 Secure development life cycle and testing: Secure ML pipelines and adversarial testing.
• 8.32 Change management: Controlled model updates and retraining.

Related standards worth knowing
• ISO/IEC 42001: AI Management System (AIMS). It is a separate certifiable standard that can integrate with ISO 27001.
• ISO/IEC 23894: Guidance on AI risk management.
• ISO/IEC 22989: AI concepts and terminology.
• ISO/IEC 27701: Privacy extension, relevant when AI processes PII.

Performance evaluation and improvement (Clauses 9 and 10)
• Monitor AI-related incidents and control effectiveness.
• Cover AI controls in internal audits.
• Address AI-related nonconformities through corrective action.

4. How an Auditor Approaches AI Risks
• Ask: Does the organisation know where AI is used, including shadow AI?
• Verify: Are AI assets in the asset inventory with owners?
• Examine: Does the risk assessment identify AI threats such as data leakage, poisoning and supplier dependency?
• Check: Do supplier agreements with AI vendors address confidentiality, data residency and use of customer data for training?
• Sample: Look for evidence of awareness training, an acceptable use policy and DLP controls.
• Evaluate: Is there human oversight and change control for model updates?
• Remember: The auditor does not need to be an AI engineer. The auditor must determine whether the ISMS processes adequately address the risk, based on objective evidence.

5. Example Scenario
During an audit, staff are observed using a public generative AI tool to summarise customer contracts. The organisation has no AI policy, and the risk register makes no mention of AI.
• Potential finding: Nonconformity against Clause 6.1.2 (the risk assessment did not identify relevant risks), and possibly control 5.10 (acceptable use) if it was included in the SoA.
• Classification: This could be major or minor depending on how widespread and systemic the issue is. A systemic failure to consider a significant risk source leans towards major.
• Auditor action: Record objective evidence, refer to the requirement, and do not prescribe a specific solution.

Exam Tips: Answering Questions on Artificial Intelligence and Machine Learning Risks
• Tip 1 – Think risk-based, not technology-based: ISO 27001 does not mandate AI-specific controls. The correct answer usually says AI must be addressed through the organisation's risk assessment and treatment process (Clauses 6.1.2 and 6.1.3).
• Tip 2 – Map to CIA: When a question describes an AI threat, classify it. Data leakage is confidentiality, poisoning or hallucination is integrity, and service dependency is availability. This helps you eliminate wrong options.
• Tip 3 – Know the attack names: Data poisoning corrupts training data. Adversarial examples manipulate inputs at inference time. Model inversion or extraction steals data or the model. Prompt injection manipulates LLM instructions. Exams often test whether you can tell these apart.
• Tip 4 – Link to Annex A controls: Be ready to name relevant controls. Supplier security (5.19 to 5.23) applies to third-party AI. Acceptable use (5.10), DLP (8.12), asset inventory (5.9), privacy (5.34), secure development (8.25) and awareness (6.3) are the others to recall.
• Tip 5 – Act as an auditor, not a consultant: In scenario questions, the auditor identifies conformity or nonconformity based on evidence. Avoid answers where the auditor designs or implements AI controls, as that threatens impartiality.
• Tip 6 – Distinguish ISO 27001 from ISO 42001: ISO 42001 is the AI management system standard. If asked which standard certifies AI governance specifically, choose 42001. For information security risks of AI within an ISMS, choose 27001.
• Tip 7 – Watch for shadow AI scenarios: Unauthorised staff use of public AI tools typically points to gaps in policy, awareness, acceptable use and risk identification.
• Tip 8 – Supplier questions: For cloud AI services, the key audit concerns are contractual clauses on data use, retention, location, subcontractors and right to audit (5.20, 5.23).
• Tip 9 – Grade findings correctly: A missing single control on one AI tool is often minor. A complete failure to consider AI in the risk assessment when AI is core to operations may be major. Always justify the grade by impact on the ISMS's ability to achieve its intended outcomes.
• Tip 10 – Remember context: AI adoption is an internal or external issue (Clause 4.1). AI regulators and customers are interested parties (Clause 4.2). Questions may test whether you link emerging technology to context analysis.
• Tip 11 – Choose the 'most complete' answer: When several options look correct, pick the one that includes identification, assessment, treatment and monitoring, which reflects the full PDCA approach.
• Tip 12 – Avoid absolute statements: Be wary of options saying 'AI must be banned' or 'AI is exempt from the ISMS.' ISO 27001 favours proportionate, risk-based decisions.

Quick Revision Summary
• AI/ML is a risk source within the ISMS, not a separate requirement of ISO 27001.
• The key threats are poisoning, adversarial inputs, model inversion or extraction, prompt injection, data leakage, bias, drift, hallucinations and vendor dependency.
• The key clauses are 4.1, 4.2, 6.1.2, 6.1.3, 8.2, 8.3, 9.2 and 10.2.
• The key controls are 5.9, 5.10, 5.12, 5.19 to 5.23, 5.31, 5.34, 6.3, 8.11, 8.12, 8.16, 8.25 to 8.29 and 8.32.
• The related standards are ISO/IEC 42001, 23894 and 22989.
• The auditor's role is to gather evidence, assess conformity and stay impartial.