Likelihood and Impact Assessment for AI Risk
5 minutes
5 Questions
Likelihood and Impact Assessment is the analytical core of AI Risk Program Management, translating identified AI risks into prioritized, actionable insights. It evaluates two dimensions for each risk scenario: how probable it is that a harmful event will occur, and how severe the consequences would be if it did. Likelihood assessment in AI contexts is uniquely challenging because many risks are emergent, non-deterministic, and shaped by data drift, adversarial behavior, and model opacity. Practitioners estimate probability using historical incident data, red-teaming results, model evaluation metrics, threat intelligence, and expert elicitation. Factors raising likelihood include high model complexity, unmonitored deployment, reliance on third-party foundation models, exposure to adversarial inputs, and weak governance. Likelihood is typically scored on an ordinal scale (e.g., rare to almost certain) or expressed probabilistically where sufficient evidence exists. Impact assessment considers the magnitude of harm across multiple dimensions: safety (physical or psychological harm), fairness (discriminatory outcomes against protected groups), privacy (data exposure or re-identification), security (model theft, poisoning, prompt injection), financial loss, legal and regulatory exposure (e.g., EU AI Act penalties), reputational damage, and societal effects such as misinformation or erosion of trust. Advanced programs weigh reversibility, scale of affected populations, and duration of harm, recognizing that AI failures can propagate rapidly and systemically. The two dimensions are combined, often via a risk matrix or heat map, to produce an inherent risk rating. Controls such as human oversight, monitoring, bias testing, and access restrictions are then evaluated to derive residual risk. This supports risk-based decisions on acceptance, mitigation, transfer, or avoidance, and aligns with frameworks like NIST AI RMF (Map, Measure, Manage) and ISO/IEC 23894. Because AI systems evolve post-deployment, likelihood and impact assessments must be iterative, revisited at retraining, scope changes, or incident occurrence, ensuring the risk program remains dynamic rather than a one-time compliance exercise.
Likelihood and Impact Assessment for AI Risk (ISACA AAIR – AI Risk Program Management)
Why Likelihood and Impact Assessment for AI Risk Matters
Every AI risk program eventually faces the same question: of all the things that could go wrong with our AI systems, which ones deserve attention, budget, and controls first? Likelihood and impact assessment is the discipline that answers that question. It converts a long, unprioritized list of identified AI risks (bias, hallucination, data poisoning, model drift, privacy leakage, regulatory non-compliance, loss of human oversight, and more) into a ranked set of exposures that leadership can act on.
For the ISACA Advanced in AI Risk (AAIR) certification, this topic sits at the heart of the AI Risk Program Management domain. It links risk identification (what could happen) to risk response (what we will do about it). Without a sound assessment, organizations either over-control low-risk systems, wasting resources and slowing innovation, or under-control high-risk systems, exposing themselves to regulatory penalties, reputational damage, safety incidents, and loss of stakeholder trust. The EU AI Act, NIST AI RMF, ISO/IEC 23894 and ISO/IEC 42001 all expect a structured, repeatable evaluation of likelihood and impact, so the topic is also tightly connected to compliance.
What It Is
Likelihood and impact assessment is the risk analysis step in the AI risk management lifecycle. It estimates two dimensions for each identified risk scenario:
• Likelihood (probability): How probable is it that the risk event will occur within a defined timeframe? For AI, likelihood is driven by factors such as model complexity, data quality, the degree of autonomy, exposure to adversarial actors, frequency of use, maturity of controls, and the rate of change in the operating environment.
• Impact (consequence/severity): If the event occurs, how bad will it be? AI impact is multidimensional and should be assessed across financial loss, legal and regulatory exposure, reputational harm, operational disruption, safety and physical harm, harm to individuals (discrimination, privacy violation, loss of autonomy), and broader societal or environmental effects.
Combining the two produces a risk rating or risk level, usually expressed as Likelihood × Impact, and typically visualized on a heat map or risk matrix. The output is inherent risk (before controls), and after evaluating existing controls, residual risk (what remains). Residual risk is compared to the organization's risk appetite and risk tolerance to decide whether treatment is needed.
Key concepts you must be able to distinguish:
• Risk scenario: A structured description of a threat, a vulnerability, an asset (the AI system or its data), and a consequence. Scenarios make likelihood and impact estimation concrete.
• Inherent vs. residual risk: Inherent is pre-control; residual is post-control. Exams frequently test which one is compared to risk appetite (residual).
• Risk appetite vs. risk tolerance: Appetite is the broad amount of risk the enterprise is willing to accept in pursuit of objectives; tolerance is the acceptable variation around specific metrics or objectives.
• Qualitative vs. quantitative assessment: Qualitative uses ordinal scales (Low/Medium/High); quantitative uses numeric estimates (probability percentages, monetary loss, FAIR-style analysis). Semi-quantitative assigns numbers to qualitative categories.
• Velocity and persistence: How fast an AI risk materializes once triggered (e.g., a viral hallucinated statement) and how long its effects last. Increasingly included alongside likelihood and impact.
How It Works – The Assessment Process
1. Establish context and criteria. Before scoring anything, define the scales. A typical likelihood scale might be 1 = Rare (less than once in 5 years), 2 = Unlikely, 3 = Possible, 4 = Likely, 5 = Almost Certain (expected multiple times per year). Impact scales define thresholds per dimension (e.g., financial: Level 5 = loss greater than $10M; regulatory: Level 5 = license revocation or enforcement action; harm to individuals: Level 5 = severe or irreversible harm to many people). Clear, documented criteria ensure consistency across assessors and over time.
2. Identify and document AI risk scenarios. Use the AI system inventory, use-case classification, threat modeling (including AI-specific threats such as prompt injection, data poisoning, model inversion, and membership inference), and stakeholder input to build scenarios. Each scenario should name the AI asset, the vulnerability, the threat source, and the potential consequence.
3. Estimate likelihood. Consider:
• Historical incident data and industry benchmarks.
• Exposure factors: number of users, external-facing vs. internal, volume of decisions, use of third-party or open-source models.
• Technical factors: data drift rates, model opacity, lack of testing, training data provenance.
• Human factors: user skill, overreliance (automation bias), weak oversight.
• Threat landscape: attractiveness to attackers, known exploits against similar models.
Be alert that AI likelihood can change rapidly; a model that performed well at deployment may drift, and generative models can produce novel failure modes.
4. Estimate impact. Assess each relevant dimension and usually take the highest dimension as the overall impact (worst-case rule), unless the methodology specifies averaging. For AI, consider:
• Scale: how many individuals or decisions are affected.
• Reversibility: can the harm be undone?
• Vulnerability of affected groups (children, patients, protected classes).
• Criticality of the decision (credit, employment, healthcare, law enforcement, safety-critical systems).
• Cascading effects when AI outputs feed other systems.
Regulatory classifications (e.g., EU AI Act high-risk categories) are strong signals of high impact.
5. Calculate and classify the risk level. Plot the scenario on the matrix. Define which cells are Low, Medium, High, Critical and what each level requires (e.g., Critical requires executive escalation and treatment plan within 30 days).
6. Evaluate controls and determine residual risk. Identify existing controls (bias testing, human-in-the-loop review, input filtering, monitoring, access controls, model cards, kill switches). Assess their design and operating effectiveness, then re-score likelihood and/or impact to arrive at residual risk.
7. Compare to risk appetite and decide response. If residual risk exceeds appetite or tolerance, choose a response: mitigate (add controls), avoid (do not deploy or decommission), transfer/share (insurance, contractual terms with vendors), or accept (with documented approval by the appropriate risk owner).
8. Document, communicate, and monitor. Record assumptions, data sources, assessor identities, and dates in the risk register. Report to governance bodies. Set Key Risk Indicators (KRIs) such as drift metrics, fairness disparity thresholds, hallucination rates, or incident counts, and trigger reassessment when they breach thresholds or when the system, data, or regulation changes.
AI-Specific Challenges in Likelihood and Impact Assessment
• Lack of historical data: Novel AI failure modes have little incident history, so likelihood estimates rely heavily on expert judgment and scenario analysis; document the uncertainty.
• Opacity: Black-box models make it hard to predict when and how they will fail, raising both likelihood uncertainty and impact severity.
• Dynamic behavior: Continuous learning and drift mean a static assessment quickly becomes stale; reassessment must be periodic and event-driven.
• Diffuse and societal harms: Impacts such as discrimination, misinformation, or erosion of trust are hard to monetize but must still be scored; ethical and human-rights impact criteria address this.
• Third-party dependency: When using foundation models or vendor AI, likelihood depends on the provider's practices, requiring vendor due diligence and shared-responsibility analysis.
• Cognitive biases: Optimism bias, anchoring, and groupthink can distort scores; use calibrated scales, diverse assessment teams, and independent review.
Roles and Accountability
The risk owner (typically the business owner of the AI use case) is accountable for the assessment and for accepting residual risk within their authority. Data scientists and engineers provide technical likelihood input. Legal, compliance, privacy, and ethics functions inform impact ratings. The AI risk or second-line function owns the methodology and challenges results. Internal audit independently assures the process. Senior management and the board set risk appetite and receive aggregated reporting.
Exam Tips: Answering Questions on Likelihood and Impact Assessment for AI Risk
• Know the formula and the sequence. Risk = Likelihood × Impact. The order is identify → analyze (likelihood and impact) → evaluate (compare to criteria/appetite) → respond → monitor. If a question asks what comes first before scoring, the answer usually involves defining criteria/scales or establishing context, not jumping to controls.
• Residual risk is compared to risk appetite. When a question asks what determines whether further treatment is needed, choose the option referencing residual risk versus appetite or tolerance, not inherent risk.
• Watch for the 'highest dimension' rule. If a scenario has low financial impact but severe harm to individuals or regulatory exposure, the overall impact is high. Exams test whether you recognize that non-financial harm drives AI impact.
• Distinguish likelihood drivers from impact drivers. Factors like exposure, frequency of use, control weakness, and attacker interest change likelihood. Factors like number of people affected, reversibility, and decision criticality change impact. Misclassifying these is a common trap.
• Qualitative vs. quantitative. If the question emphasizes limited data, speed, or communicating to non-technical stakeholders, qualitative is appropriate. If it emphasizes cost-benefit justification, insurance, or precise comparison, quantitative is better. 'Semi-quantitative' is the usual answer for balancing the two.
• 'MOST' and 'BEST' questions favor governance-level answers. ISACA typically prefers the answer that aligns with organizational objectives, involves the risk owner, uses documented criteria, or ensures consistency and repeatability over a purely technical fix.
• Reassessment triggers. Expect questions on when to reassess: material model changes, retraining on new data, drift or KRI breaches, new regulation, incidents, or change in use context. 'Only annually' is rarely the best answer for AI.
• Controls reduce likelihood or impact, not both automatically. Monitoring and human oversight often reduce impact (early detection, intervention); input validation, access control, and robust training reduce likelihood. Be ready to say which.
• Accountability questions. The business/risk owner accepts residual risk; the risk function facilitates and challenges; the board sets appetite. Do not pick 'data scientist accepts the risk.'
• Read the scenario for regulatory cues. Mentions of hiring, credit, healthcare, biometric identification, or critical infrastructure signal high impact under frameworks like the EU AI Act, pushing the rating up regardless of likelihood.
• Eliminate absolutes. Options stating that AI risk can be fully eliminated, that likelihood can always be precisely quantified, or that inherent risk is what should be reported to the board are usually distractors.
• Document uncertainty. If asked how to handle a risk with no historical data, the best answer typically involves expert judgment, scenario analysis, documenting assumptions, and planning to refine estimates, rather than excluding the risk or assigning it an arbitrary low score.
Summary
Likelihood and impact assessment turns AI risk identification into actionable prioritization. Master the scales, the Likelihood × Impact logic, the inherent-to-residual journey, the comparison to risk appetite, the multidimensional nature of AI impact, and the AI-specific factors that make likelihood volatile. In the exam, think like a risk program manager: consistent criteria, documented judgment, business ownership, and continuous reassessment are the themes that lead to the correct answer.
Every AI risk program eventually faces the same question: of all the things that could go wrong with our AI systems, which ones deserve attention, budget, and controls first? Likelihood and impact assessment is the discipline that answers that question. It converts a long, unprioritized list of identified AI risks (bias, hallucination, data poisoning, model drift, privacy leakage, regulatory non-compliance, loss of human oversight, and more) into a ranked set of exposures that leadership can act on.
For the ISACA Advanced in AI Risk (AAIR) certification, this topic sits at the heart of the AI Risk Program Management domain. It links risk identification (what could happen) to risk response (what we will do about it). Without a sound assessment, organizations either over-control low-risk systems, wasting resources and slowing innovation, or under-control high-risk systems, exposing themselves to regulatory penalties, reputational damage, safety incidents, and loss of stakeholder trust. The EU AI Act, NIST AI RMF, ISO/IEC 23894 and ISO/IEC 42001 all expect a structured, repeatable evaluation of likelihood and impact, so the topic is also tightly connected to compliance.
What It Is
Likelihood and impact assessment is the risk analysis step in the AI risk management lifecycle. It estimates two dimensions for each identified risk scenario:
• Likelihood (probability): How probable is it that the risk event will occur within a defined timeframe? For AI, likelihood is driven by factors such as model complexity, data quality, the degree of autonomy, exposure to adversarial actors, frequency of use, maturity of controls, and the rate of change in the operating environment.
• Impact (consequence/severity): If the event occurs, how bad will it be? AI impact is multidimensional and should be assessed across financial loss, legal and regulatory exposure, reputational harm, operational disruption, safety and physical harm, harm to individuals (discrimination, privacy violation, loss of autonomy), and broader societal or environmental effects.
Combining the two produces a risk rating or risk level, usually expressed as Likelihood × Impact, and typically visualized on a heat map or risk matrix. The output is inherent risk (before controls), and after evaluating existing controls, residual risk (what remains). Residual risk is compared to the organization's risk appetite and risk tolerance to decide whether treatment is needed.
Key concepts you must be able to distinguish:
• Risk scenario: A structured description of a threat, a vulnerability, an asset (the AI system or its data), and a consequence. Scenarios make likelihood and impact estimation concrete.
• Inherent vs. residual risk: Inherent is pre-control; residual is post-control. Exams frequently test which one is compared to risk appetite (residual).
• Risk appetite vs. risk tolerance: Appetite is the broad amount of risk the enterprise is willing to accept in pursuit of objectives; tolerance is the acceptable variation around specific metrics or objectives.
• Qualitative vs. quantitative assessment: Qualitative uses ordinal scales (Low/Medium/High); quantitative uses numeric estimates (probability percentages, monetary loss, FAIR-style analysis). Semi-quantitative assigns numbers to qualitative categories.
• Velocity and persistence: How fast an AI risk materializes once triggered (e.g., a viral hallucinated statement) and how long its effects last. Increasingly included alongside likelihood and impact.
How It Works – The Assessment Process
1. Establish context and criteria. Before scoring anything, define the scales. A typical likelihood scale might be 1 = Rare (less than once in 5 years), 2 = Unlikely, 3 = Possible, 4 = Likely, 5 = Almost Certain (expected multiple times per year). Impact scales define thresholds per dimension (e.g., financial: Level 5 = loss greater than $10M; regulatory: Level 5 = license revocation or enforcement action; harm to individuals: Level 5 = severe or irreversible harm to many people). Clear, documented criteria ensure consistency across assessors and over time.
2. Identify and document AI risk scenarios. Use the AI system inventory, use-case classification, threat modeling (including AI-specific threats such as prompt injection, data poisoning, model inversion, and membership inference), and stakeholder input to build scenarios. Each scenario should name the AI asset, the vulnerability, the threat source, and the potential consequence.
3. Estimate likelihood. Consider:
• Historical incident data and industry benchmarks.
• Exposure factors: number of users, external-facing vs. internal, volume of decisions, use of third-party or open-source models.
• Technical factors: data drift rates, model opacity, lack of testing, training data provenance.
• Human factors: user skill, overreliance (automation bias), weak oversight.
• Threat landscape: attractiveness to attackers, known exploits against similar models.
Be alert that AI likelihood can change rapidly; a model that performed well at deployment may drift, and generative models can produce novel failure modes.
4. Estimate impact. Assess each relevant dimension and usually take the highest dimension as the overall impact (worst-case rule), unless the methodology specifies averaging. For AI, consider:
• Scale: how many individuals or decisions are affected.
• Reversibility: can the harm be undone?
• Vulnerability of affected groups (children, patients, protected classes).
• Criticality of the decision (credit, employment, healthcare, law enforcement, safety-critical systems).
• Cascading effects when AI outputs feed other systems.
Regulatory classifications (e.g., EU AI Act high-risk categories) are strong signals of high impact.
5. Calculate and classify the risk level. Plot the scenario on the matrix. Define which cells are Low, Medium, High, Critical and what each level requires (e.g., Critical requires executive escalation and treatment plan within 30 days).
6. Evaluate controls and determine residual risk. Identify existing controls (bias testing, human-in-the-loop review, input filtering, monitoring, access controls, model cards, kill switches). Assess their design and operating effectiveness, then re-score likelihood and/or impact to arrive at residual risk.
7. Compare to risk appetite and decide response. If residual risk exceeds appetite or tolerance, choose a response: mitigate (add controls), avoid (do not deploy or decommission), transfer/share (insurance, contractual terms with vendors), or accept (with documented approval by the appropriate risk owner).
8. Document, communicate, and monitor. Record assumptions, data sources, assessor identities, and dates in the risk register. Report to governance bodies. Set Key Risk Indicators (KRIs) such as drift metrics, fairness disparity thresholds, hallucination rates, or incident counts, and trigger reassessment when they breach thresholds or when the system, data, or regulation changes.
AI-Specific Challenges in Likelihood and Impact Assessment
• Lack of historical data: Novel AI failure modes have little incident history, so likelihood estimates rely heavily on expert judgment and scenario analysis; document the uncertainty.
• Opacity: Black-box models make it hard to predict when and how they will fail, raising both likelihood uncertainty and impact severity.
• Dynamic behavior: Continuous learning and drift mean a static assessment quickly becomes stale; reassessment must be periodic and event-driven.
• Diffuse and societal harms: Impacts such as discrimination, misinformation, or erosion of trust are hard to monetize but must still be scored; ethical and human-rights impact criteria address this.
• Third-party dependency: When using foundation models or vendor AI, likelihood depends on the provider's practices, requiring vendor due diligence and shared-responsibility analysis.
• Cognitive biases: Optimism bias, anchoring, and groupthink can distort scores; use calibrated scales, diverse assessment teams, and independent review.
Roles and Accountability
The risk owner (typically the business owner of the AI use case) is accountable for the assessment and for accepting residual risk within their authority. Data scientists and engineers provide technical likelihood input. Legal, compliance, privacy, and ethics functions inform impact ratings. The AI risk or second-line function owns the methodology and challenges results. Internal audit independently assures the process. Senior management and the board set risk appetite and receive aggregated reporting.
Exam Tips: Answering Questions on Likelihood and Impact Assessment for AI Risk
• Know the formula and the sequence. Risk = Likelihood × Impact. The order is identify → analyze (likelihood and impact) → evaluate (compare to criteria/appetite) → respond → monitor. If a question asks what comes first before scoring, the answer usually involves defining criteria/scales or establishing context, not jumping to controls.
• Residual risk is compared to risk appetite. When a question asks what determines whether further treatment is needed, choose the option referencing residual risk versus appetite or tolerance, not inherent risk.
• Watch for the 'highest dimension' rule. If a scenario has low financial impact but severe harm to individuals or regulatory exposure, the overall impact is high. Exams test whether you recognize that non-financial harm drives AI impact.
• Distinguish likelihood drivers from impact drivers. Factors like exposure, frequency of use, control weakness, and attacker interest change likelihood. Factors like number of people affected, reversibility, and decision criticality change impact. Misclassifying these is a common trap.
• Qualitative vs. quantitative. If the question emphasizes limited data, speed, or communicating to non-technical stakeholders, qualitative is appropriate. If it emphasizes cost-benefit justification, insurance, or precise comparison, quantitative is better. 'Semi-quantitative' is the usual answer for balancing the two.
• 'MOST' and 'BEST' questions favor governance-level answers. ISACA typically prefers the answer that aligns with organizational objectives, involves the risk owner, uses documented criteria, or ensures consistency and repeatability over a purely technical fix.
• Reassessment triggers. Expect questions on when to reassess: material model changes, retraining on new data, drift or KRI breaches, new regulation, incidents, or change in use context. 'Only annually' is rarely the best answer for AI.
• Controls reduce likelihood or impact, not both automatically. Monitoring and human oversight often reduce impact (early detection, intervention); input validation, access control, and robust training reduce likelihood. Be ready to say which.
• Accountability questions. The business/risk owner accepts residual risk; the risk function facilitates and challenges; the board sets appetite. Do not pick 'data scientist accepts the risk.'
• Read the scenario for regulatory cues. Mentions of hiring, credit, healthcare, biometric identification, or critical infrastructure signal high impact under frameworks like the EU AI Act, pushing the rating up regardless of likelihood.
• Eliminate absolutes. Options stating that AI risk can be fully eliminated, that likelihood can always be precisely quantified, or that inherent risk is what should be reported to the board are usually distractors.
• Document uncertainty. If asked how to handle a risk with no historical data, the best answer typically involves expert judgment, scenario analysis, documenting assumptions, and planning to refine estimates, rather than excluding the risk or assigning it an arbitrary low score.
Summary
Likelihood and impact assessment turns AI risk identification into actionable prioritization. Master the scales, the Likelihood × Impact logic, the inherent-to-residual journey, the comparison to risk appetite, the multidimensional nature of AI impact, and the AI-specific factors that make likelihood volatile. In the exam, think like a risk program manager: consistent criteria, documented judgment, business ownership, and continuous reassessment are the themes that lead to the correct answer.
Test mode:
Unlock Premium Access
Advanced in AI Risk
- Access to ALL Certifications: Study for any certification on our platform with one subscription
- 3020 Superior-grade Advanced in AI Risk practice questions
- Unlimited practice tests across all certifications
- Detailed explanations for every question
- AAIR: 5 full exams plus all other certification exams
- 30-Day Money-Back Guarantee: Full refund within 30 days if unsatisfied
- Risk-Free: 7-day free trial with all premium features!
More Likelihood and Impact Assessment for AI Risk questions
20 questions (total)
Other AI Risk Program Management concepts
Access and Identity Controls for AI SystemsAdversarial Attacks: Evasion and Data PoisoningAggregating AI Risk into Enterprise MetricsAI Bill of Materials and Component TransparencyAI Contract Clauses: Data Usage, IP and LiabilityAI Control Gaps and RemediationAI Control Objectives and FrameworksAI Control Selection and DesignAI Control Testing and Operating EffectivenessAI Control Validation and Independent AssuranceAI Incident Communication and NotificationAI Incident Definition and ClassificationAI Incident Root Cause and Lessons LearnedAI Metric Thresholds, Triggers and EscalationAI Risk Appetite and ToleranceAI Risk Classification and TieringAI Risk Identification Techniques and TaxonomiesAI Risk Profile and DashboardsAI Risk Reporting to Board and ManagementAI Risk Response OptionsAI Risk Scenario DevelopmentAI Risk Transfer: Insurance and ContractsAI Risk Treatment Plans and Action TrackingAI Service Levels and Right to AuditAI Threat ModelingAI Use Case Restriction and AvoidanceAI Vendor Exit Strategy and Lock-InAutomation Bias and Over-RelianceBusiness Continuity Planning for AI DependenciesBusiness Impact Analysis for AI-Dependent ProcessesCompensating Controls and GenAI GuardrailsContainment: Kill Switches and Model RollbackContinuous Monitoring of AI SystemsCost-Benefit Analysis of AI Risk TreatmentsDetecting and Triaging AI IncidentsDisaster Recovery for Models, Data and PipelinesEmbedded AI in SaaS and Vendor Feature ChangesEmerging AI Risk and Horizon ScanningFoundation Model and API Provider RiskFourth-Party and Concentration RiskHallucination and Output Reliability RiskInput Validation and Output Filtering ControlsIntegrating AI Scenarios into Incident ResponseKey Performance and Control Indicators for AIKey Risk Indicators for AILogging, Traceability and Audit Trails for AIMapping AI Controls to Existing TaxonomiesModel Inversion, Extraction and Membership InferenceModel Performance and Fairness Monitoring MetricsMonitoring Organizational Processes for AI RiskOngoing Vendor Monitoring and AttestationsOpen-Source Models and Components RiskPreventive, Detective and Corrective AI ControlsPrioritizing AI Risk TreatmentsPrompt Injection and JailbreaksResidual Risk and Risk AcceptanceResilience of Third-Party AI ServicesRisk and Control Self-Assessment for AITesting AI Incident and Continuity PlansThird-Party AI Risk Assessment and Due DiligenceUsing AI to Support the Risk ProgramVulnerability Assessment of AI Programs