AI Impact Assessments and Regulatory Conformity

1. Why This Topic Matters

AI systems do not fail only in technical ways. They can discriminate, expose personal data, make opaque decisions that affect people's rights, and create security exposures that traditional IT risk processes never anticipated. Regulators have responded with binding frameworks (most notably the EU AI Act) and with widely adopted voluntary standards (NIST AI RMF, ISO/IEC 42001, ISO/IEC 23894). For an AI security manager, the AI Impact Assessment (AIIA) is the mechanism that turns abstract principles such as fairness, transparency, safety and privacy into concrete, documented risk decisions, and regulatory conformity is the proof that the organization can demonstrate those decisions to regulators, auditors, customers and courts.

In the ISACA AAISM (Advanced in AI Security Management) body of knowledge, this topic sits inside the domain AI Risk Management for Security. ISACA expects candidates to understand that impact assessments are a risk identification and treatment tool, that conformity is a governance and assurance outcome, and that security management is the bridge between the two. Exam questions in this area test judgment: when an assessment is required, who owns it, what it must contain, how it links to controls, and how to prove conformity to a regulator or auditor.

2. What an AI Impact Assessment Is

An AI Impact Assessment is a structured, documented evaluation of the potential positive and negative effects an AI system may have on individuals, groups, the organization and society, performed before deployment and repeated throughout the lifecycle. It is broader than a classic IT risk assessment because it looks outward at harms to people, not only inward at harms to the enterprise.

Related and overlapping assessment types you should recognize:
• DPIA (Data Protection Impact Assessment) – required under GDPR Article 35 for high-risk processing of personal data; focuses on privacy and data protection.
• FRIA (Fundamental Rights Impact Assessment) – required under the EU AI Act (Article 27) for certain deployers of high-risk AI systems (e.g., public bodies, credit scoring, insurance pricing); focuses on rights such as non-discrimination, dignity and due process.
• Algorithmic Impact Assessment (AIA) – e.g., the Canadian Treasury Board model; scores automated decision systems by impact level, which then dictates required controls.
• Human Rights Impact Assessment (HRIA) – broad societal and rights lens, often used by large platforms.
• Security Risk Assessment – threat and vulnerability analysis (adversarial attacks, model theft, data poisoning, prompt injection). In AAISM thinking, the AIIA should incorporate or reference the security risk assessment rather than run in isolation.

Typical contents of a well-formed AIIA:
• System description, intended purpose, and reasonably foreseeable misuse.
• Data provenance, lineage, quality, representativeness, and lawful basis.
• Affected stakeholders and vulnerable groups.
• Identified harms (bias, privacy, safety, security, autonomy, environmental, economic) with likelihood and severity.
• Risk classification against applicable law (e.g., EU AI Act prohibited / high-risk / limited-risk / minimal-risk).
• Existing and planned controls, including human oversight, logging, explainability, robustness testing and security hardening.
• Residual risk, acceptance decision and approver.
• Monitoring plan, review triggers and sunset criteria.

3. What Regulatory Conformity Is

Regulatory conformity means an AI system and its governing processes meet the obligations of applicable laws, regulations and standards, and that the organization can evidence this. Key regimes to know:

• EU AI Act – risk-based tiers. Unacceptable risk systems are prohibited (social scoring, manipulative techniques, certain biometric categorization). High-risk systems (Annex III areas such as employment, education, critical infrastructure, law enforcement, essential services) must pass a conformity assessment before being placed on the market, carry CE marking, be registered in the EU database, and maintain a risk management system, data governance, technical documentation, record-keeping (logs), transparency to deployers, human oversight, and accuracy/robustness/cybersecurity. Limited-risk systems carry transparency duties (e.g., disclosing chatbots and deepfakes). General-purpose AI (GPAI) models have separate documentation and, for systemic-risk models, evaluation and incident reporting duties. Conformity assessment is usually self-assessment (internal control) but requires a notified body (third party) for certain categories, notably biometric systems where harmonized standards are not fully applied.
• NIST AI RMF 1.0 – voluntary U.S. framework with four functions: Govern, Map, Measure, Manage. Impact assessment activity lives mostly in Map (context, stakeholders, impacts) and Measure (metrics for trustworthiness characteristics: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, fair with harmful bias managed).
• ISO/IEC 42001 – certifiable AI management system standard (AIMS). Requires an AI risk assessment and an AI system impact assessment as distinct processes (clauses 6.1.2 and 6.1.4; Annex A controls A.5). Conformity is demonstrated by certification audit.
• ISO/IEC 23894 – guidance on AI risk management, aligned to ISO 31000.
• ISO/IEC 42005 – guidance specifically on AI system impact assessment.
• GDPR – DPIA, automated decision-making rights (Article 22), data minimization.
• Sector and regional laws – e.g., Colorado AI Act (algorithmic discrimination in consequential decisions), NYC Local Law 144 (bias audits for automated employment tools), U.S. EEOC guidance, financial-services model risk guidance (SR 11-7), medical device rules for AI as a medical device.

The AAISM perspective is that conformity is not a one-time certificate; it is a continuous state maintained through change management, post-market monitoring, incident reporting and re-assessment.

4. How It Works – The Assessment and Conformity Lifecycle

Step 1 – Inventory and triage. Maintain an AI system inventory. Apply a screening questionnaire to determine whether a full AIIA is needed and what regulatory classification applies. Triggers include use of personal or sensitive data, decisions with legal or similarly significant effects, safety-critical functions, public-sector use, and inclusion in a regulated high-risk list.

Step 2 – Scoping and stakeholder identification. Define intended purpose, deployment context, users, affected persons, and third parties (model providers, data suppliers). Identify vulnerable or marginalized groups.

Step 3 – Harm and risk identification. Use structured techniques: harm taxonomies, threat modeling (including adversarial ML threats such as evasion, poisoning, model inversion, prompt injection, supply-chain compromise), scenario analysis, and consultation with affected communities where required.

Step 4 – Analysis and evaluation. Rate likelihood and severity; consider scale, reversibility and whether harm falls on individuals or society. Compare against risk appetite and legal thresholds.

Step 5 – Treatment and control mapping. Select controls: data governance and bias testing, model robustness and red-teaming, access control and model protection, logging and traceability, explainability tooling, human-in-the-loop or human-on-the-loop oversight, kill switches, user transparency notices. Map each control to the regulatory requirement it satisfies (a requirements traceability matrix).

Step 6 – Documentation and approval. Produce the technical documentation and the assessment report. Obtain sign-off from the accountable owner and, where required, the Data Protection Officer or AI governance committee. Register the system where law requires.

Step 7 – Conformity assessment. Internal self-assessment against the applicable requirements, or engagement of a notified body or external auditor. Issue a declaration of conformity and apply CE marking where applicable. For ISO/IEC 42001, undergo certification audit.

Step 8 – Monitoring and re-assessment. Post-market monitoring, drift and performance metrics, incident and serious-incident reporting to authorities within mandated timelines, periodic review, and mandatory re-assessment on substantial modification (change of purpose, retraining on new data, new deployment context).

Roles you should keep straight: the provider (develops and places on market) carries primary conformity obligations; the deployer (uses the system) carries use-related obligations such as human oversight, FRIA where applicable, and monitoring; importers and distributors verify documentation. Security managers typically support both provider and deployer roles by supplying threat assessments, control evidence and incident data.

5. The Security Management Lens

ISACA expects you to connect impact assessment to security practice:
• Cybersecurity is an explicit high-risk requirement in the EU AI Act (Article 15: accuracy, robustness and cybersecurity). Your adversarial-ML threat assessment is therefore conformity evidence, not just good practice.
• Logging and record-keeping requirements double as forensic and detection capabilities.
• Data governance requirements overlap with data classification, integrity controls and supply-chain assurance.
• Human oversight requirements must be designed so that overrides cannot be bypassed by attackers or automation bias.
• Residual risk acceptance must be made by an accountable business owner, not by the security team alone.
• Third-party and foundation-model dependencies require contractual flow-down of documentation, transparency and incident notification obligations.

6. How to Answer Exam Questions on This Topic

AAISM questions are scenario based and ask for the best or most appropriate action. Apply these reasoning rules:

Rule 1 – Timing: before, not after. Impact assessments are performed before deployment and repeated on material change. If an option says to deploy first and assess later, it is almost always wrong.

Rule 2 – Classification drives obligation. When a scenario describes an AI use case, first silently classify it (prohibited, high-risk, limited, minimal; or personal data vs not). The correct answer usually follows from the classification. Hiring, credit, education, law enforcement, critical infrastructure and biometrics are high-risk signals.

Rule 3 – People impact over enterprise impact. If the question is about an impact assessment, the right answer emphasizes effects on individuals and society (rights, fairness, safety), not only financial or reputational loss to the company.

Rule 4 – Governance before technology. When choices include a policy, process or accountability step versus a purely technical fix, ISACA generally favors establishing the governance step first (define scope, assign ownership, classify, document), then selecting controls.

Rule 5 – Evidence and traceability. Conformity questions reward answers involving documentation, traceability of requirements to controls, logs, declarations of conformity and audit trails. An untested claim of compliance is never the best answer.

Rule 6 – Right role, right accountability. Risk acceptance belongs to the business owner or accountable executive; the security manager advises and implements. Legal interpretation involves counsel or the DPO. Independent assurance comes from internal audit or external bodies, not the development team.

Rule 7 – Continuous, not point-in-time. Answers that build in monitoring, review triggers and re-assessment beat answers that treat conformity as a one-off certificate.

Rule 8 – Watch for absolutes and distractors. Options containing words like 'always', 'never', 'eliminates risk', 'guarantees compliance' or 'no further action required' are usually distractors. Also beware answers that satisfy only one regulation when the scenario clearly spans several (e.g., both GDPR and the AI Act).

7. Exam Tips: Answering Questions on AI Impact Assessments and Regulatory Conformity

• Know the vocabulary cold. AIIA vs DPIA vs FRIA vs conformity assessment vs certification. If a question says 'fundamental rights' think FRIA and the EU AI Act deployer duty; if it says 'personal data processing likely to result in high risk' think DPIA and GDPR Article 35.
• Memorize the EU AI Act risk tiers and at least five Annex III high-risk categories, the core high-risk obligations (risk management system, data governance, technical documentation, logging, transparency, human oversight, accuracy/robustness/cybersecurity, quality management system), and the fact that most conformity assessments are internal self-assessments, with notified bodies required mainly for biometric systems or when harmonized standards are not applied.
• Memorize the NIST AI RMF functions (Govern, Map, Measure, Manage) and the seven trustworthiness characteristics. Questions may describe an activity and ask which function it belongs to; stakeholder and context identification is Map, metrics and testing is Measure, treatment and monitoring is Manage, policy and roles is Govern.
• Distinguish ISO/IEC 42001 from ISO/IEC 23894 and 42005. 42001 is the certifiable management system; 23894 is risk management guidance; 42005 is impact assessment guidance. Note that 42001 treats the AI risk assessment and the AI system impact assessment as separate but linked requirements.
• Distinguish provider from deployer. Many questions hinge on who must do what. The provider conducts the conformity assessment and CE marking; the deployer ensures human oversight, uses the system per instructions, performs a FRIA where required, and reports incidents.
• Identify the trigger for re-assessment. Retraining with new data, change of intended purpose, new jurisdiction, new user population, integration of a new foundation model, or a serious incident all require revisiting the assessment. 'Substantial modification' is the legal term to recognize.
• Look for the FIRST or BEST qualifier. If asked what to do FIRST, choose scoping, classification or inventory steps. If asked what is MOST important for conformity, choose documented evidence and accountability. If asked what MOST reduces harm, choose a control directly tied to the identified harm.
• Link security to conformity explicitly. If a scenario mentions adversarial attacks, prompt injection or model theft in a regulated context, the best answer often integrates the security threat assessment into the impact assessment and technical documentation rather than handling it separately.
• Residual risk needs an owner. Any answer in which the security team or vendor unilaterally accepts residual AI risk on behalf of the business is wrong.
• Third-party models are not a pass. Using a vendor's foundation model does not transfer the deployer's obligations. The best answer includes obtaining vendor documentation (model cards, system cards, test results) and performing your own context-specific impact assessment.
• Prefer stakeholder engagement when the scenario involves vulnerable groups. Consulting affected communities or domain experts is a recognized best practice in impact assessment and is frequently the differentiator in the correct option.
• Eliminate systematically. Cross out options that are reactive, undocumented, technology-only, role-inappropriate, or single-regulation when multiple apply. The remaining option is usually correct.

8. Worked Example Questions

Example 1. An organization plans to deploy an AI tool to rank job applicants in the EU. What should the AI security manager recommend FIRST?
A. Implement bias testing on the model outputs.
B. Classify the system against the EU AI Act and initiate an impact assessment.
C. Obtain a notified-body certificate.
D. Deploy in pilot mode and monitor complaints.
Answer: B. Employment is an Annex III high-risk category; classification and impact assessment come first and determine which controls (A) and which conformity route (C) are needed. D is deploy-then-assess, which is wrong.

Example 2. A deployer retrains a vendor-supplied high-risk AI model on its own customer data to improve accuracy. What is the MOST significant governance implication?
A. The deployer may now be treated as a provider and must re-perform conformity assessment.
B. The vendor remains solely responsible.
C. Only a DPIA update is needed.
D. No action is required because accuracy improved.
Answer: A. Substantial modification can shift provider obligations to the deployer and triggers re-assessment. B and D are distractors; C is too narrow.

Example 3. Which activity BEST demonstrates ongoing regulatory conformity for a deployed high-risk AI system?
A. A signed declaration of conformity from the launch date.
B. A requirements traceability matrix linking obligations to controls, with post-market monitoring records and incident logs.
C. A vendor's marketing statement of compliance.
D. Annual security awareness training.
Answer: B. Conformity is continuous and must be evidenced; B provides traceable, current evidence. A is point-in-time, C is unverified, D is unrelated.

Example 4. During an AI impact assessment, the team identifies a risk of model inversion exposing training data about individuals. Where should this be recorded?
A. Only in the security risk register.
B. In the impact assessment as a privacy and security harm, mapped to Article 15 cybersecurity and GDPR obligations, with controls in the technical documentation.
C. Nowhere, because it is a theoretical attack.
D. In the vendor's documentation only.
Answer: B. The AAISM approach integrates security threats into the impact assessment and conformity evidence.

9. One-Page Recap

• An AI Impact Assessment identifies and treats harms to people, organizations and society, performed before deployment and on substantial change.
• Regulatory conformity is the demonstrable, continuous satisfaction of legal and standards obligations, evidenced through documentation, traceability, monitoring and assurance.
• EU AI Act = risk tiers and conformity assessment; NIST AI RMF = Govern/Map/Measure/Manage; ISO/IEC 42001 = certifiable AIMS; GDPR = DPIA and Article 22.
• Security threat assessments are conformity evidence, not a side activity.
• On the exam: classify first, assess before deploying, document everything, assign accountability to the business owner, treat conformity as continuous, and reject absolutes and reactive options.