AI Security in the SDLC and MLOps Pipelines

1. Why This Topic Matters

Traditional software is built once and behaves deterministically. AI systems are different: they are built from data as much as from code, they learn behaviors that no developer explicitly wrote, they depend on a long chain of third-party models, datasets and libraries, and they can silently degrade or be manipulated after deployment. This means that classic secure software development lifecycle (SDLC) controls, while still necessary, are not sufficient. The ISACA AAISM exam places this topic inside the AI Governance and Program Management domain because the central lesson is a management one: AI security cannot be bolted on at the end. It must be designed into the lifecycle, enforced through pipeline gates, assigned to accountable owners, and measured continuously.

From an exam perspective, this area tests whether you can:
• Recognize how AI-specific risks (data poisoning, model theft, adversarial inputs, prompt injection, supply chain compromise, drift) map to specific lifecycle stages.
• Identify the control that belongs at each stage and who owns it.
• Apply a shift-left, risk-based and governance-first mindset rather than a purely technical fix.
• Distinguish between the responsibilities of governance bodies, security teams, data scientists and MLOps engineers.

2. What It Is

2.1 Secure SDLC for AI
A secure SDLC embeds security activities (requirements, threat modeling, secure design, secure coding, testing, deployment hardening, operations and decommissioning) into every phase of development. For AI, the lifecycle is extended to cover the data and model lifecycles in parallel with the software lifecycle. ISACA frames this as an AI system lifecycle that typically includes: business case and risk assessment, data sourcing and preparation, model design and training, validation and testing, deployment and integration, monitoring and maintenance, and retirement.

2.2 MLOps and MLSecOps
MLOps is the set of practices that automate and operationalize the machine learning lifecycle: data pipelines, experiment tracking, model training, versioning in a model registry, CI/CD for models, deployment, and monitoring. MLSecOps (sometimes called SecMLOps or AI DevSecOps) is the discipline of embedding security controls and security automation into those pipelines, in the same way DevSecOps embedded security into DevOps. The key idea is that pipeline stages become enforcement points where policy can be automatically checked.

2.3 The Expanded Attack Surface
Compared with conventional software, AI systems add new assets and new attack vectors:
• Training data – can be poisoned, biased, mislabeled, or contain sensitive personal data.
• Models – can be stolen (model extraction), inverted to reveal training data, backdoored, or replaced with a tampered artifact.
• Pre-trained and open-source models – may contain hidden malicious behavior or unsafe serialization formats (for example pickle-based files that execute code on load).
• Prompts, embeddings and vector stores – for LLM systems, prompt injection and retrieval poisoning become live threats.
• Pipelines and orchestration tools – notebooks, feature stores, experiment trackers and CI runners hold credentials and privileged access and are frequently under-secured.
• Inference endpoints – exposed APIs enable evasion attacks, denial of service, and data exfiltration through outputs.

Reference frameworks commonly cited in this context include MITRE ATLAS (adversarial threat landscape for AI systems), the OWASP Machine Learning Security Top 10, the OWASP Top 10 for LLM Applications, the NIST AI Risk Management Framework (Govern, Map, Measure, Manage), ISO/IEC 42001 (AI management systems), ISO/IEC 5338 (AI system lifecycle processes) and ISO/IEC 23894 (AI risk management). You are not expected to memorize every line of these, but you should know what each is for and that ISACA expects alignment with recognized frameworks.

3. How It Works – Security Across Each Lifecycle Stage

Stage 1: Planning, Requirements and Risk Assessment
• Define the AI use case, its business value, its risk tier and its regulatory exposure (privacy, sector rules, the EU AI Act risk categories, etc.).
• Perform an AI-specific risk assessment and threat model early. Use STRIDE or similar, extended with ATLAS tactics such as data poisoning, model evasion and model extraction.
• Write security, privacy, fairness and robustness requirements alongside functional requirements. Define acceptance thresholds (for example maximum acceptable adversarial error rate) that later test gates will enforce.
• Assign accountable owners: a model owner, a data owner, a security lead. Governance decides whether the project may proceed (go/no-go gate).
• Exam angle: the first step is almost always risk assessment and requirements definition, not tooling.

Stage 2: Data Sourcing, Preparation and Feature Engineering
• Establish data provenance and lineage: where data came from, who approved it, how it was transformed. Lineage supports both forensics and regulatory accountability.
• Validate data quality and integrity: schema checks, anomaly detection, statistical outlier detection to detect poisoning attempts.
• Apply data minimization, classification, anonymization or pseudonymization, and consent/licensing checks to manage privacy and intellectual property risk.
• Protect data at rest and in transit, enforce least-privilege access to data lakes and feature stores, and log access.
• Version datasets so that any model can be traced back to the exact data it was trained on.
• Secure labeling processes, including third-party or crowd-sourced labelers, since label manipulation is a poisoning vector.

Stage 3: Model Design, Development and Training
• Apply secure coding practices to training code and notebooks; scan code and dependencies with SAST and software composition analysis (SCA).
• Manage the AI supply chain: vet pre-trained models and datasets, prefer safe serialization formats, verify checksums and digital signatures, and record components in an AI Bill of Materials (AIBOM) or extended SBOM.
• Isolate training environments (segmented networks, hardened compute, no production credentials in notebooks). Use secrets management rather than hard-coded keys.
• Consider privacy-enhancing techniques where risk justifies them: differential privacy, federated learning, synthetic data.
• Build in robustness techniques such as adversarial training and input preprocessing when the threat model calls for them.
• Track experiments and keep reproducible builds so that results can be independently verified.

Stage 4: Validation, Testing and Security Assurance
• Test beyond accuracy: robustness against adversarial examples, bias and fairness, explainability, privacy leakage (membership inference, model inversion), and safety behaviors.
• Conduct AI red teaming and adversarial testing, including prompt injection and jailbreak testing for LLM applications, and penetration testing of the serving infrastructure and APIs.
• Implement automated CI gates: a model cannot be promoted unless it passes defined security, performance and compliance thresholds.
• Produce model cards, data sheets and risk documentation; require independent review and sign-off for high-risk models (separation of duties between the builder and the approver).

Stage 5: Deployment and Release
• Promote only signed, versioned artifacts from a controlled model registry; verify integrity at deployment time to prevent substitution of a tampered model.
• Harden the serving environment: container and image scanning, infrastructure-as-code scanning, network segmentation, encryption, and strong authentication and authorization on inference APIs.
• Apply rate limiting, input validation and output filtering to reduce extraction, evasion and data leakage risk.
• Use staged rollout patterns (shadow mode, canary, blue/green) with defined rollback procedures.
• Follow change management: documented approval, release notes, and traceability from requirement to deployed artifact.

Stage 6: Operations, Monitoring and Incident Response
• Monitor data drift and concept drift, performance degradation, anomalous query patterns (possible extraction or probing), and abuse of outputs.
• Log inputs, outputs and decisions in a privacy-respecting way to support auditability and forensics.
• Integrate AI-specific scenarios into the incident response plan: suspected poisoning, discovered backdoor, leaked model weights, harmful outputs. Define playbooks including model rollback and retraining.
• Manage retraining pipelines with the same controls as initial training; continuous learning systems are particularly exposed to online poisoning.
• Feed monitoring metrics into governance reporting (key risk indicators, key performance indicators).

Stage 7: Retirement and Decommissioning
• Securely archive or destroy models and training data in accordance with retention and privacy rules.
• Revoke credentials, remove endpoints, and update the model inventory and AIBOM.
• Retain documentation needed for audit and legal defensibility.

4. Governance and Program-Level Elements

Because this topic sits in the governance domain, the exam expects you to connect technical controls to program structures:
• Policies and standards – an AI security policy that mandates lifecycle controls, approved model sources, data handling rules and gating criteria.
• Roles and accountability – clear RACI across data scientists, ML engineers, security, privacy, legal, risk and business owners. Security is a shared responsibility but accountability must be explicit.
• Model and asset inventory – you cannot protect what you do not know you have; shadow AI is a top governance concern.
• Risk-based tiering – higher-risk models get stronger gates, independent validation and more frequent review.
• Third-party and supply chain risk management – contracts, due diligence and ongoing assessment of model, data and platform providers.
• Metrics and reporting – percentage of models with completed threat models, pipeline gate pass/fail rates, time to remediate, drift incidents, red team findings closed.
• Training and culture – data scientists often lack security background; secure AI development training is a core control.
• Continuous improvement – lessons from incidents and audits feed back into lifecycle standards.

5. Common Pitfalls the Exam Likes to Probe
• Treating the model as a black box and only securing the surrounding infrastructure.
• Allowing production data or credentials in experimentation notebooks.
• Downloading unvetted pre-trained models or datasets without provenance checks.
• Validating only on accuracy and skipping adversarial, privacy and fairness testing.
• Having no model registry, no versioning and therefore no ability to roll back or prove what is in production.
• Monitoring uptime but not drift or abuse.
• Running security review as a one-time audit instead of an automated, repeatable pipeline gate.

6. Exam Tips: Answering Questions on AI Security in the SDLC and MLOps Pipelines

Tip 1 – Think like a governance and program manager, not an engineer. ISACA exams reward answers that establish accountability, policy, risk assessment and process before specific tools. If one option is a technical control and another is to define requirements, assign ownership or perform a risk assessment, the governance answer is usually the BEST or FIRST choice unless the question explicitly describes an operational emergency.

Tip 2 – Watch for the words FIRST, BEST, MOST and PRIMARY. FIRST usually points to risk assessment, threat modeling or requirements definition. BEST usually points to the most comprehensive, preventive and sustainable control (for example an automated pipeline gate rather than a manual review). PRIMARY purpose questions often expect an answer about managing risk to acceptable levels or supporting accountability.

Tip 3 – Shift left. When asked where security should be addressed, the answer is as early as possible in the lifecycle and continuously thereafter. Options that defer security to pre-production testing or post-deployment monitoring alone are generally distractors.

Tip 4 – Match the threat to the stage. Data poisoning and label manipulation belong to data preparation and training. Supply chain and malicious model artifacts belong to sourcing and build. Evasion, extraction, prompt injection and inference attacks belong to deployment and operations. Drift belongs to monitoring. If a question describes a symptom, identify the stage, then pick the control native to that stage.

Tip 5 – Prefer prevention and automation over detection and manual effort. Signed artifacts, registry-based promotion, CI gates with defined thresholds, and automated scanning are stronger than ad hoc reviews. However, detection (monitoring, logging) is the right answer when the question is about operations or about identifying attacks that cannot be fully prevented.

Tip 6 – Recognize separation of duties and independence. The person who builds the model should not be the sole approver. Independent validation, especially for high-risk models, is a recurring correct answer.

Tip 7 – Provenance, lineage and inventory are governance gold. If an option mentions model inventory, data lineage, AIBOM, model cards or versioning, it often addresses the root cause of accountability and traceability problems described in the scenario.

Tip 8 – Do not ignore classic controls. AI pipelines still need access control, secrets management, encryption, change management, dependency scanning and secure configuration. A question may be testing whether you remember that MLOps platforms are infrastructure with ordinary vulnerabilities.

Tip 9 – Align with recognized frameworks. When an option proposes adopting or aligning with NIST AI RMF, ISO/IEC 42001, MITRE ATLAS or OWASP guidance to structure the program, it is usually favored over building an unstructured custom approach.

Tip 10 – Eliminate extremes and absolutes. Options such as banning all open-source models, stopping all AI development, or relying on a single vendor assurance are rarely correct. ISACA favors proportionate, risk-based controls.

Tip 11 – Read for the stakeholder in the question. If the scenario asks what the AI security manager should recommend to the board or steering committee, answer with risk posture, metrics and accountability. If it asks what should be implemented in the pipeline, answer with the specific gate or control. Mismatching the audience is a common way to pick a distractor.

Tip 12 – Remember retraining and continuous learning. Questions about models that update themselves in production test whether you recognize that each retraining cycle reintroduces data and model risk and must pass through the same controls and approvals.

7. Quick Review Summary
• AI security in the SDLC extends secure development to data and model lifecycles.
• MLSecOps turns pipeline stages into automated policy enforcement points.
• Key AI-specific threats: poisoning, supply chain compromise, extraction, inversion, evasion, prompt injection, drift.
• Key controls: early risk assessment and threat modeling, data lineage and validation, vetted and signed artifacts, AIBOM, isolated training environments, adversarial and privacy testing, CI gates, model registry, hardened inference APIs, drift and abuse monitoring, AI-aware incident response, secure retirement.
• Governance glue: policy, accountability, inventory, risk tiering, third-party management, metrics, training and continuous improvement.
• Exam mindset: governance first, shift left, risk-based, preventive and automated, aligned with recognized frameworks, matched to the lifecycle stage described in the scenario.