OWASP Top 10 for LLM Applications and MITRE ATLAS

Why This Topic Matters

Large language models (LLMs) and other AI systems have introduced an entirely new class of security failures that traditional application security frameworks do not fully describe. A firewall does not stop a prompt injection, and a code review does not detect a poisoned training dataset. Security managers therefore need shared, industry-recognized vocabularies to identify, communicate, and govern AI-specific threats. Two frameworks have emerged as the de facto references: the OWASP Top 10 for LLM Applications and MITRE ATLAS.

Within the ISACA AAISM (Advanced in AI Security Management) body of knowledge, these frameworks sit inside the AI Governance and Program Management domain because they are not merely technical checklists. They are governance instruments. A security program uses them to define a threat model, prioritize controls, structure risk assessments, train developers, write policy, and demonstrate due diligence to regulators, auditors, and boards. An AI security manager who cannot map an organizational control set to these frameworks will struggle to prove the program is comprehensive. The exam tests whether you understand what each framework is, how each is structured, how they differ, and how a manager applies them within a program.

Part 1: OWASP Top 10 for LLM Applications

What It Is
The Open Worldwide Application Security Project (OWASP) is best known for its classic Top 10 web application risks. In 2023 OWASP released a dedicated Top 10 for LLM Applications, created by a large volunteer community of practitioners, and it was significantly revised for the 2025 edition. The list identifies the ten most critical security risks specific to applications that build on or integrate LLMs. It is an awareness and prioritization document, written for developers, architects, security teams, and leadership. It is not a complete control catalog and it is not a certification standard.

The 2025 OWASP Top 10 for LLM Applications

LLM01: Prompt Injection – Attackers manipulate model behavior through crafted inputs. Direct injection comes from the user (jailbreaks, instruction overrides). Indirect injection comes from external content the model processes, such as a web page, email, or document containing hidden instructions. This is consistently ranked number one because the model cannot reliably distinguish instructions from data. Mitigations: privilege limitation, human-in-the-loop for sensitive actions, input and output filtering, segregating external content, and treating the LLM as an untrusted user.

LLM02: Sensitive Information Disclosure – The model reveals confidential data, PII, proprietary algorithms, or secrets, whether memorized from training data, leaked from context windows, or drawn from connected data stores. Mitigations: data sanitization, access controls on retrieval sources, output filtering, differential privacy, and user education that inputs may be retained.

LLM03: Supply Chain – Vulnerabilities in pre-trained models, third-party datasets, fine-tuning adapters (such as LoRA), plugins, and model hubs. Includes tampered or backdoored models and outdated components. Mitigations: model provenance, AI/ML Bill of Materials (AIBOM or SBOM extensions), vendor assessment, signing and integrity verification, and inventory management.

LLM04: Data and Model Poisoning – Manipulation of pre-training, fine-tuning, or embedding data to introduce backdoors, biases, or degraded behavior. Mitigations: data provenance tracking, anomaly detection in training data, sandboxing, red teaming, and validating model behavior before deployment.

LLM05: Improper Output Handling – Downstream components accept LLM output without validation, leading to cross-site scripting, SQL injection, remote code execution, or SSRF when output is passed to browsers, shells, databases, or APIs. Mitigations: treat model output as untrusted, encode and validate outputs, apply least privilege to downstream systems.

LLM06: Excessive Agency – The LLM-based system is granted too much functionality, too many permissions, or too much autonomy (for example, an agent that can delete files or send payments without approval). Mitigations: minimize tools and permissions, require human approval for high-impact actions, log and monitor agent activity, limit autonomy.

LLM07: System Prompt Leakage – Exposure of the system prompt, which may contain credentials, business logic, or filtering rules that attackers can exploit. The key lesson is that system prompts should not be treated as secrets or as a security control. Mitigations: keep sensitive data out of prompts, enforce controls outside the model, and assume the prompt can be extracted.

LLM08: Vector and Embedding Weaknesses – Risks in retrieval-augmented generation (RAG) pipelines, including unauthorized access to vector stores, cross-tenant data leakage, embedding inversion attacks, and poisoned knowledge bases. Mitigations: fine-grained access control on vector databases, data classification, validation of ingested documents, and monitoring of retrieval behavior.

LLM09: Misinformation – The model produces false, fabricated (hallucinated), or misleading content that users or systems trust, including fabricated citations, insecure code, or non-existent software packages (package hallucination). Mitigations: RAG grounding, human review, output verification, user interface disclaimers, and education against overreliance.

LLM10: Unbounded Consumption – Excessive or uncontrolled resource use leading to denial of service, denial of wallet (runaway cloud costs), or model extraction through high-volume querying. Mitigations: rate limiting, input size limits, quotas, cost monitoring, timeouts, and watermarking or query monitoring to detect extraction.

How the List Evolved
The 2023/2024 edition (version 1.1) included Insecure Plugin Design, Overreliance, Model Denial of Service, Training Data Poisoning, and Model Theft. The 2025 version merged, renamed, and expanded these: Insecure Plugin Design was absorbed into Excessive Agency and Supply Chain; Overreliance became Misinformation; Model Denial of Service and Model Theft merged into Unbounded Consumption; and new entries were added for System Prompt Leakage and Vector and Embedding Weaknesses to reflect the explosion of RAG and agentic architectures. Exam questions may reference either edition, so know both names.

Part 2: MITRE ATLAS

What It Is
MITRE ATLAS stands for Adversarial Threat Landscape for Artificial-Intelligence Systems. It is a globally accessible, living knowledge base of adversary tactics, techniques, and real-world case studies targeting AI-enabled systems, maintained by the MITRE Corporation with input from industry, government, and academia. ATLAS is deliberately modeled on the structure of MITRE ATT&CK, the widely adopted framework for conventional cyber adversary behavior. In fact, ATLAS is designed to complement ATT&CK: it adds AI-specific tactics and techniques (prefixed AML, for Adversarial Machine Learning) while reusing ATT&CK techniques where traditional methods apply.

How It Is Structured

Tactics represent the adversary's goal or why a step is taken. ATLAS tactics follow the familiar attack lifecycle: Reconnaissance, Resource Development, Initial Access, ML Model Access (unique to ATLAS), Execution, Persistence, Privilege Escalation, Defense Evasion, Credential Access, Discovery, Collection, ML Attack Staging (unique to ATLAS), Command and Control, Exfiltration, and Impact.

Techniques describe how an adversary achieves a tactic. Examples include: AML.T0051 LLM Prompt Injection (with direct and indirect sub-techniques), AML.T0054 LLM Jailbreak, AML.T0020 Poison Training Data, AML.T0018 Backdoor ML Model, AML.T0043 Craft Adversarial Data, AML.T0024 Exfiltration via ML Inference API, AML.T0040 ML Model Inference API Access, AML.T0010 ML Supply Chain Compromise, AML.T0057 LLM Data Leakage, and AML.T0029 Denial of ML Service.

Case Studies document real-world or realistic red-team incidents mapped to the matrix, such as evasion attacks on commercial malware detectors, poisoning of a chatbot, model theft via API extraction, and prompt injection attacks on LLM-powered assistants. Case studies make ATLAS valuable for threat-informed defense because they show that the techniques are not theoretical.

Mitigations (AML.M identifiers) provide defensive measures mapped to techniques, such as adversarial training, input restoration, model hardening, limiting public release of model information, restricting the number of queries, and sanitizing training data.

The ATLAS Navigator is a visualization tool, adapted from the ATT&CK Navigator, that lets teams overlay detections, control coverage, or incident data onto the ATLAS matrix to identify gaps.

How ATLAS Is Used in Practice
Security managers use ATLAS for threat modeling (walking an AI system through each tactic to ask what an adversary could do), red team planning (selecting techniques to emulate), detection engineering (mapping monitoring coverage), incident classification (describing an AI incident in standard language), and risk communication. Because ATLAS speaks the same language as ATT&CK, it integrates well with existing security operations and threat intelligence programs rather than creating a silo.

Part 3: Comparing and Combining the Two Frameworks

The single most tested distinction is this: OWASP Top 10 for LLM is a vulnerability and risk awareness list focused on LLM applications; MITRE ATLAS is an adversary behavior knowledge base covering AI systems broadly.

Scope: OWASP focuses specifically on LLM-based applications and the development lifecycle. ATLAS covers all machine learning and AI systems, including computer vision, classifiers, and recommendation systems, as well as LLMs.

Perspective: OWASP is defender- and developer-oriented, describing weaknesses to avoid. ATLAS is adversary-oriented, describing how attackers operate step by step.

Structure: OWASP is a ranked list of ten risks with descriptions, examples, and mitigations. ATLAS is a matrix of tactics and techniques with case studies and mitigations, plus tooling.

Lineage: OWASP Top 10 for LLM extends the OWASP application security tradition. ATLAS extends MITRE ATT&CK.

Governance use: OWASP is ideal for secure development standards, developer training, vendor questionnaires, and baseline risk assessments. ATLAS is ideal for threat modeling, red teaming, detection coverage analysis, and incident taxonomy.

The frameworks are complementary, not competing. A mature AI security program maps them together: for example, OWASP LLM01 Prompt Injection corresponds to ATLAS technique AML.T0051, OWASP LLM04 Data and Model Poisoning corresponds to AML.T0020 and AML.T0018, and OWASP LLM10 Unbounded Consumption relates to AML.T0024 and AML.T0029. OWASP tells you where your application is weak; ATLAS tells you how an adversary would exploit it and what the attack looks like in your telemetry. Both are commonly referenced by broader governance frameworks such as the NIST AI Risk Management Framework, ISO/IEC 42001, and the EU AI Act compliance efforts as sources of technical threat knowledge.

Part 4: Role of the AI Security Manager

From an AAISM governance standpoint, the manager's responsibilities include: adopting these frameworks formally in policy and standards; requiring their use in AI risk assessments and threat models; ensuring secure development lifecycle (SDLC) checkpoints address each OWASP LLM risk; incorporating ATLAS techniques into red team scope and tabletop exercises; mapping existing controls to both frameworks to identify gaps; using them in third-party and model supply chain due diligence; maintaining awareness of framework updates since both are living documents; and reporting coverage to leadership using framework language. The manager does not need to personally execute technical tests, but must ensure the program is threat-informed, risk-based, and auditable.

Exam Tips: Answering Questions on OWASP Top 10 for LLM Applications and MITRE ATLAS

Tip 1: Know the one-line identity of each framework. If a question asks for a framework that catalogs adversary tactics and techniques against AI systems with real-world case studies, the answer is MITRE ATLAS. If it asks for the most critical security risks in LLM-based applications, the answer is OWASP Top 10 for LLM Applications. Do not confuse either with NIST AI RMF (a risk management process), ISO/IEC 42001 (a management system standard), or MITRE ATT&CK (conventional cyber adversary behavior).

Tip 2: Memorize the ten OWASP LLM risks and their core idea. Expect scenario questions describing an incident and asking which risk it represents. Hidden instructions in a document processed by the model is indirect prompt injection. LLM output passed to a shell without sanitization is improper output handling. An agent making unauthorized purchases is excessive agency. Runaway API costs or model extraction is unbounded consumption. Fabricated citations trusted by staff is misinformation. A compromised pre-trained model from a public hub is supply chain. Cross-tenant leakage from a vector database is vector and embedding weaknesses.

Tip 3: Distinguish prompt injection from jailbreaking and from poisoning. Prompt injection manipulates the model at inference time through inputs; jailbreaking is a subset aimed at bypassing safety guardrails; poisoning manipulates the model at training or fine-tuning time through data. Timing (training versus inference) is the fastest way to tell poisoning from injection.

Tip 4: Remember that the system prompt is not a security control. Questions that present a design relying on instructions in the system prompt to prevent misuse are pointing you toward the correct answer that enforcement must occur outside the model (access controls, output validation, least privilege).

Tip 5: Recognize ATLAS structural terms. Tactics are goals, techniques are methods, case studies are documented incidents, mitigations are defenses, and the Navigator is the visualization tool. The two tactics unique to ATLAS versus ATT&CK are ML Model Access and ML Attack Staging. Technique identifiers begin with AML.T.

Tip 6: Think like a manager, not an engineer. The AAISM exam favors answers that emphasize governance actions: integrating frameworks into policy, risk assessment methodology, SDLC gates, vendor due diligence, training, and reporting to stakeholders. When asked for the best or first step, select the option that establishes a structured, risk-based, repeatable process over a one-time technical fix.

Tip 7: Favor complementary use. When a question asks how to build a comprehensive AI threat model or how OWASP and ATLAS relate, the correct answer typically describes using both together: OWASP for vulnerability awareness and secure development, ATLAS for adversary emulation, detection, and incident classification.

Tip 8: Watch for edition changes. If you see Insecure Plugin Design, Overreliance, Model Denial of Service, or Model Theft, recognize them as earlier OWASP terms that map to Excessive Agency, Misinformation, and Unbounded Consumption in the 2025 list. The underlying concepts are still valid.

Tip 9: Apply the least privilege and untrusted output principles. A large share of correct mitigations across OWASP LLM risks reduce to two principles: treat the model and its outputs as untrusted, and grant the model and its tools the minimum permissions and autonomy required. When in doubt between options, the one embodying these principles is usually correct.

Tip 10: Eliminate absolutes. Answers claiming a control completely prevents prompt injection or fully eliminates hallucination are almost always wrong. Both frameworks emphasize defense in depth, monitoring, and human oversight because these risks can be reduced but not eliminated.

Summary

The OWASP Top 10 for LLM Applications provides a prioritized, developer-friendly list of the most critical risks in LLM-based systems: prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption. MITRE ATLAS provides an adversary-centric, ATT&CK-aligned knowledge base of tactics, techniques, case studies, and mitigations spanning all AI systems. For the AAISM candidate, the key competencies are recognizing each framework's purpose and structure, classifying scenarios correctly, understanding how they complement one another, and articulating how an AI security manager embeds both into governance, risk management, secure development, and program reporting.