Insecure Output Handling and Excessive Agency

Why This Topic Matters

Large language models (LLMs) and agentic AI systems do not simply answer questions anymore. They generate code, draft database queries, call APIs, send emails, write to files and trigger downstream business processes. Two of the most consequential risks in this environment are Insecure Output Handling and Excessive Agency. Both appear in the OWASP Top 10 for LLM Applications (Insecure/Improper Output Handling and Excessive Agency), and both sit squarely inside the ISACA AAISM domain of AI Risk Management for Security, because they describe how an AI system can be turned into a weapon against the organization that deployed it.

For the AAISM exam, this topic is important because it tests whether you can think like a security manager rather than a developer. You are expected to recognize that an AI model is an untrusted component, that the model's outputs and actions must be governed by the same controls applied to any external input, and that the business owner, not the vendor, is accountable for the consequences of what the AI does. Questions in this area frequently combine technical understanding with governance judgment: least privilege, segregation of duties, human oversight, risk appetite and control ownership.

Part 1: Insecure Output Handling

Definition
Insecure Output Handling occurs when an application accepts the output of an LLM and passes it to another system, user or process without adequate validation, sanitization or encoding. The core mistake is trusting the model. Because the model's output is shaped by its input, and because that input can be influenced by attackers (through prompt injection, poisoned documents, malicious web pages or manipulated retrieval sources), the output can contain hostile content. If the downstream system executes or renders that content blindly, the LLM effectively becomes an injection vector.

How it works in practice
Think of the LLM as a translator that converts attacker intent into a form your systems will execute. Common attack chains include:
• Cross-site scripting (XSS): A chatbot answer containing a script tag is rendered directly in a web page without HTML encoding, executing in the victim's browser.
• SQL or NoSQL injection: An LLM generates a database query from natural language and the application runs it without parameterization, allowing data exfiltration or deletion.
• Remote code execution: An AI coding assistant produces code that is executed automatically in a build pipeline or sandbox that is not actually isolated.
• Server-side request forgery (SSRF): The model produces a URL that a backend service fetches, reaching internal systems the user should never access.
• Command injection: An agent passes model-generated text straight into a shell command.
• Phishing and social engineering: The model generates persuasive content containing attacker-controlled links that are forwarded to employees or customers without review.

Why it happens
Developers often assume the model is a trusted internal component because it was deployed by the organization. They also assume that a system prompt telling the model to behave is a security control. It is not. Prompt instructions are guidance, not enforcement. The model can be manipulated, so anything it produces must be treated as untrusted user input.

Key controls
• Treat all model output as untrusted data; apply a zero-trust mindset to the model itself.
• Validate and sanitize output against an allow-list of expected formats before it reaches any interpreter, browser, database or shell.
• Use context-appropriate output encoding (HTML encoding for web, parameterized queries for databases, escaping for shells).
• Enforce strict schemas for structured output (for example JSON schema validation) and reject anything that does not conform.
• Apply Content Security Policy and other browser-side defenses as compensating controls.
• Run any model-generated code only in isolated, ephemeral sandboxes with no network or credential access.
• Log and monitor model outputs so that anomalous or malicious content can be detected and investigated.
• Integrate output handling tests into secure development lifecycle and AI red-teaming programs.

Part 2: Excessive Agency

Definition
Excessive Agency is the condition in which an AI system has been granted more functionality, more permissions or more autonomy than is necessary for its intended purpose, so that when the model behaves unexpectedly (through hallucination, prompt injection, ambiguous instructions or simple error) it can take damaging actions in the real world. The OWASP framing identifies three root causes:
1. Excessive functionality: The agent has access to tools or plugins it does not need (for example, a document-summarization agent that can also delete files or send emails).
2. Excessive permissions: The agent operates with credentials broader than the task requires (for example, a read-only reporting agent using an account with write and administrative rights).
3. Excessive autonomy: The agent can execute high-impact or irreversible actions without any human confirmation.

How it works in practice
Agentic AI chains together reasoning, planning and tool use. Each tool call is an action with real consequences. Illustrative scenarios include:
• An email assistant with access to the full mailbox is manipulated by a crafted inbound email to forward confidential threads to an external address.
• A customer service agent with refund capability and no transaction limit is tricked into issuing large refunds to a fraudulent account.
• An infrastructure automation agent with administrative cloud credentials misinterprets an instruction and deletes production resources.
• A procurement agent with authority to approve purchases is exploited through a poisoned supplier document to approve fraudulent invoices.
• An AI agent chained to other agents escalates privileges because the downstream agent trusts the upstream agent implicitly.

The defining feature is that the damage is caused by legitimate capabilities being exercised in an unintended way. The attacker does not need to break the system; they only need to redirect it.

Key controls
• Least privilege: Grant the agent only the minimum tools and permissions required for its defined function. Use dedicated service identities per agent rather than inherited user or admin credentials.
• Minimize functionality: Expose narrow, purpose-built tools (for example, a read-only lookup function) instead of general-purpose interfaces (a full shell or unrestricted API).
• Human-in-the-loop: Require explicit human approval for high-risk, irreversible or financially significant actions. Design the approval step so the human sees what will actually happen, not a model summary.
• Rate limits, thresholds and guardrails: Cap transaction values, limit the number of actions per session and block prohibited categories of action.
• Segregation of duties: Do not let a single agent both initiate and approve an action. Apply the same separation principles used for human roles.
• Authorization in the downstream system: Enforce access control at the API or data layer, not only in the agent. The agent should never be the sole gatekeeper.
• Comprehensive logging and traceability: Record every tool call, the inputs that triggered it and the identity under which it ran, to support audit and incident response.
• Kill switches and rollback: Build the ability to halt an agent and reverse its actions where possible.
• Risk-based autonomy levels: Classify use cases by impact and assign autonomy accordingly; fully autonomous operation should be reserved for low-impact, reversible tasks within approved risk appetite.

Part 3: How the Two Risks Interact

Insecure Output Handling and Excessive Agency are closely linked and frequently appear together in exam scenarios. Insecure output handling is about what the model says being executed unsafely; excessive agency is about what the model can do being too broad. A prompt injection attack typically exploits both: the injected instruction produces a malicious output (output handling failure), and that output is turned into a harmful action because the agent has the permissions to carry it out (agency failure). Defense in depth means addressing both layers: constrain the output and constrain the authority. If one control fails, the other should limit the blast radius.

Part 4: The Governance Perspective Expected by AAISM

The AAISM exam is a management-level certification. Beyond the technical controls above, you should be able to articulate:
• Accountability: The organization deploying the AI system owns the risk of its actions, regardless of whether the model is built in-house or procured. Vendor contracts and shared-responsibility models matter, but they do not transfer accountability.
• Risk assessment: Each AI use case should be assessed for the potential impact of erroneous or malicious output and action, and autonomy should be calibrated to risk appetite and tolerance.
• Policy and standards: Secure AI development standards should mandate output validation, least privilege for agents and mandatory human approval for defined action categories.
• Third-party and supply-chain risk: Plugins, tools and external data sources extend the attack surface and must be vetted and monitored.
• Monitoring and incident response: AI actions must be observable, and incident playbooks should include steps to disable agents and revoke credentials.
• Assurance: Red teaming, penetration testing of AI integrations and periodic access reviews of agent permissions provide evidence that controls operate effectively.
• Alignment with frameworks: Map these risks to the NIST AI RMF (Govern, Map, Measure, Manage), ISO/IEC 42001 and the OWASP LLM Top 10 to show structured management rather than ad hoc fixes.

Exam Tips: Answering Questions on Insecure Output Handling and Excessive Agency

1. Identify which risk the scenario describes. If the problem is that model-generated text was rendered, executed or queried without checking, the answer relates to Insecure Output Handling. If the problem is that the AI was able to perform an action it should never have been permitted to perform, the answer relates to Excessive Agency. Watch for scenarios that involve both and ask for the primary or root cause.

2. Remember the golden rule: the model is untrusted. Any answer option that relies on the model policing itself (better system prompts, telling the model to be careful, trusting the vendor's safety training) is almost always a distractor. Correct answers enforce controls outside the model: validation, encoding, sandboxing, authorization at the API layer, human approval.

3. Favor least privilege and least functionality. When asked for the most effective control against excessive agency, look for options that reduce permissions, narrow tool scope or remove unnecessary capabilities. Removing a capability is stronger than monitoring its misuse.

4. Prefer preventive controls over detective ones, but know when the question asks otherwise. If the question asks for the BEST way to prevent harm, choose validation, least privilege or human approval. If it asks how to detect or respond, choose logging, monitoring, alerting or kill switches.

5. Think in terms of defense in depth. When a scenario describes a control that has already failed (for example, prompt injection succeeded), the best next answer is often the layer that limits impact: restricted permissions, sandboxing or human-in-the-loop, rather than another attempt to block the injection.

6. Apply management judgment. AAISM questions often ask what the security manager, CISO or risk owner should do FIRST. Typical correct sequences are: assess the risk and impact of the use case, define acceptable autonomy levels, establish policy and standards, then implement technical controls. Do not jump straight to a technical fix if the question is about governance.

7. Watch for accountability traps. Answers suggesting the AI vendor, the model provider or the plugin developer is accountable for actions taken by your deployed agent are usually wrong. The deploying organization retains accountability.

8. Recognize human-in-the-loop as a control for high-impact actions only. Requiring human approval for every action is impractical and is often a distractor when the question stresses efficiency or scale. The best answer typically applies human approval selectively, based on risk classification of the action.

9. Link to familiar security principles. The exam rewards recognizing that these AI risks are modern expressions of classic principles: input validation, output encoding, least privilege, segregation of duties, zero trust and defense in depth. If an option reflects a long-standing security principle applied to the AI component, it is likely correct.

10. Use elimination on wording. Options containing absolutes such as eliminate all risk or fully trust the model are rarely correct. Options that emphasize proportionality, risk-based decisions, validation and oversight usually are.

Quick Reference Summary
• Insecure Output Handling: Model output passed downstream without validation. Leads to XSS, injection, SSRF, RCE. Fix with validation, encoding, schema enforcement, sandboxing and treating output as untrusted input.
• Excessive Agency: Too much functionality, permission or autonomy. Leads to unauthorized actions, data loss, fraud. Fix with least privilege, narrow tools, downstream authorization, human approval for high-impact actions, limits, logging and kill switches.
• Management view: Organization is accountable; calibrate autonomy to risk appetite; embed controls in policy, SDLC, third-party management, monitoring and assurance.

Mastering this topic means internalizing one idea: an AI system is a powerful but untrusted actor inside your environment, and its words and actions must be constrained by controls you own, test and can prove are working.