Resilience of Third-Party AI Services

Why This Topic Matters

Most organizations do not build their own AI from scratch. They consume foundation models through APIs, embed vendor-supplied AI features in SaaS platforms, rely on cloud-hosted inference services, and depend on external data providers and model marketplaces. This creates a concentrated dependency: if a third-party AI service degrades, is withdrawn, changes behavior, or is compromised, the organization's own processes, customers, and compliance posture are directly affected. The ISACA Advanced in AI Risk (AAIR) certification treats resilience of third-party AI services as a core element of AI Risk Program Management because the risk owner cannot outsource accountability even when they outsource the capability. Regulators (for example under the EU AI Act, DORA, and financial-services outsourcing guidance) increasingly expect organizations to demonstrate that AI supply chain dependencies are identified, assessed, contracted for, monitored, and recoverable.

What It Is

Resilience of third-party AI services is the organization's ability to anticipate, withstand, recover from, and adapt to disruptions, degradations, or adverse changes in AI capabilities provided by external parties. It extends classic third-party risk management (TPRM) and business continuity concepts to the unique characteristics of AI, including:

• Availability risk – API outages, rate limiting, capacity constraints, regional service failures.
• Performance and model drift risk – the vendor silently updates or retrains a model, changing outputs, accuracy, tone, or safety behavior without notice.
• Deprecation and lock-in risk – model versions are retired, pricing changes, or the vendor exits the market, leaving the organization with tightly coupled prompts, fine-tunes, or embeddings that are hard to migrate.
• Data and confidentiality risk – prompts, documents, and outputs transiting the vendor may be logged, used for training, or exposed in a breach.
• Integrity and security risk – supply chain compromise of models, poisoned training data, insecure plugins, or vulnerabilities in the vendor's stack.
• Concentration and systemic risk – many critical processes, or many organizations in a sector, depend on a small number of AI providers or underlying cloud/hardware layers (the "fourth-party" problem).
• Compliance and transparency risk – inability to obtain model documentation, bias testing evidence, or audit rights needed to meet legal obligations.

How It Works: Building Resilience Into the AI Risk Program

1. Inventory and classification. Maintain an AI asset register that identifies every third-party AI service, the business processes it supports, the data it touches, and its criticality tier. Include embedded AI inside vendor products ("shadow AI" and AI features turned on by SaaS providers) and map fourth-party dependencies such as the underlying foundation model or cloud region.

2. Due diligence and risk assessment. Before onboarding, assess the provider's security posture (SOC 2, ISO 27001, ISO/IEC 42001), model documentation (model cards, system cards), data handling and retention commitments, training-data provenance, bias and safety testing, incident history, financial viability, and geographic/legal exposure. Assess impact tolerance: how long can the process operate without the service, and what is the acceptable degradation?

3. Contractual controls. Resilience is largely secured through contracts: service level agreements (uptime, latency, support response), change notification clauses (advance notice of model updates or deprecations), version pinning rights, data-use restrictions (no training on customer data), confidentiality and breach notification, audit and assurance rights, exit assistance and data/model portability, liability and indemnity terms, and sub-processor transparency.

4. Architectural resilience. Design systems so dependence on any one vendor is manageable: use abstraction layers or gateways that allow model switching, maintain fallback models (an alternative provider, a smaller self-hosted model, or a rules-based process), implement graceful degradation and human-in-the-loop fallback, cache or store critical outputs, and avoid hard-coding provider-specific behavior where possible. Keep prompts, evaluation datasets, and fine-tuning data under organizational control so they can be reused with another provider.

5. Continuous monitoring. Monitor availability, latency, error rates, cost, and—critically—output quality and behavior over time using evaluation sets and drift detection. Track vendor announcements, model version changes, security advisories, and regulatory developments. Reassess risk at defined intervals and upon material changes.

6. Business continuity, incident response, and exit planning. Integrate third-party AI dependencies into business impact analysis (BIA), continuity plans, and incident response playbooks. Define triggers for failover, communication plans, and recovery objectives. Maintain a tested exit strategy, including the ability to retrieve data, revoke access, and migrate to an alternative within the impact tolerance. Conduct tabletop exercises simulating vendor outage, harmful model behavior change, or vendor compromise.

7. Governance and accountability. Assign clear ownership for each third-party AI relationship, report concentration risk and key vendor health to risk committees, and ensure the three lines model applies: business owners manage the relationship, risk/compliance challenges and sets policy, and internal audit provides independent assurance.

Key Concepts to Remember

• Outsourcing the service does not outsource the risk or accountability.
• AI adds behavioral resilience concerns (silent model changes, drift) on top of traditional availability concerns.
• Concentration risk and fourth-party dependencies are distinctive and frequently tested.
• Contractual controls (change notice, version pinning, audit rights, exit assistance) are the primary lever when you cannot control the vendor's technology.
• Fallback and abstraction are the primary technical levers.
• Monitoring must cover output quality, not only uptime.
• Exit strategies must be documented and tested.

Exam Tips: Answering Questions on Resilience of Third-Party AI Services

Tip 1 – Think "lifecycle," not "point in time." If a question asks for the BEST way to manage third-party AI risk, prefer answers that describe ongoing processes (continuous monitoring, periodic reassessment, change management) over one-time actions such as a single pre-contract questionnaire.

Tip 2 – Identify which failure mode the scenario describes. Read for cues: "outputs changed unexpectedly" points to model drift/change-notification controls; "service became unavailable" points to fallback and continuity planning; "vendor announced end of life" points to exit strategy and portability; "vendor was breached" points to data minimization and incident response; "several critical processes rely on the same provider" points to concentration risk.

Tip 3 – Match the control to the lever. When you cannot change the vendor's technology, the correct answer is usually contractual (SLA, notice period, version pinning, audit rights). When you need operational protection, the correct answer is architectural (abstraction layer, alternative model, human fallback). Distractors often propose a technical fix for a contractual gap or vice versa.

Tip 4 – "FIRST" questions favor inventory and impact analysis. When asked what should be done FIRST, the answer is typically to identify and classify dependencies or perform a business impact analysis, because you cannot prioritize controls without knowing criticality.

Tip 5 – "MOST important" or "BEST" questions favor governance and accountability. ISACA exams reward answers that establish ownership, define risk appetite and impact tolerances, and report to appropriate governance bodies over purely operational answers.

Tip 6 – Remember accountability stays in-house. Any option implying that liability, regulatory responsibility, or risk acceptance transfers fully to the vendor is almost always wrong.

Tip 7 – Watch for fourth-party and concentration language. If the scenario mentions a vendor relying on another cloud or foundation model provider, the best answer usually involves mapping sub-processors and assessing systemic/concentration risk, not just evaluating the direct vendor.

Tip 8 – Testing beats documenting. Between an answer that says "document an exit plan" and one that says "test the exit plan and fallback through exercises," the tested option demonstrates true resilience.

Tip 9 – Output quality is part of availability for AI. A service that is "up" but producing degraded, biased, or unsafe results is a resilience failure. Choose answers that include output monitoring and evaluation benchmarks rather than uptime-only metrics.

Tip 10 – Eliminate extremes. Options such as "prohibit all third-party AI," "always build in-house," or "rely solely on vendor certifications" are typically distractors. The exam favors proportionate, risk-based approaches aligned to criticality.

Sample Question Approach

An organization's customer-service chatbot relies on a single external foundation model via API. The vendor retains the right to update the model at any time. Which of the following would MOST effectively reduce the resilience risk?

Analysis: The threat is silent behavioral change plus single-vendor dependence. A vendor questionnaire (distractor) is point-in-time. Increasing uptime SLA (distractor) addresses availability, not behavior change. The strongest answer combines negotiating change-notification and version-pinning rights with implementing an abstraction layer and a validated fallback model, supported by continuous output evaluation. Look for the option that addresses both the contractual and architectural levers, or if only one is offered, the one that directly targets the described failure mode.

Summary

Resilience of third-party AI services means treating external AI as a critical dependency that must be inventoried, assessed, contracted for, architected around, monitored, and recoverable. For the AAIR exam, anchor your reasoning in lifecycle thinking, map scenarios to specific failure modes, choose controls that fit the lever available, prioritize governance and impact analysis in "FIRST/MOST" questions, and never accept an answer that lets accountability leave the organization.