Forecasting Secondary and Unintended Uses of AI: A Comprehensive Guide
Introduction
When AI systems are deployed, they are typically designed with a specific primary purpose in mind. However, once released into the real world, these systems are frequently repurposed, adapted, or misused in ways their creators never anticipated. Understanding how to forecast secondary and unintended uses of AI is a critical competency for AI governance professionals. This topic falls under the broader domain of Governing AI Deployment and Use and is essential for anyone preparing for the AIGP (AI Governance Professional) certification exam.
Why Is This Topic Important?
Forecasting secondary and unintended uses of AI matters for several key reasons:
1. Risk Mitigation: Unintended uses can introduce significant risks, including harm to individuals, communities, and organizations. If an AI system designed for one purpose is repurposed for another, the original risk assessments, fairness evaluations, and safety checks may no longer be valid.
2. Legal and Regulatory Compliance: Many regulations (such as the EU AI Act) classify AI systems based on their use case and risk level. A secondary use could shift an AI system into a higher risk category, triggering additional regulatory obligations that the deploying organization may not be prepared to meet.
3. Ethical Responsibility: AI developers and deployers have an ethical obligation to consider the broader impacts of their technologies. Failure to anticipate misuse can lead to discrimination, surveillance overreach, privacy violations, and erosion of public trust.
4. Organizational Reputation: When AI systems are misused—even by third parties—the original developer or deployer may suffer reputational damage. Proactive forecasting demonstrates responsible AI practices.
5. Dual-Use Concerns: Some AI technologies have inherent dual-use potential, meaning they can be used for both beneficial and harmful purposes. Facial recognition, for instance, can aid accessibility efforts but also enable mass surveillance.
What Are Secondary and Unintended Uses of AI?
It is important to distinguish between these two closely related concepts:
Secondary Uses: These are uses of an AI system that differ from its original intended purpose but may still be foreseeable. For example, a language model designed for customer service chatbots might be secondarily used to generate marketing copy. Secondary uses may or may not be problematic, but they require separate evaluation because the AI system was not specifically designed, tested, or validated for that purpose.
Unintended Uses: These are uses that the developers or deployers did not foresee and did not design the system to support. Unintended uses can range from benign to highly harmful. For example, an AI-powered image enhancement tool designed for photography could be unintentionally used to create deepfakes.
Additional related concepts include:
- Misuse: Deliberate use of an AI system for harmful purposes, such as using a chatbot to generate phishing emails.
- Foreseeable Misuse: Harmful uses that a reasonable person or organization should have anticipated, even if they were not the intended purpose.
- Function Creep: The gradual expansion of an AI system's use beyond its original purpose, often without explicit authorization or adequate governance oversight.
- Off-Label Use: Borrowing from pharmaceutical terminology, this refers to using an AI system for purposes other than those for which it was originally developed and validated.
How Does Forecasting Secondary and Unintended Uses Work?
Effective forecasting involves structured processes, diverse perspectives, and ongoing vigilance. Here are the key mechanisms and approaches:
1. Pre-Deployment Risk Assessment
Before deploying an AI system, organizations should conduct comprehensive risk assessments that explicitly consider secondary and unintended uses. This includes:
- Identifying the system's capabilities broadly, not just its intended application
- Brainstorming potential alternative uses (both beneficial and harmful)
- Assessing the accessibility of the system and whether it could easily be repurposed
- Evaluating the data the system processes and whether secondary uses might create privacy risks
2. Stakeholder Engagement and Red Teaming
Diverse perspectives are essential for anticipating unintended uses:
- Red teaming: Engaging adversarial testers to deliberately try to misuse the system
- Diverse stakeholder consultation: Including civil society groups, affected communities, ethicists, domain experts, and end users
- Cross-functional teams: Involving legal, compliance, engineering, product, and policy professionals
- Public input: Soliciting feedback from the broader public or specific user communities
3. Scenario Planning and Threat Modeling
Organizations should develop structured scenarios that explore how AI systems might be used in different contexts:
- Consider different user personas (legitimate users, malicious actors, uninformed users)
- Map out potential downstream applications if the AI system's outputs are used as inputs for other systems
- Evaluate how the system might behave in contexts different from its training environment
- Consider geopolitical, cultural, and socioeconomic factors that might influence use patterns
4. Technical Safeguards and Use Restrictions
Based on forecasting exercises, organizations can implement technical measures:
- Use restrictions and terms of service: Clearly defining permitted and prohibited uses
- Access controls: Limiting who can use the system and for what purposes
- Output filtering: Preventing the system from generating certain types of harmful content
- Monitoring and auditing: Tracking how the system is being used in practice
- Model cards and system documentation: Clearly communicating intended uses and known limitations
5. Post-Deployment Monitoring
Forecasting does not end at deployment:
- Continuously monitor usage patterns for signs of secondary or unintended use
- Establish incident reporting mechanisms
- Conduct periodic re-assessments as the technology ecosystem evolves
- Maintain feedback loops between deployers, users, and developers
- Update risk assessments and governance measures as new use patterns emerge
6. Documentation and Transparency
Thorough documentation supports forecasting efforts:
- Intended use specifications: Clearly documenting what the system is designed to do
- Known limitations: Describing contexts where the system may not perform well
- Out-of-scope uses: Explicitly listing uses for which the system was not designed
- Risk registers: Maintaining records of identified risks, including secondary use risks
Frameworks and Standards Relevant to This Topic
Several frameworks and regulatory instruments address secondary and unintended uses:
- NIST AI Risk Management Framework (AI RMF): Emphasizes the importance of identifying and managing risks associated with AI systems across their lifecycle, including risks from unintended uses. The GOVERN and MAP functions are particularly relevant.
- EU AI Act: Classifies AI systems by risk level based on their intended purpose. Changes in use may reclassify a system, triggering different regulatory requirements. The Act also addresses general-purpose AI models that may be used in many downstream applications.
- ISO/IEC 42001: Provides an AI management system standard that includes requirements for risk assessment and treatment, which should encompass secondary use risks.
- OECD AI Principles: Call for transparency, accountability, and robustness, all of which relate to managing secondary and unintended uses.
Real-World Examples
Understanding real-world examples helps contextualize the concepts:
- Facial recognition technology: Developed for security and authentication, but repurposed for mass surveillance, tracking protesters, and discriminatory policing.
- Large language models: Designed for text generation and assistance, but used to create misinformation, phishing emails, academic fraud, and manipulative content.
- Social media recommendation algorithms: Intended to increase engagement, but inadvertently promoting extremist content, misinformation, and addictive usage patterns.
- AI hiring tools: Developed to screen candidates efficiently, but found to discriminate against certain demographic groups due to biased training data.
- GPS technology: Originally a military navigation tool that became a consumer product with unforeseen uses in stalking and surveillance.
Key Principles to Remember
- Secondary and unintended uses are distinct from intended uses and require separate governance consideration.
- Forecasting should be proactive, not reactive—it should happen before deployment and continue throughout the AI system's lifecycle.
- No forecasting exercise is exhaustive. The goal is to identify and mitigate as many risks as reasonably possible, not to achieve certainty.
- Diverse perspectives are essential for effective forecasting because homogeneous teams are more likely to have blind spots.
- General-purpose AI systems pose particular challenges because they can be applied to a vast range of use cases.
- The capability of an AI system (what it can do) often exceeds its intended purpose (what it was designed to do), creating space for secondary and unintended uses.
Exam Tips: Answering Questions on Forecasting Secondary and Unintended Uses of AI
1. Know the Definitions: Be crystal clear on the distinction between secondary uses, unintended uses, misuse, foreseeable misuse, and function creep. Exam questions may test your ability to classify a scenario correctly.
2. Think Lifecycle: Remember that forecasting is not a one-time exercise. If a question asks about best practices, emphasize that forecasting should occur pre-deployment AND continue post-deployment through monitoring.
3. Emphasize Multi-Stakeholder Approaches: When asked about methods for forecasting, highlight the importance of involving diverse stakeholders, including affected communities, red teams, ethicists, and cross-functional teams. This is a recurring theme in AI governance.
4. Connect to Risk Management Frameworks: If questions reference the NIST AI RMF, EU AI Act, or other frameworks, demonstrate that you understand how secondary uses fit within broader risk management processes. For NIST, think about the MAP function (contextualizing risks) and MANAGE function (treating risks).
5. Consider General-Purpose AI: Questions may specifically address general-purpose AI models (like foundation models or large language models). These require special attention because their broad capabilities make secondary and unintended uses particularly likely and difficult to forecast.
6. Link to Regulatory Implications: Under the EU AI Act, a change in the intended purpose of an AI system can change its risk classification. Be prepared to explain how secondary uses might trigger new regulatory obligations.
7. Balance Technical and Governance Measures: Good answers will reference both technical safeguards (access controls, output filtering, monitoring) and governance measures (policies, documentation, stakeholder engagement, terms of use).
8. Use Real-World Examples: If the exam allows for scenario-based reasoning, draw on well-known examples (facial recognition, LLMs, recommendation algorithms) to illustrate your understanding.
9. Watch for Trick Answers: Be cautious of answer choices that suggest forecasting can eliminate all risks of secondary use. The correct perspective is that forecasting reduces risk but cannot eliminate it entirely. Similarly, avoid answers that suggest only the AI developer is responsible—deployers, users, and regulators all share responsibility.
10. Remember Proportionality: The depth and rigor of forecasting efforts should be proportionate to the risk level and capabilities of the AI system. A low-risk, narrow AI tool does not require the same level of forecasting as a general-purpose foundation model.
11. Documentation Matters: If asked about governance best practices, always include documentation—model cards, intended use statements, risk registers, and out-of-scope use descriptions are key artifacts.
12. Read Questions Carefully: Distinguish between what an organization should do (normative best practices) and what an organization is required to do (regulatory obligations). The exam may test both, and the correct answer depends on the framing of the question.
Summary
Forecasting secondary and unintended uses of AI is a foundational element of responsible AI governance. It requires a combination of proactive risk assessment, diverse stakeholder engagement, technical safeguards, ongoing monitoring, and thorough documentation. As AI systems become more capable and general-purpose, the challenge of anticipating secondary and unintended uses only grows. AI governance professionals must be equipped to lead these forecasting efforts and embed them into organizational AI governance frameworks. Mastering this topic demonstrates both technical understanding and ethical awareness—qualities that are central to the AIGP certification and to the responsible deployment of AI in practice.
