Model Selection: Build, Buy, Fine-Tune or Prompt

1. Why This Topic Matters

Every AI initiative begins with a sourcing decision: will the organization build a model from scratch, buy a vendor model or AI-enabled product, fine-tune an existing foundation model with its own data, or simply prompt a general-purpose model through an API or interface? In the ISACA AAISM (Advanced in AI Security Management) body of knowledge, this decision sits inside the AI Technologies and Controls domain because it determines almost everything that follows: who owns the data, where the risk lives, what controls are technically possible, how much visibility the security team has into the model, and how the shared-responsibility boundary with third parties is drawn.

From a security management perspective, the sourcing choice is not a purely technical or financial decision. It drives:
• Risk ownership — a built model places full accountability for bias, robustness, privacy and security on the organization; a bought model shifts much of it (but never all of it) to a vendor.
• Data exposure — fine-tuning and prompting both move organizational data toward the model, raising confidentiality, residency and retention concerns.
• Control feasibility — you cannot inspect weights, perform full adversarial testing, or retrain a closed vendor model; you can with your own.
• Compliance and explainability — regulators increasingly expect documentation of model provenance, training data lineage and decision logic. Each option supports this to a different degree.
• Cost, time and talent — the classic business trade-off that security leaders must integrate rather than ignore.

AAISM candidates are expected to advise executives on the risk-informed selection of an approach, recognize the security implications of each path, and design the appropriate control set for whichever is chosen.

2. What It Is: The Four Options Defined

Build (Train from Scratch)
The organization designs the architecture, collects and curates the training data, trains, validates and deploys the model entirely in-house or through a contracted development team. Historically this meant classical machine learning models (regression, tree ensembles, small neural networks). Today it can also mean training a domain-specific foundation model, though this is rare because of the enormous compute, data and expertise required.
Characteristics: maximum control, maximum customization, maximum intellectual property ownership, maximum cost and time, full accountability for every risk.

Buy (Procure a Vendor Model or AI-Enabled Product)
The organization licenses or subscribes to a pre-built model or a product with embedded AI (for example, a fraud-detection SaaS, an HR screening tool, or a commercial LLM accessed as a managed service). The model is a black box or near black box. The organization configures and integrates it but does not alter its parameters.
Characteristics: fastest time to value, lowest internal expertise required, dependence on the vendor's security posture and transparency, limited customization, vendor lock-in risk, third-party and supply-chain risk management becomes the primary control lever.

Fine-Tune (Adapt a Pre-Trained Model)
The organization takes an existing foundation model (open-weight such as Llama or Mistral, or a vendor model that permits tuning) and continues training it on a smaller, domain-specific dataset. Variants include full fine-tuning, parameter-efficient fine-tuning (PEFT, LoRA, adapters), instruction tuning, and reinforcement learning from human feedback (RLHF).
Characteristics: middle ground on cost and control; inherits the risks and biases of the base model while adding new risks from the tuning data (data poisoning, privacy leakage, memorization); requires MLOps capability; produces a derivative asset whose licensing must be checked.

Prompt (Prompt Engineering / Retrieval-Augmented Generation without Retraining)
The organization uses a general-purpose model as-is and shapes its behavior through system prompts, few-shot examples, prompt templates, guardrails, and often retrieval-augmented generation (RAG), where enterprise documents are retrieved at inference time and inserted into the context window. No model weights are changed.
Characteristics: lowest barrier to entry, highly agile, data stays outside the model's parameters (but is sent to the model at inference), vulnerable to prompt injection and jailbreaks, results can be less consistent, heavy reliance on the provider's data handling terms.

Note that these are not mutually exclusive. A mature program frequently combines them: buy a foundation model via API, apply RAG and prompt engineering for most use cases, fine-tune for a few high-volume specialized tasks, and build small classical models where the problem is narrow and data is proprietary.

3. How It Works: The Decision Framework

AAISM frames model selection as a structured, risk-informed decision. The following factors should be assessed, documented and approved through governance (typically an AI steering committee or model risk committee).

A. Business and Use-Case Factors
• Strategic differentiation: Is the capability a competitive differentiator (favor build or fine-tune) or a commodity (favor buy or prompt)?
• Performance requirements: Does the task require domain-specific accuracy that general models cannot reach? Fine-tuning or building may be justified.
• Time to value: Prompt and buy deliver in days or weeks; fine-tune in weeks or months; build in months or years.
• Total cost of ownership: Include compute, data acquisition and labeling, talent, licensing, inference costs per token or per call, monitoring, retraining and decommissioning.

B. Data Factors
• Data availability and quality: Building requires large, representative, well-governed datasets. Fine-tuning needs smaller but high-quality labeled data. Prompting needs almost none.
• Data sensitivity and classification: Highly sensitive or regulated data (PHI, PCI, classified) may prohibit sending it to an external provider, pushing toward self-hosted build or fine-tune options, or toward strict contractual and technical controls if buying.
• Data residency and sovereignty: Cross-border transfer rules may rule out certain vendors or regions.
• Data lineage and consent: Was the training data of a vendor or open model lawfully obtained? Copyright, licensing and consent issues are inherited by the adopter.

C. Security and Risk Factors
• Transparency and explainability: Regulated decisions (credit, employment, healthcare) often require explanations. Black-box vendor models may not satisfy this; building or using interpretable models may be necessary.
• Attack surface: Prompt-based systems face prompt injection, indirect injection via retrieved content, jailbreaks and data exfiltration through outputs. Fine-tuned models face training-data poisoning and membership inference. Built models face every risk in the lifecycle. Bought models face supply-chain compromise and opaque updates.
• Control feasibility: Can you red-team it, retrain it, roll back a version, pin a model version, or audit its weights? Vendors frequently update models silently, which can change behavior and invalidate prior testing.
• Third-party risk: For buy and prompt, vendor due diligence, contractual terms (data use for training, retention, breach notification, SLAs, exit rights), SOC 2 / ISO 27001 / ISO 42001 attestations, and right-to-audit clauses become primary controls.
• Concentration and lock-in risk: Dependence on a single provider creates business continuity and pricing risk. Abstraction layers and portability planning mitigate this.

D. Organizational Factors
• Talent and maturity: Building requires data scientists, ML engineers, MLOps and AI security specialists. Prompting requires far less, though guardrail engineering is a real skill.
• Governance readiness: Does the organization have a model inventory, model risk management policy, acceptable-use policy and incident response plan for AI?
• Risk appetite: The board's stated appetite for AI risk should constrain which options are acceptable for which use-case tiers.

E. Legal, Regulatory and Ethical Factors
• Regulations such as the EU AI Act impose obligations based on risk tier; high-risk systems require documentation, human oversight and robustness testing that may be hard to evidence for an opaque bought model.
• Open-weight licenses (Apache, Llama community license, etc.) contain usage restrictions that must be reviewed before fine-tuning or commercial deployment.
• Output ownership, liability for harmful outputs, and indemnification differ by option and must be contractually clear.

4. Comparison at a Glance

Control over model behavior: Build (highest) > Fine-tune > Prompt > Buy (lowest, unless the product is configurable).
Speed to deploy: Prompt and Buy (fastest) > Fine-tune > Build (slowest).
Internal expertise required: Build (highest) > Fine-tune > Prompt > Buy.
Data exposure to third parties: Prompt and Buy via API (highest, data leaves the boundary at inference) > Fine-tune on hosted service > Self-hosted fine-tune or Build (lowest).
Inherited risk from base model: Buy, Prompt and Fine-tune all inherit the base model's biases, training data issues and vulnerabilities; Build does not inherit but must create everything correctly.
Transparency and auditability: Build (highest) > open-weight Fine-tune > Prompt on documented model > closed Buy (lowest).
Primary control lever: Build — secure ML lifecycle and MLOps; Fine-tune — training data governance and model validation; Prompt — input/output guardrails and RAG access control; Buy — third-party risk management and contracts.

5. Security Controls Mapped to Each Option

Build: secure development lifecycle for ML, data provenance and integrity controls, training environment isolation, adversarial robustness testing, bias and fairness testing, model cards and datasheets, model registry with versioning and signing, access control to weights, continuous monitoring for drift, secure decommissioning.

Buy: vendor security assessment and attestations, contractual data-use restrictions (no training on customer data), data retention and deletion terms, model version pinning or change notification, right to audit or to receive assurance reports, exit and portability plans, integration testing, output monitoring, incident notification SLAs, review of the vendor's own model documentation and known limitations.

Fine-Tune: all Build-style controls for the tuning phase plus base-model license review, base-model vulnerability and bias assessment, tuning-data poisoning defenses (validation, anomaly detection, provenance), privacy techniques (de-identification, differential privacy where appropriate), tests for memorization and leakage of training data, regression testing to confirm safety behaviors were not degraded by tuning (fine-tuning can erode a base model's safety alignment), secure storage of derived weights.

Prompt: system prompt protection and hardening, input validation and prompt-injection defenses, output filtering and content moderation, RAG source authorization (the retriever must enforce the same access controls as the source systems), context-window data minimization, logging of prompts and responses with appropriate retention, rate limiting, human-in-the-loop for consequential outputs, monitoring for jailbreak patterns, provider terms that prohibit using prompts for training.

6. Governance Integration

The selection decision should be recorded in a model inventory entry that captures the sourcing approach, owner, risk tier, data classification involved, vendor (if any), version, and approval date. The decision should be revisited when: the use case changes, the vendor materially changes the model, regulatory requirements shift, incidents occur, or cost or performance assumptions are invalidated. Many organizations formalize this as a build-versus-buy decision record reviewed by a cross-functional committee including security, legal, privacy, data governance, procurement and the business owner. The security leader's role is to ensure risks are surfaced honestly, that the chosen approach has a feasible control set, and that residual risk is accepted by the appropriate authority rather than defaulted into silently.

7. How to Answer Exam Questions on Model Selection

AAISM questions in this area are typically scenario-based. They describe an organization with certain characteristics (regulated industry, sensitive data, limited AI talent, tight timeline, need for explainability, existing vendor relationship) and ask for the most appropriate approach, the primary risk, the first action, or the best control. Approach them like this:

Step 1 — Identify the dominant constraint. Read the scenario for the one factor that is non-negotiable: data cannot leave the country, decisions must be explainable to regulators, deployment must happen this quarter, no in-house ML team exists, the capability is core intellectual property. That constraint usually eliminates two or three options immediately.

Step 2 — Map constraint to option. Sensitive data plus strong need for control points to build or self-hosted fine-tune. Commodity capability plus no talent points to buy. Rapid experimentation with general tasks points to prompt. Domain-specific language or format needs with moderate resources points to fine-tune.

Step 3 — Think like a risk manager, not an engineer. ISACA exams reward answers that address governance, accountability and risk before technical tweaks. If asked what to do first, the answer is almost always to assess risk, classify data, define requirements, or engage governance — not to start training or sign the contract.

Step 4 — Remember that accountability never transfers. Even when buying, the organization remains accountable for outcomes. Options that imply the vendor is fully responsible are distractors.

Step 5 — Match control to option. If the scenario is a bought model, the best control is usually contractual or third-party assurance. If it is a prompted model, the best control is usually input/output guardrails or RAG access control. If fine-tuned, the best control concerns training data integrity or post-tuning safety regression testing. If built, the best control concerns the secure ML lifecycle.

Exam Tips: Answering Questions on Model Selection: Build, Buy, Fine-Tune or Prompt

Tip 1 — Watch for the word 'control'. When a scenario emphasizes the need for control, customization or intellectual property ownership, the exam is steering toward build (or, as a compromise, fine-tune on open weights). When it emphasizes speed, cost or lack of expertise, it is steering toward buy or prompt.

Tip 2 — Fine-tuning is not free of base-model risk. A common distractor suggests fine-tuning 'removes' bias or vulnerabilities from the foundation model. It does not; it adds a layer on top and can even weaken built-in safety behaviors. The correct answer will mention assessing and inheriting base-model risk.

Tip 3 — Prompting still exposes data. Candidates sometimes assume that because weights are not changed, no data is at risk. Prompts and retrieved documents are transmitted to the provider at inference. If the scenario involves confidential data and a public API, expect the answer to involve contractual data-use restrictions, data minimization, a private deployment, or choosing a different approach.

Tip 4 — Buying shifts work, not accountability. If an option says the vendor is accountable for compliance or for harmful outputs, it is wrong. The organization is accountable; the vendor is responsible for specific contracted obligations. Look for answers involving due diligence, contractual clauses and ongoing monitoring.

Tip 5 — The 'first' step is almost always governance. When asked what should be done first before selecting an approach: define business requirements, classify the data, perform a risk assessment, determine the regulatory tier, or consult the AI governance committee. Technical actions (training, benchmarking, integrating) come later.

Tip 6 — Explainability points away from black boxes. If the scenario involves decisions about individuals (hiring, lending, medical triage) and regulatory scrutiny, prefer approaches that allow inspection and documentation — build or open-weight fine-tune — or at least require the vendor to supply model documentation and support explanation mechanisms.

Tip 7 — Silent vendor updates are a recurring theme. A scenario describing unexpected changes in a bought or API-accessed model's behavior is testing whether you know to require version pinning, change notification and re-validation after updates.

Tip 8 — Lock-in is a business continuity risk. Questions about dependency on a single AI provider want answers involving abstraction layers, portability, exit clauses and contingency planning — not simply 'choose a bigger vendor'.

Tip 9 — Hybrid is often the mature answer. If an option proposes a tiered approach — prompt for low-risk uses, fine-tune for specialized needs, self-host for sensitive data — and the scenario describes a large organization with diverse use cases, that is likely the best answer over any single approach.

Tip 10 — Eliminate absolutes. Options containing 'always', 'never', 'eliminates risk' or 'guarantees compliance' are rarely correct. ISACA prefers risk-based, proportionate language such as 'reduces', 'aligns with risk appetite' and 'commensurate with risk tier'.

Tip 11 — Cost questions include hidden costs. If asked about total cost of ownership, the best answer includes monitoring, retraining, inference costs, compliance effort and decommissioning — not just licensing or compute.

Tip 12 — License review is a control. For open-weight models, a scenario asking about the first legal or compliance concern before fine-tuning is usually pointing at license terms and training data provenance, not at technical performance.

8. Quick Recall Summary

• Build = control and ownership, highest cost and full accountability; primary controls are the secure ML lifecycle.
• Buy = speed and low expertise, opaque and vendor-dependent; primary controls are third-party risk management and contracts.
• Fine-Tune = specialization at moderate cost, inherits base-model risk and adds tuning-data risk; primary controls are data governance and post-tuning validation.
• Prompt = agility and minimal data preparation, exposed to injection and inference-time data leakage; primary controls are guardrails and RAG access control.
• Selection must be risk-informed, documented, approved through governance, recorded in the model inventory and revisited when conditions change.
• Accountability for outcomes remains with the organization regardless of the option chosen.

Master the mapping between scenario constraint → appropriate option → primary control, and you will be well prepared for AAISM questions on model selection.