Most production AI project in 2026 will have at least one adversarial finding surface after launch; a jailbreak chain, a hallucination customers complain about, a prompt-injection attack from a third-party document, or a red-team report against the OWASP LLM Top 10. The cost of remediation is not generic engineering scope; it is bounded operational work with specific class-by-class costs against specific incident frameworks. Budgets that do not name an insurance line absorb these costs as scope creep, change orders, or silent quality degradation. The fix is paperwork: a 5 to 10 percent insurance reserve in most AI project budget, sized against the OWASP LLM Top 10 class profile, with explicit scope boundaries; what’s covered, what’s not, and the named playbook that triggers when an incident lands. This piece names the insurance line, sizes it against the OWASP LLM Top 10 and NIST AI RMF, and prescribes the boundaries.
It is a spoke under the AI project economics manifesto, which argues that AI economics has shifted from feature cost to evaluation cost; and that adversarial findings are a recurring evaluation event, not an exception. Article #100 in our editorial-600 series.
Why AI projects need an insurance line
Traditional software projects budget for security through three lines: code review and SAST tooling, penetration testing during build, and a security retainer for post-launch remediation. The lines are sized against a deterministic threat model; bugs in code, misconfigurations in infrastructure, vulnerabilities in dependencies. Mature programs hit a known cost per vulnerability and a known frequency.
AI projects break the threat model in a specific way. Adversarial inputs are not bugs in code; they are inputs that exercise the model’s behavior outside its safety training. Hallucinations are not exceptions; they are outputs the model produces with full confidence under conditions the eval suite did not cover. Prompt injections through third-party documents are not configuration errors; they are emergent behaviors of retrieval-augmented systems. None of the three are caught by SAST. None are reliably caught by classical pen-testing.
Three properties make AI risk surface different. Adversarial findings come from external researchers and adversarial users at a higher rate than classical vulnerabilities; the OWASP LLM Top 10 documents class after class of finding patterns that did not exist in the 2018 threat model. Remediation requires eval-suite extension and prompt or retrieval changes, not code patches. The fix is iterative; the first remediation often surfaces a related finding within weeks.
A budget without an insurance line absorbs these costs as scope creep. The buyer surfaces a complaint about hallucinated outputs; engineering scrambles. A red-team report lands; the team treats it as an emergency. A jailbreak chain is documented; remediation gets billed as a change order. Across an enterprise project, this pattern compounds into 5 to 12 percent of total project cost being absorbed invisibly; work that was real, but rarely named.
The insurance line names the work upfront. The cost is the same. The visibility, the planning quality, and the procurement defensibility are different.
What the insurance line covers
Five categories of post-launch adversarial finding.
Jailbreak chains. Prompts that bypass safety training to elicit prohibited outputs. Found by red teams, surfaced in academic papers, demonstrated by adversarial users. Remediation: prompt-level guard adjustments, retrieval filtering, model-side abstention training, or compensating controls (a separate classifier checking outputs).
Hallucination remediation. A specific class of output that customers identify as factually wrong. Distinct from “hallucinations in general” (those are caught by eval suite); this is the long tail that lands in production despite eval validation. Remediation: eval-suite extension to cover the class, retrieval grounding adjustments, output verification layer.
Prompt injection through third-party content. A document the system retrieves contains instructions that hijack the model’s behavior. Especially relevant for systems that ingest customer-provided documents, web-scraped content, or email. Remediation: input sanitization, retrieval-side filtering, structured-output enforcement.
Data leakage. The system unintentionally surfaces training data, system prompt content, or other tenants’ data through carefully-crafted queries. Remediation: prompt-level controls, post-processing filters, system-prompt isolation patterns.
Security advisory remediation. A vendor or research group publishes a finding affecting the model class the system is built on. Remediation: per-system assessment, mitigation deployment, communication to the buyer.
Each category has a similar cost shape: 1 to 4 engineering weeks for assessment, eval-extension, remediation, and verification. A serious enterprise project with active production traffic should expect 2 to 5 of these incidents per year of operation, weighted toward the first six months post-launch when adversarial users are most active.
What the insurance line does NOT cover
Equally important: what is structurally outside the insurance line.
PR and crisis communications. A jailbreak chain that reaches Twitter requires PR response. The communications cost is real but separate from the engineering remediation; it should be a separate line in the budget, owned by communications and legal, not engineering.
Legal exposure and regulatory penalties. A privacy violation surfaced by an adversarial query may trigger regulatory disclosure obligations. The legal cost is separate from the engineering remediation; it should be in the legal reserve, not the AI insurance line.
Pre-launch red team scope. Pre-launch red team is build scope, not insurance scope. The insurance line covers post-launch findings; work that emerges after the system is in production despite the pre-launch testing.
Routine eval-suite expansion. Adding a new test case because the team thought of an edge case is build scope or maintenance retainer scope. The insurance line covers eval-suite expansion specifically triggered by an adversarial finding from outside the team.
Architectural rebuilds. A finding that requires fundamental architecture change (move from naive RAG to a structured-output pipeline) is build scope, not insurance scope. The insurance line covers tactical remediation; architectural changes get re-scoped as a project.
The boundary matters because insurance lines that try to cover everything end up covering nothing; they become contingency budgets that get cut on review. A clearly-bounded insurance line, sized at 5 to 10 percent against named scope, is defensible. A vague insurance line at 20 percent against unnamed scope is not.
The OWASP LLM Top 10 class profile
The OWASP LLM Top 10 (current as of 2026) names ten classes of finding that AI systems should expect to encounter. The insurance line should be sized against the project’s exposure profile to each class.
LLM01 Prompt Injection. Direct (user prompt) and indirect (retrieved content) prompt injection. High exposure for retrieval systems, customer-document-ingestion systems, agentic systems with tool use.
LLM02 Insecure Output Handling. Outputs flowing into downstream systems without validation (SQL injection via LLM-generated SQL, XSS via LLM-generated HTML). High exposure for systems where LLM output drives action.
LLM03 Training Data Poisoning. Less relevant for systems built on frontier-vendor models (the vendor handles this); more relevant for systems doing custom finetuning.
LLM04 Model Denial of Service. Inputs designed to maximize cost or latency. Directly tied to the cost-spike alert tax in the 6 hidden taxes on most AI project.
LLM05 Supply Chain Vulnerabilities. Vulnerable libraries, model artifacts from third parties, plugin ecosystems. Standard supply-chain hygiene applies, plus model-specific concerns.
LLM06 Sensitive Information Disclosure. Training-data leakage, system-prompt leakage, cross-tenant leakage. High exposure for multi-tenant systems and systems with sensitive system prompts.
LLM07 Insecure Plugin Design. Plugin or tool integrations without proper authorization, input validation, or output sanitization.
LLM08 Excessive Agency. Agentic systems that have more permissions than they need. High exposure for systems that can execute code, send emails, or modify external state.
LLM09 Overreliance. Users acting on incorrect outputs without verification. Surfaces as customer complaints when the model confidently asserts something wrong.
LLM10 Model Theft. Less relevant for vendor-API systems; more relevant for systems running self-hosted models.
A project’s exposure profile across the ten classes drives the insurance reserve sizing. A multi-tenant retrieval system with agentic tool use and customer-document ingestion has high exposure across LLM01, LLM06, LLM07, and LLM08; the reserve should be at the high end of the 5 to 10 percent range. A single-tenant internal tool with no tool use and no customer document ingestion has low exposure across most classes; the reserve can be at the low end.
How to size the insurance reserve
Three factors set the reserve.
Factor 1: Exposure profile. Multi-tenant systems, customer-facing systems, systems with agentic tool use, systems ingesting customer-provided content many have higher exposure. Internal-only, single-tenant, no-tool-use systems have lower exposure. Variance across the field is roughly 1.5x to 2x.
Factor 2: Regulatory profile. Healthcare, finance, legal, government systems carry higher remediation cost per finding because the disclosure and verification rigor is heavier. Variance is roughly 1.3x to 1.5x.
Factor 3: Public attention profile. Systems with high public visibility (consumer products, named-brand integrations) attract more adversarial researcher attention than internal systems. Variance is roughly 1.5x.
The sizing table.
| Project profile | Reserve size |
|---|---|
| Internal single-tenant tool, no tool use | 5 percent |
| Customer-facing system, single-tenant, light tool use | 7 percent |
| Multi-tenant SaaS with retrieval and tool use | 9 percent |
| Regulated multi-tenant production system, high public attention | 10 percent |
Above 10 percent is usually a signal that pre-launch red team or architectural review should absorb cost that would otherwise land on insurance. Below 5 percent is structurally underprovisioned for any system processing real production traffic.
The incident response playbook
When an adversarial finding lands, the playbook has six stages.
Stage 1: Triage (hours 0 to 24). Severity assessment; is this active exploitation, a researcher disclosure, a customer complaint, or a routine red-team finding. Initial scope estimate. Decision: emergency response, planned remediation, or accept as documented limitation.
Stage 2: Containment (hours 0 to 72 for emergencies, days 0 to 7 for planned). Compensating controls if the finding is active; a temporary prompt-level guard, a feature toggle, a rate limit. The goal is to bound the exposure window while remediation is built.
Stage 3: Eval extension (days 1 to 7). Add test cases that capture the finding to the eval suite. Verify the existing system fails the test. The eval extension is what prevents regression on the same finding class.
Stage 4: Remediation implementation (days 3 to 21). Implement the fix; prompt change, retrieval-side filter, output verification layer, model swap, whatever the finding class demands. Validate against the eval extension.
Stage 5: Disclosure and stakeholder communication (days 7 to 30). Internal post-mortem. Buyer-facing report covering what happened, what was fixed, what the new posture is. External disclosure if regulatory or contractual obligations apply.
Stage 6: Pattern analysis and playbook update (day 30+). Was this finding a one-off or part of a class? Should the eval suite be expanded to cover the class proactively? Does the architecture need a refactor? The pattern analysis is what turns the insurance line from a contingency into a learning loop.
The playbook is light-weight discipline. The cost is small. The cost of not having the playbook is each incident becoming a fire-drill rather than a planned operational response.
NIST AI RMF as governance backstop
The NIST AI Risk Management Framework provides a governance backstop above the insurance line. Where the insurance line covers the cost of remediating findings, NIST AI RMF covers the discipline of categorizing and tracking findings against a recognized framework.
Three useful primitives from NIST AI RMF for AI project budgeting. The MAP function; context establishment for AI risk; useful for sizing the insurance reserve against project-specific exposure. The MEASURE function; quantitative tracking of AI risk events; useful as the audit trail for insurance reserve consumption. The MANAGE function; structured response to identified risks; aligns with the incident response playbook above.
For regulated industries (healthcare, finance, government), NIST AI RMF compliance is increasingly expected by procurement. For unregulated industries, NIST AI RMF is useful as a defensible governance posture that shows the buyer the project takes risk management seriously, without committing to industry-specific compliance regimes.
Combined with the OWASP LLM Top 10 (which classifies the technical findings) and the insurance line (which budgets the remediation), NIST AI RMF gives the project a complete risk-management story: classification, governance, and budget. Procurement organizations responding to the trio increasingly recognize this as the table-stakes posture for serious 2026 AI engagements.
Frequently asked questions
How is the insurance line different from a contingency budget?
Contingency budgets are unscoped reserves that get cut on review. The insurance line is a named, sized, scoped reserve covering specific incident classes (jailbreak, hallucination, prompt injection, data leakage, security advisory) with a documented playbook and named owner. Named lines get defended; contingency gets cut. The discipline of naming is what makes the line survive procurement review.
What’s the empirical incident frequency on a serious 2026 AI engagement?
Two to five incidents per year of operation on a customer-facing enterprise system, weighted toward the first six months post-launch when adversarial users are most active. Internal tools see fewer (zero to two per year). Public consumer products see more (three to ten per year). The insurance reserve is sized against the expected count weighted by per-incident cost (1 to 4 engineering weeks).
Should pre-launch red team be in the insurance line or build budget?
Build budget. Pre-launch red team is part of build scope and should be sized as a discrete line in the build budget (typically 3 to 8 percent of build cost, depending on exposure). The insurance line covers post-launch findings specifically; work that emerges after launch despite pre-launch testing. The split matters because the two work-streams have different cost profiles, different owners, and different timing.
Does the insurance line cover hallucinations or just adversarial findings?
Both. Adversarial findings (jailbreaks, prompt injections) and hallucination remediation share the same cost shape (eval-extension, prompt or retrieval changes, verification) and the same playbook. The insurance line covers the long tail of post-launch findings, whether those findings come from researchers, customers, or internal monitoring.
How does this relate to the OWASP LLM Top 10?
The OWASP LLM Top 10 is the classification of finding types the insurance line covers. A project’s exposure profile across the ten classes drives the insurance reserve sizing; high-exposure systems (multi-tenant, agentic, customer-document ingestion) carry reserves at the high end of the 5 to 10 percent range. The insurance line is the budget; OWASP LLM Top 10 is the threat taxonomy.
What about cyber-insurance; does that cover AI incidents?
Conditionally and partially. Cyber-insurance policies in 2026 are still evolving on AI coverage. Most policies cover data breach and ransomware classes that AI incidents can trigger; few cover hallucination remediation or jailbreak engineering as a covered event. The AI insurance line in the project budget is the engineering-remediation reserve, distinct from cyber-insurance which covers liability exposure. Both should exist; neither replaces the other.
What’s the minimum viable insurance line for a small project?
A 5-percent reserve, a named owner, and the six-stage incident response playbook. The playbook can be a one-page document. The owner can be the lead engineer. The reserve can be a single line in the budget. Below that minimum, the project is structurally exposed to incident cost surfacing as scope creep.
How does this connect to the rest of the AI project economics framework?
The insurance line is one of the late-stage budget components in the project economics manifesto, sitting alongside the model-deprecation reserve, the model-upgrade re-eval line, and the post-launch operations retainer. Together these form the lifecycle cost surface that the 2018 software template did not have categories for. Naming each as a separate budget line is the difference between a defensible 2026 AI project budget and a budget that absorbs cost as recurring scope creep.
Key takeaways
- Most production AI project will have post-launch adversarial findings: jailbreaks, hallucinations, prompt injections, data leakage, security advisories. Two to five per year on a serious enterprise system.
- The insurance line is a 5 to 10 percent reserve, sized against the OWASP LLM Top 10 exposure profile, regulatory profile, and public attention profile of the project. It is named, scoped, and owned; distinct from contingency.
- The line covers engineering remediation: eval extension, prompt and retrieval changes, output verification layers, communication to the buyer. It does NOT cover PR, legal, regulatory penalties, pre-launch red team, or architectural rebuilds; those are separate lines.
- The incident response playbook has six stages: triage, containment, eval extension, remediation, disclosure, pattern analysis. Light-weight discipline; the cost of not having it is each incident becoming a fire-drill.
- NIST AI RMF combined with OWASP LLM Top 10 gives the project a complete risk-management story: classification, governance, and budget. Procurement organizations increasingly recognize this trio as table-stakes for serious 2026 AI engagements.
The insurance line is paperwork. The cost of having it is naming a reserve in the budget and an owner in the SOW. The cost of not having it is each adversarial finding surfacing as scope creep on a timeline the project did not plan for.
Arthur Wandzel
