Every shipping AI feature has a hallucination rate. Most specs do not name it. That gap is the single most expensive omission in a 2026 idea-to-product engagement. A team that builds against an unstated quality bar will hit it, declare success, and ship a product the founder later discovers fails on the inputs that matter most. The fix is a budget written into the PRD: a target rate per failure tier, the eval method that measures it, and the production burn rule that catches drift. The format below is two paragraphs in the spec and one clause in the SLA. The founder writes the first version. The engineer fills the rates. The vendor signs both.
This article builds on the eval-first build playbook, part of the idea-to-product manifesto. The vocabulary lives in AI hallucination explained; the eval contract this budget references lives in the eval rubric template every non-technical founder should ask for. The engineering work this budget commissions is described in AI model evaluation testing services.
Scope
A founder-facing operating doc — the contract format, not the eval engineering. PRD lines and SLA clauses are copy-pasteable. Example tier ranges are starting points; the budget for any specific product depends on user trust, regulatory exposure, and commercial sensitivity, and must be set by the founder with their team and (where relevant) counsel.
Why “no errors” is the wrong target
The founder’s reflex is to ask the team to eliminate hallucination. The serious engineer’s reflex is to nod and ship against a bar nobody named. Both are mistakes.
Hallucination has no zero-rate fix. Vectara’s Hughes Hallucination Evaluation Model reports frontier models in 2026 hallucinating on 1.5–4% of summarization tasks, down from 5–15% in 2023. The curve is bending, not converging to zero. OpenAI’s September 2025 research post frames the residual rate as a property of the next-token-prediction objective itself. The floor is load-bearing.
A vague target is operationally identical to no target. When the spec says “minimize hallucinations,” the team picks a bar in private — usually the bar the model passes on the first 50 prompts they tested. The first time the realized rate rises after a model upgrade or retrieval regression, there is no contract to enforce.
The error-budget framework, ported from SRE, fixes this. The Google SRE Book established the move twenty years ago: define an objective, derive an error budget, gate releases against it, freeze deploys when the budget is burned. The AI version is the same act, with rates sliced per failure tier and the eval suite doing the measurement. The budget is not a goal. It is a contract about what counts as shipped.
The three failure tiers
A single hallucination rate is the wrong unit of decision. Two errors of equal frequency can carry wildly different commercial consequences. The budget slices the rate by impact tier.
Tier 1 — Recoverable
The user notices the error in the same session and corrects it with one action. No external system was touched, no decision was committed. Examples: a chatbot mislabels a support ticket the user reclassifies; a coding assistant proposes a function signature the developer rejects.
Commercial cost: friction. Measured in lost trust, not lost dollars. The product tolerates the highest rate.
Tier 2 — Inconvenient
The error escapes the session and reaches another system — an email is sent, a record is created, a workflow runs on the wrong input. The user discovers it later. Examples: a CRM bot writes the wrong industry into a lead record; a meeting-notes AI summarizes the wrong action item.
Commercial cost: operational. Time-to-discover plus time-to-remediate, plus the trust hit. A meaningfully lower rate than Tier 1.
Tier 3 — Dangerous
The error reaches an irreversible external commitment, a safety threshold, or a regulated decision. Money moves, a contract is honored, a person receives guidance with health, legal, or financial consequence. Air Canada (BC CRT, Feb 2024) and Mata v. Avianca (SDNY, 2023, fabricated case citations) sit here. Examples: a financial bot quotes a settlement amount the company must honor; a medical-summary AI emits a wrong dosage that reaches the chart.
Commercial cost: catastrophic — legal exposure, regulatory fines, lost customer, personal harm. The product tolerates a rate that is, by orders of magnitude, the lowest of the three.
A budget that does not separate the tiers implicitly sets Tier 3 at the same rate as Tier 1 — the operating mode of every team that shipped an embarrassing AI failure in 2024 and 2025.
Example rate ranges per tier
Working ranges, not absolutes. The founder’s starting point with the engineering team.
| Tier | Pre-launch target | Production alert | Production hard freeze |
|---|---|---|---|
| 1 — Recoverable | ≤ 5% on representative eval set | Burn rate above 8% over rolling 7 days | Above 12% over 7 days |
| 2 — Inconvenient | ≤ 1% on representative eval set | Burn rate above 1.5% over 7 days | Above 2.5% over 7 days |
| 3 — Dangerous | Zero observed in eval suite of N ≥ 500 | Any single observed incident | Two incidents in 30 days |
Two anchoring points:
- Vectara HHEM rates put frontier models in 2026 between 1.5% and 4% on a benchmark closer to Tier-2 difficulty. A 1% Tier-2 budget means the production system must do better than the bare model — which is why retrieval grounding and structured outputs are not optional for Tier-2-heavy products.
- Stanford HAI’s study of purpose-built legal AI tools (Magesh et al., arXiv:2405.20362, May 2024) measured 17–34% hallucination rates on case-citation generation — a Tier-3 task — on tools sold to law firms. That is the cost of shipping without a Tier-3 budget enforced by an architecture that cannot produce a fabricated citation.
The point is not to publish defensible numbers for every domain. It is to force the conversation in the spec. “What’s our Tier 3 number?” is a question every founder can ask. The answer is a number, and the number is in the PRD.
Writing the budget into a PRD
Two paragraphs in the spec, one clause in the SLA. Replace the bracketed values; the structure is invariant.
PRD section: Hallucination budget
Hallucination budget. This feature commits to the following hallucination rates, measured by the eval suite defined in §[X] of this spec.
- Tier 1 (recoverable in-session): ≤ [5]% on the launch eval set. Alert if rolling-7-day rate exceeds [8]%; hard freeze on new releases above [12]%.
- Tier 2 (escapes session): ≤ [1]% on the launch eval set. Alert above [1.5]%; hard freeze above [2.5]%.
- Tier 3 (irreversible or regulated): zero observed in the launch eval set of N ≥ [500] inputs. Single Tier-3 incident in production triggers a P0; two incidents in 30 days trigger a hard freeze and a tier-architecture review.
Burn rule. A model update, prompt revision, corpus change, or material code change does not promote to production until the eval suite re-runs and realized rates remain inside budget. The budget is the gate; passing the previous release is not.
SLA clause: Hallucination budget enforcement
§[X] Hallucination budget. Vendor commits to the per-tier rates and burn rules in §[X] of the PRD. Vendor publishes the realized per-tier rate to Founder on a rolling 7-day basis. Tier-3 incidents are reported within 24 hours. A hard-freeze breach pauses any in-flight billing milestone until realized rates return inside budget. Acceptance of any milestone is conditional on the eval suite passing the per-tier budget.
The budget references the eval suite, it does not embed it. The suite lives in the rubric (format in the eval rubric template); the budget tells the rubric what rate counts as acceptable. The SLA clause makes the budget commercially load-bearing — without it the budget is a wishlist; with it the budget is the gate on payment.
Measuring the budget pre-launch
The wrinkle: hallucination rates are estimated from a sample, and sample size determines how trustworthy the estimate is.
Build the eval set per tier. The founder ensures the set contains representative Tier-3 inputs — high-impact, low-frequency — not just the easy Tier-1 majority. A 200-input set with zero Tier-3 inputs cannot estimate the Tier-3 rate. The Tier-3 slice is the rarest and most expensive to label, which is why teams quietly skip it. The PRD line forces the slice into existence.
Size the suite for the Tier-3 rate. For Tier 1 and Tier 2, 100–200 inputs give reasonable estimates. For Tier 3, the credibility of “zero observed in N inputs” scales with N. A practical floor is N = 500; regulated domains go to 2,000 or more. Rule of three plus headroom: zero failures in 500 is consistent with a true rate up to roughly 0.6%; zero in 2,000 tightens that to about 0.15%. The team does the math; the spec line forces them to either deliver the inputs or argue for a different N.
Use an LLM-as-judge with human tie-breaks on ambiguity. The judge is a frontier model (Claude Opus 4.8, GPT-5, or Gemini 2.5 Pro in 2026) prompted with the rubric. Where the judge is uncertain or Tier-3 stakes are high, a human reviewer breaks the tie.
Freeze the suite before recording the launch number. Adding inputs after seeing the realized rate contaminates the suite. Lock, run, record. Future suite changes are documented and dated.
Measuring the budget in production
The pre-launch number is the gate. The production number is the operating contract.
Sample, do not grade everything. A typical setup grades 1–5% of Tier-1 traffic, 5–20% of Tier-2, and 100% of Tier-3, maintaining rolling-rate estimates per slice. Alerts trigger when the rolling rate crosses the warning band; hard freezes trigger at the freeze band — identical mechanic to an SRE burn alert.
Detect drift, not just rate spikes. Drift is the slow movement of inputs away from the eval-set distribution. The rate looks fine for a week, then the user base shifts and creeps up. A drift monitor — cluster comparison or rising abstention rates — surfaces the regime change before the burn rule triggers.
Re-run the suite on every model change. Frontier vendors update on a multi-week cadence; switching from Claude Opus 4.8 to 4.8 is a routine vendor decision and a non-routine budget event. The SLA clause makes the re-run a condition of any model swap.
Publish the realized rate. A budget the vendor measures privately is a budget that does not exist. A weekly report — three numbers, one per tier — to the founder is the minimum operating loop.
Who enforces what
The budget is a joint artifact. The split decides whether it survives contact with the build.
The founder owns:
- Tier definitions. What counts as “irreversible” is a commercial call. The same output is Tier 2 in a free-tier app and Tier 3 in a financial-services product. Set in plain English before the engineer estimates.
- Per-tier rate decisions. “Are we okay with 1% on Tier 2?” is a domain question. The engineer advises on what’s achievable at what cost; the founder owns the number.
- The release gate. When the realized rate is over budget, the founder says “we do not ship.” Every team will, under pressure, propose shipping with caveats. The PRD line and SLA clause make the gate enforceable.
- The weekly report. Three numbers, three minutes.
The engineer owns:
- Suite construction and measurement. Eval set, judge instrumentation, production sample, rolling rates, drift monitors. The SLA makes them billable.
- Architecture that hits the rate. Once the budget is named, the engineer picks the guardrail stack — retrieval grounding, structured outputs, refusal triggers, human-in-loop. The four-layer stack is covered in AI hallucination explained.
A worked example: a clinical-summary AI
A healthcare product summarizes patient encounters for a clinician’s next visit. The summary appears in-app; a clinician reviews and accepts before any record is touched.
- Tier 1: a wrong minor symptom the clinician corrects in review.
- Tier 2: a quote attributed to the wrong family member; clinician notices later, corrects.
- Tier 3: a fabricated medication or dosage the clinician accepts under time pressure.
Budget in the PRD:
- Tier 1: ≤ 6% on 200 representative encounter notes; alert above 9%, freeze above 14%.
- Tier 2: ≤ 1% on the launch eval set; alert above 1.5%, freeze above 2.5%.
- Tier 3 (medication or dosage fabrication): zero observed in N = 1,000 encounter notes. Single Tier-3 incident is a P0; two in 30 days triggers a hard freeze and architecture review.
The Tier-3 budget forces the architecture. Medications and dosages are not generated — they are extracted from a structured-extraction pass and validated against source notes before they appear. The generation model never decides a medication should appear. The Tier-3 rate is zero because the architecture cannot produce a fabricated medication, not because the model promises not to.
The Tier-2 budget shapes the review UI: attributions, dates, and amounts are highlighted for explicit confirmation before save. Tier 1 is the residual — high because cost is low and recovery is the same screen.
The SLA references the budget. The suite measures it. The product ships when the suite passes. That is the budget operating as a control loop.
FAQ
What is a hallucination budget?
A written, per-tier acceptable hallucination rate in the AI product’s spec. It names the target rate, the eval method that measures it, and the production rule that catches drift — enforced by the vendor SLA. The AI-feature equivalent of an SRE error budget.
Why three tiers and not one?
A product cannot operate against a single rate because failures of equal frequency have unequal commercial cost. A 1% rate on in-session corrections is fine; a 1% rate on irreversible regulated decisions is unacceptable. Three tiers is the minimum slicing that lets the founder match rates to cost structure.
Where do the example rate ranges come from?
The Tier-2 anchor (≤ 1%) sits below the Vectara HHEM range of 1.5–4% for frontier 2026 models on summarization; hitting it requires a guardrail stack, not the bare model. The Tier-3 anchor (zero in N ≥ 500) is the rule-of-three floor — zero failures in 500 trials is consistent with a true rate up to roughly 0.6%. Starting points; regulated domains tighten them.
How is this different from an SLA uptime target?
Uptime SLAs target binary outcomes — request succeeded or failed. Hallucination is a continuous-quality failure mode invisible to status codes. The budget adapts the SRE mechanic to a domain where the failure is graded by an eval suite, not counted.
What if the vendor refuses to commit to a per-tier rate?
That refusal is the signal. A serious vendor negotiates the numbers and ships the SLA clause. A vendor that refuses to commit is signaling either no eval engineering practice or a desire to retain the option to ship anything and call it done.
How big does the Tier-3 eval set need to be?
To credibly assert “zero observed at a true rate below 1%,” N ≥ 300 is a starting point; below 0.5% needs N ≥ 600; below 0.1% needs N ≥ 3,000. Rule of three is the back-of-envelope. The founder requires the team to state N; the team defends it.
Does a higher hallucination budget mean the product is worse?
Not necessarily. A higher Tier-1 budget on a product where recovery is immediate is the right commercial choice. Pushing Tier 1 from 5% to 2% may be a poor trade against pushing Tier 3 from 0.3% to 0.05%. The budget makes the trade visible.
How often should the budget be re-set?
Tier definitions are stable — they reflect commercial structure. Rates are revisited at major releases, after material model upgrades, and when the user distribution shifts. Serious products re-run the suite on every model change.
What is the relationship between the budget and the eval rubric?
The rubric defines what correct means per task. The budget defines what rate of incorrectness is tolerable per tier. Separate artifacts that reference each other.
Next step
The hallucination budget is one of four artifacts a defensible AI MVP spec carries — alongside the eval contract, the rubric, and the architecture decision record. SFAI Labs’ idea-to-product engagement publishes all four in the first 14 days.
Download the AI MVP Scoping Worksheet — a founder-facing checklist with the templates above ready to copy. The first thing a non-engineer founder uses to interview a build team, and the first thing a build team fills in to commit to a quality bar.
Dirk Jan van Veen, PhD
