Every AI feature in production fails in five ways, and the fallback path is what your users see when it does. If your PRD names an AI task and does not name a fallback for it, you have not specified the feature. You have specified the happy path — roughly half the contract. The other half — what the user sees when the model times out, refuses on content policy, returns malformed output, returns a low-confidence answer, or the vendor is down — is where founders quietly ship broken products.
This extends the idea validation playbook within the idea-to-product manifesto. The playbook describes the 1–2 week pre-build sequence; this article argues that the fallback row is the column most PRDs are missing.
Table of Contents
- The five failure modes every AI feature ships with
- The five fallback designs in your design space
- Which fallback for which failure mode
- Worked example — a contract review assistant
- The PRD discipline — one fallback row per task
- Frequently Asked Questions
- Closing
The five failure modes every AI feature ships with
A 2018 SaaS feature failed in two ways — the network was down or the database threw — both producing the same 500 page and the same retry-and-complain fallback. A 2026 AI feature fails in five ways, and treating them as one is the most common pre-build mistake. Anthropic’s Building Effective Agents treats failure as a first-order property of agent systems; the five modes below are the operational decomposition a founder needs at PRD time.
Timeout. The model takes longer than the surface tolerates. A chat answer at 4 seconds is fine; the same answer at 47 seconds is broken even though the bytes arrived. Timeout is the failure mode most often ignored at PRD time — the founder is thinking about quality, not latency.
Content-policy refusal. The model refuses. “I can’t help with that” arrives in place of the task output. Refusal is the vendor’s safety layer doing its job. Refusals cluster in predictable domains: medical, legal, financial, anything touching minors. A founder in those domains who does not pre-test refusal rates will discover them at customer call one.
Malformed output. The model returns text that does not parse. The PRD asked for JSON with three fields; the model returned four, or one nested wrong, or with markdown fencing the parser rejects. The Stack Overflow 2026 Developer Survey identifies malformed structured output as a top production complaint. Detection is clean — a schema validator catches it deterministically — but the founder still has to decide what to do when it triggers.
Low-confidence answer. The model returns parseable output, but the answer is wrong or evidently uncertain. The failure mode without a clean detection signal — the model does not raise an exception when it is wrong. Detection requires a proxy: LLM-as-judge, consistency check, self-critique pass. McKinsey’s State of AI implicates this mode in the pilot-to-production gap.
Vendor outage. The provider is down. OpenAI, Anthropic, and Google AI each post measurable outage minutes per quarter. A founder on a single provider has accepted that uptime as their own. Cheapest mode to design for (failover is a routing decision) and the one founders most often deprioritise on the assumption that the vendor SLA suffices. It does not — the product’s SLA is the founder’s.
The five modes are not equally probable. Timeout and malformed output hit every team in build week one. Refusal is domain-dependent. Low-confidence is the silent one. Vendor outage is the rarest but the most public.
The five fallback designs in your design space
A founder facing one of the five failure modes has five product moves available. These are not five tools — they are five distinct design choices with different costs, latency profiles, and user-trust implications.
Silent retry. Retry the call with exponential backoff. The user sees a spinner, then either the answer or escalation. Cheapest design but valid only when the failure is transient (timeout, vendor outage, sometimes malformed output). Invalid for refusal and dangerous for low-confidence (retrying returns a similar low-confidence answer with new bytes).
Cached response. Return a previously stored answer for a similar input. Requires a cache key and a defensible hit rate. Right when staleness is acceptable and freshness is not. Wrong when the user expects a personalised, fresh answer.
Rule-based path. Run a non-AI implementation: a regex, a decision tree, a SQL query, a hand-coded heuristic. The most defensive design — it removes all five failure modes at once by removing the model — but it pays for that defence with quality.
Human-in-loop. Escalate to a human reviewer. Output is a queue ticket, not a response. Right when the cost of a wrong answer exceeds the cost of latency (medical triage, legal review, payment dispute). Wrong when the user expects a real-time response.
Explicit “I don’t know.” Tell the user the feature cannot help, and ideally why. The design founders avoid because it feels like admitting failure — but the only fallback that scales without lying. “I cannot review medical advice — please consult your physician” is a usable surface. A retry that eventually returns a wrong answer is not.
The designs are a menu, not a hierarchy. The founder’s job at PRD time is to pick one per failure mode per surface.
Which fallback for which failure mode
| Failure mode | First-choice fallback | Why this pairing |
|---|---|---|
| Timeout | Silent retry → cached response | Most timeouts are transient; cached covers persistent slowness. |
| Content-policy refusal | Explicit “I don’t know” | Retry does not change policy. Honesty preserves trust. |
| Malformed output | Silent retry once → rule-based path | Retry catches sampling artifacts; rule-based catches persistent schema drift. |
| Low-confidence | Human-in-loop or “I don’t know” | The model cannot self-correct reliably; the answer must be removed from the user’s path. |
| Vendor outage | Failover to secondary provider → cached response | Multi-provider routing eliminates the failure at infrastructure layer. |
Three observations. The surface drives the pairing — the same failure mode on chat versus a batch report gets different fallbacks. A chat timeout needs “I don’t know” after 8 seconds; a batch report can run 90 seconds with no visible change. “I don’t know” appears in three rows — the most underused fallback in production and the only one that scales without misrepresenting the model’s capability. Rule-based path is the only fallback that survives a full model collapse — for anything compliance-sensitive, it is the audit trail.
Worked example — a contract review assistant
A solo non-technical founder is building a feature that reviews vendor contracts and flags risky clauses. Primary model is Claude Opus 4.8; failover is GPT-5. A lawyer or paralegal uploads a PDF and gets a report within 60 seconds. One feature, five fallback rows.
Row 1 — Timeout. Budget: 45 seconds end-to-end. Model gets 35; parser and rendering get 10. At 35s, abort and retry once on the secondary provider. If the secondary also times out, return “The review is taking longer than expected. We have queued it for asynchronous processing and will email the report when ready.” Cached response is not used — each contract is unique.
Row 2 — Refusal. Legal review is rarely refused, but contracts referencing NSFW-content licensing, weapons exports, or sanctioned jurisdictions can trigger one. Fallback: explicit “I don’t know” — “This contract contains content our review system cannot process. Please consult counsel directly.” Log the refusal. No retry; no human-in-loop in v1.
Row 3 — Malformed output. Detection: a JSON schema validator. Expected shape has three top-level fields — a clauses array, a risks array, a summary string. On validation failure, silent retry once with a stricter prompt. On second failure, rule-based path — a regex extractor flagging high-risk patterns (indemnification, unlimited liability, perpetual term, automatic renewal). Output labelled “Basic review only — for detailed review please retry.”
Row 4 — Low-confidence. Detection: a self-critique pass. A cheaper model (Claude Haiku 4.5) rates each flag’s confidence 1–5. A score below 3 triggers the fallback: the clause surfaces with a “Low confidence — recommend human review” badge. Output is delivered; the badge is visible.
Row 5 — Vendor outage. Detection: 5xx response or connection refused. Silent failover to the secondary provider. If both are down — rare but observed during cross-provider incidents — surface “Our review service is temporarily unavailable. We will resume processing within 30 minutes.”
The PRD now has a section the 2018 feature-list shape would have skipped:
Feature: Contract clause review. Task: Flag risky clauses in an uploaded contract within 60 seconds. Fallback rows:
- Timeout (45s): secondary-provider retry → “I don’t know” with async queue.
- Refusal: explicit “I don’t know” with counsel-referral message.
- Malformed output: stricter-prompt retry → rule-based regex extractor (labelled “basic review”).
- Low-confidence: self-critique flag → “low confidence” badge.
- Vendor outage: provider failover → “I don’t know” with retry-in-30-minutes message.
Each row names a detection signal and a product behaviour. That is what the implementation team builds against. A team handed the same feature without these rows will silently substitute its own choices.
The PRD discipline — one fallback row per task
The eval set is one half of the falsifiable AI PRD; see the eval-first PRD for that argument. The fallback row is the other half.
The discipline is one sentence: every named AI task in the PRD has a fallback row covering all five failure modes, with detection signal and product behaviour for each.
A task without a fallback row is one whose failure modes the founder has not specified — so the implementation team will specify them, at code-review time, with whichever pattern feels right that afternoon. The cumulative drift surfaces in production.
The fallback row is where the “AI-powered” phrase decomposes into something concrete. A PRD that says “AI-powered contract review” without fallback rows is one the vendor can interpret loosely and the founder cannot enforce. The five rows above are a contract.
A common objection: “this is over-engineering for an MVP.” It is not. The five rows can be authored in 30 minutes per task by a non-engineer founder using the matrix above. Authoring cost is dwarfed by the cost of the first production incident the founder cannot explain to a customer. The same pattern shows up in vendor proposals that claim “production-ready” without specifying the failure-handling surface — the phrase is unfalsifiable absent the fallback rows.
The discipline is also a procurement filter. A vendor shown a PRD with five fallback rows per task responds in one of three ways. Good: “we will run a baseline against these five rows in week one.” Acceptable: “we will design two at MVP and three at v2 — here is the trade-off.” Disqualifying: “the model rarely fails, so the fallback is overhead.” The third response identifies a vendor who has not shipped production AI long enough to know better.
Frequently Asked Questions
What is an AI no-AI fallback? A no-AI fallback is the product behaviour your feature exhibits when the AI call fails — a non-AI code path the user sees instead of the model output. It can be a cached response, a rule-based heuristic, a human reviewer, an “I don’t know” message, or a silent retry. Every shipping AI feature needs at least one fallback per failure mode it can hit in production.
Do I really need all five fallback designs, or can I pick one? You need at least one fallback for every failure mode your feature can hit. The five designs are a menu, not a checklist. A simple chat surface might use only “silent retry → I don’t know” for all five modes. A regulated domain will likely use four of the five.
Can the same fallback work for multiple failure modes? Yes. The most common reuse is “I don’t know” — it covers refusal, persistent timeout, persistent malformed output, and persistent vendor outage with one surface treatment. The PRD still names each mode separately because the detection signal differs.
How do I detect a low-confidence answer? Three patterns work in production. Self-critique: a second cheaper-model pass rates the primary output’s confidence. LLM-as-judge: a third-party grading model scores against a rubric. Downstream consistency: validate against a known-good source (a database, a retrieval index, a rules engine). Pick one per task.
Is the rule-based path really worth building if the model usually works? For low-stakes features, no — “I don’t know” is cheaper to ship and more honest. For compliance-sensitive features, the rule-based path is the audit trail. When a regulator asks why a customer was advised a certain way, “the model said so” is not an answer.
Should I show users when the fallback fires? For most failure modes, yes. Trust degrades faster from silent low-quality output than from explicit labelling. The exception is silent retry on a transient timeout that recovers on the second attempt. The rule: if the output’s quality differs from the happy path, the user sees a label.
How do I budget fallback design effort against the rest of the build? A reasonable allocation is 20–30% of feature-level engineering effort in the first 6 weeks. Less and the fallbacks are afterthoughts that break in production. More and the founder is over-investing before the happy path is validated against an eval.
Does the fallback design change when the model gets better? The rows survive model upgrades. Detection signals (schema validators, confidence proxies, status checks) and product behaviours (cached response, rule-based path, “I don’t know”) are model-independent. What changes is the rate at which each fallback fires.
What is the most underused fallback in MVPs we see? Explicit “I don’t know.” Founders prefer silent retry or cached response because admitting non-knowledge feels like product weakness. Empirically the opposite holds — users tolerate honest non-knowledge but punish silent low-quality answers. An MVP with no “I don’t know” surfaces is almost certainly over-promising.
Closing
Every AI feature ships with five failure modes whether or not the founder names them. The fallback path is what the user sees in each case. The PRD discipline is one fallback row per task, naming detection signal and product behaviour for all five.
A feature-list PRD without fallback rows is a happy-path contract. The implementation team fills in the gaps silently, and the founder discovers them at customer call one — usually after the first refusal, the first cross-provider outage, or the first malformed output that reached a paying user.
If the spec on the table does not include fallback rows, send it back with one sentence: add the fallback rows, or the feature is not specified.
Get the AI MVP scoping worksheet — a one-page template with an eval row and a fallback row per task that a non-engineer founder can take into a vendor conversation tomorrow.
Dirk Jan van Veen, PhD
