You cannot optimize for capability, cost, and latency at the same time. Every AI feature lives at one corner of that triangle — best capability, lowest cost, or fastest latency — and the founders who ship in 2026 are the ones who pick the corner before they start building, not after. Founders who refuse to pick default to “all three” and ship none.
This is the decision frame for a non-engineer founder scoping an AI feature. It sits within the eval-first build playbook and the idea-to-product manifesto.
Why founders want all three (and why that fails)
The founder instinct is universal: build the smartest possible AI, run it for cents per query, have it answer in under a second. Investor decks promise it. Vendor marketing implies it. Demos at conferences feel like it.
None of that is production. In production, the corners pull against each other physically. A frontier-tier model costs roughly an order of magnitude more per token than its workhorse sibling — and the workhorse is also 2–3× faster. Reasoning modes that lift capability add latency. Caching schemes that cut cost trade memory for staleness. No engineering trick delivers best-capability, lowest-cost, fastest-latency on the same workload.
Teams that try to optimize all three end up mediocre on every axis: cheaper than the prestige option but still expensive enough to spook finance; faster than the reasoning model but slow enough that users bounce; smart enough to demo but not smart enough that anyone pays. The triangle is a constraint geometry to pick a position inside, not three KPIs to maximize.
The three corners of the triangle
The corner is the dimension you optimize ruthlessly. The other two are constraints you accept inside a budget.
Capability corner
You take the smartest available model for the task class, regardless of cost or speed. This is where high-stakes reasoning workloads live: contract drafting, clause extraction, clinical decision support, multi-step research agents, financial analysis read by a board. A 30-second response is acceptable if the answer is correct; a 3-second response that is subtly wrong is worse than useless.
Profile: frontier-tier model (Claude Opus 4.8, GPT-5, Gemini 2.5 Pro); cents to dollars per task; latency tolerated in seconds; quality bar measured by domain-expert evals.
Cost corner
You take the cheapest model that clears your quality bar at the volume you need. This is where high-throughput, lower-stakes workloads live: bulk classification, retrieval ranking, content tagging, customer-support triage, ad-relevance scoring, batch summarization across millions of documents. Getting it 95% right on 10 million inputs may beat 99.5% right on 100,000.
Profile: workhorse or lightweight model (Claude Haiku 4.5, GPT-5 nano, Gemini 2.5 Flash); sub-cent per-task cost; 1–3 second latency, batched where possible; quality measured statistically across the dataset.
Latency corner
You take the fastest model that clears a minimum acceptable quality bar, sized for a user who feels every millisecond. Voice agents, autocomplete, sales-call copilots, fraud decisioning at checkout, in-game dialogue, customer-service chat. A 400 ms response feels smarter than a 4-second response even when the slower model is technically more accurate.
Profile: workhorse or lightweight model with aggressive caching and prompt-prefix sharing; sub-second p50, sub-2-second p95 (the tail matters more than the mean); middle cost range; quality measured by completion rate and user-perceived satisfaction.
The three decision rules
The corner is not a guess. It is decidable from the product, the user, and the volume.
Rule 1: pick the corner that is the user’s actual pain
Ask one question. What would your user pay more to fix?
Pay double for a 5% more accurate answer: capability corner. Switch providers to save 30% on a monthly bill: cost corner. Pay double for a response that arrives a second sooner: latency corner.
Real users will not answer this question explicitly. You read it from behavior. The legal partner re-runs the analysis manually because the AI draft is not trustworthy: capability. The ops manager dropped the tool because the monthly bill outgrew the team’s budget: cost. The sales-call user closed the tab because the assistant interrupted the conversation: latency.
Rule 2: the other two corners become budget constraints, not goals
Once the corner is picked, the other two become ceilings.
If you pick the capability corner, set a latency ceiling (“p95 under 20 seconds”) and a cost ceiling (“under $0.50 per task at target volume”). The ceiling is the worst acceptable value, not the value you optimize toward. Stop tuning the constrained axes once you are inside the ceiling; spend every remaining optimization budget on the corner.
This is the discipline most teams skip. They pick the corner intuitively, then keep trying to optimize the other two, and end up excellent at nothing. Budget constraint is a hard line: under the line is good enough; over the line is unshippable.
Rule 3: re-evaluate the corner every quarter
The corner moves as the product matures.
A new consumer chatbot lives at the latency corner — engagement is fragile, users bounce on slow responses. Six months in, retention is solved and the bottleneck shifts: users want the assistant to do harder things. The corner moves to capability.
A bulk classification product launches at the cost corner — volume is the thesis, capability is “good enough.” Two quarters in, an enterprise customer asks for compliance-grade audit trails, and per-decision capability matters more than per-task cost. The corner moves.
Quarterly re-evaluation is not optional. Calendar the review. Re-run the eval set. Ask the user-pain question again. Most teams miss corner shifts because they treat the original choice as permanent, then watch a competitor that read the shift earlier overtake them.
The four hidden gotchas
The framework is clean; the implementation has four traps every founder steps into at least once.
Gotcha 1: latency tail (p95 / p99), not the mean
The user does not feel mean latency. The user feels the tail. A voice agent with a 600 ms mean and a 4.2-second p95 is a broken product even though the dashboard says it is fast. One in twenty turns hangs for four seconds, and the memory of that one turn poisons the perception of the nineteen fast ones. Measure p50, p95, and p99. The latency corner is decided by p95 and p99, not p50.
Gotcha 2: cost-per-task, not cost-per-token
Token pricing is misleading because agents loop. A single user request to a multi-step agent might trigger 12 model calls, 4 tool invocations, retrieval over 50 documents, and a final synthesis pass. The token count blows up an order of magnitude beyond the user-visible request. Cost-per-token at $5 per million looks cheap; cost-per-completed-task at $0.40 per user query is the number that decides whether the unit economics work. Build your cost dashboard around finished tasks. The cost-per-query unit economics frame walks the accounting.
Gotcha 3: capability ceiling, not capability floor
You need to know where the model breaks, not where it works. Every frontier and workhorse model handles the easy cases. The cases that matter are the hard ones — the contract clause with ambiguous language, the clinical case with three overlapping presentations, the financial filing with one footnote that changes the interpretation. Build the eval set against the ceiling: curate the 20–30 hardest cases your product will see in production, run them weekly. If the ceiling rises after a model update, drop a tier. If it falls, you have an early warning before users notice.
Gotcha 4: observability cost is part of the cost corner
Evals, logs, traces, replay storage, monitoring — none of it is free, and the bill arrives where you optimize hardest. The capability corner runs more evals; the cost corner runs more logs; the latency corner runs more traces. Observability typically runs 20–35% of the inference bill. Fold it into your cost ceiling from day one. The hidden cost of AI evals walks the budget line items most teams miss.
Worked example: an AI legal-doc reviewer
A non-technical founder is building an AI tool that reviews commercial contracts and flags risky clauses. The user is in-house counsel or a transactional partner at a small firm. Walk the triangle.
Pick the corner (Rule 1). A miss on a high-risk clause costs the user a deal or a lawsuit. A 30-second wait on a contract that takes a human partner three hours to review is invisible. A $4 per-contract cost on a deal worth $500K in fees is rounding error. The user would pay double for higher accuracy on hard clauses. Capability corner.
Set the budget ceilings (Rule 2). Cost ceiling: under $5 per contract review at target volume. Latency ceiling: p95 under 60 seconds. Both have headroom; inside them, optimize capability only.
Build the eval against the ceiling (Gotcha 3). Curate 40 contract examples that exercise the failure modes that matter — vague indemnification language, IP-assignment clauses with hidden exclusions, multi-jurisdiction governing-law clauses, change-of-control triggers buried in subordinate clauses. Run the eval against Claude Opus 4.8, GPT-5, and Gemini 2.5 Pro every two weeks. The model that wins on the hard cases is the production pick. The eval-first PRD frame covers the day-one rubric.
Track cost-per-task (Gotcha 2). A multi-clause review fires several model calls — segmentation, per-clause classification, risk scoring, cross-clause consistency check, final synthesis. The token bill compounds. Build the dashboard around per-contract cost. If per-contract cost creeps toward the $5 ceiling, cache prompt prefixes or batch the classification step — do not drop the model tier and compromise the corner.
Re-evaluate quarterly (Rule 3). Quarter three, the firm asks for a “quick scan” mode to triage 30 contracts in an hour. That is a second product at the latency corner — same model family, different tier, different ceilings. One triangle, two product positions, two eval sets.
How the corner choice shapes your eval set
The corner decides what your eval set measures.
| Corner | What the eval measures | Cadence |
|---|---|---|
| Capability | Quality on the hardest 20–30 cases your product sees | Weekly + on every model update |
| Cost | Statistical quality across a representative production sample | Monthly + on every cost-driving change |
| Latency | p50 / p95 / p99 across realistic user sessions | Continuous, with tail-regression alerting |
A capability-corner eval that measures average quality is the wrong eval. A cost-corner eval that measures p99 latency is the wrong eval. The corner picks the metric; the metric picks the model; the model picks the cost. The AI agency quality system walks how three eval cadences run alongside each other when multiple features sit at different corners.
Next step
The corner is the first scoping decision in an AI feature spec — before the model family, before the architecture, before the budget. Pick the corner that names your user’s pain, set the ceilings on the other two, and write the eval set against the corner.
If you are scoping an AI feature now, download the AI MVP Scoping Worksheet — a four-page PDF that walks your idea through the triangle, the eval-set draft, and the cost-per-task projection in under an hour.
Download the AI MVP Scoping Worksheet →
FAQ
What is the AI capability-cost-latency triangle?
A decision frame for scoping AI features. You can optimize ruthlessly along one of three axes — capability (best quality), cost (lowest unit price), or latency (fastest response) — but not all three at once. Pick the corner first, then treat the other two as budget constraints with hard ceilings.
How do I know which corner to pick?
Ask what your user would pay more to fix. If they would pay double for a more accurate answer, you are at the capability corner. If they would switch providers to save 30% on cost, the cost corner. If they would pay double for a response that arrives a second sooner, the latency corner. Read the answer from user behavior, not from a survey.
Can I sit in the middle and optimize all three a little?
Not effectively. Teams that try produce a product that is mediocre on every axis — too expensive for the cost-sensitive market, too slow for the latency-sensitive market, not accurate enough for the capability-sensitive market. The triangle is a constraint geometry, not a Cartesian plane.
How often should I re-evaluate which corner I am at?
Once per quarter at minimum, and every time a major user-behavior shift happens. A consumer chatbot starts at the latency corner and may move to capability once retention is solved. A bulk-classification product starts at the cost corner and may move toward capability when an enterprise customer asks for higher-stakes outputs.
Why cost-per-task and not cost-per-token?
Because agents loop. A single user request to a multi-step agent can trigger many model calls, tool invocations, and retrieval passes. Token pricing looks cheap but compounds into a per-task cost that breaks the unit economics. The honest unit is cost per finished task — the user pressed a button, the system reached an end state.
What is the difference between capability ceiling and capability floor?
The floor is what the model handles correctly on easy inputs — every frontier and workhorse model clears it. The ceiling is where the model starts failing on hard inputs. The ceiling matters because that is where the user catches the model being wrong. Build your eval set against the ceiling.
How do I measure latency the right way?
Track p50, p95, and p99. The mean is misleading — it hides the tail, and the user experiences the tail. A voice agent with 600 ms mean and a 4-second p95 feels slow even when the dashboard says it is fast.
Is the latency corner only for voice agents?
No. Latency-corner products include voice agents, autocomplete, sales-call copilots, fraud decisioning at checkout, in-game dialogue, and customer-service chat. Anywhere the user is watching a cursor blink, latency is the corner.
Does fine-tuning let me escape the triangle?
No. Fine-tuning improves capability or reduces cost within a model family — it shifts your position on the corner you picked, but does not eliminate the constraint. The triangle is the constraint geometry of inference itself.
How does the corner choice connect to my eval set?
The corner decides what the eval set measures. Capability-corner evals measure quality on the hardest 20–30 cases, run weekly. Cost-corner evals measure statistical quality across a representative sample at production volume, run monthly. Latency-corner evals measure p50 / p95 / p99 across real user sessions, run continuously. Pick the corner; the eval set follows.
Dirk Jan van Veen, PhD
