The standard AI ROI calculator; cost divided by hours-saved times salary; is wrong in a way that reliably under-prices the projects that compound and over-prices the ones that do not. It misses four AI-specific dynamics that determine whether the investment was worth making, and it produces a number that survives a budget review while killing the projects most likely to matter eighteen months out.
This piece names what those calculators get wrong and proposes a 4-component value model; capability earned, time-to-value, downside-risk-reduced, optionality-created; that finance teams can use without inventing fake precision. It is a spoke under the AI project economics manifesto, which argues for the broader economics framework this model implements.
What standard ROI calculators measure
The canonical AI ROI calculator looks like this:
ROI = (hours_saved_per_month × loaded_hourly_rate × 12 − annual_project_cost) / annual_project_cost
It is a productivity-substitution model. It assumes the AI feature replaces a known volume of human time at a known cost, and that the saved time is fungible; a saved hour is the same as a paid hour. The output is a percentage, comparable to a CapEx return, and it slides cleanly onto a budget review slide.
The model is not stupid. It is the right model for a narrow class of AI applications: well-scoped automation of a high-volume, low-stakes task with a clear human baseline (data entry, simple summarization, routine triage). For those projects the calculator is approximately correct.
For everything else; agents that change what the team can attempt, AI features whose accuracy threshold determines whether they ship at many, projects whose payoff is the eval library and prompt registry rather than the launched feature itself; the productivity-substitution model is the wrong shape and produces the wrong number. The math is fine. The model is not.
The four AI-specific dynamics they miss
Eval threshold effects
AI features ship in a binary on the dimension that matters most. Either the system passes the eval threshold the buyer locked, or it does not. A project at 0.78 weighted score on a 0.80 threshold has produced approximately zero deployable value; a project at 0.81 has produced enormous value. The standard ROI calculator cannot represent this curve. It treats AI quality as continuous and the value as proportional, when in reality the value is a step function around the threshold.
The implication: a project’s ROI curve has a sharp discontinuity that legacy ROI math smooths over. A 5 percent gain in eval score from 0.76 to 0.81 might be the entire value of the project. A 5 percent gain from 0.85 to 0.90 might be marginal. Linear ROI cannot tell the difference.
Model upgrade resets
The model that the project shipped on is not the model the project will run on six months from now. Anthropic, OpenAI, and Google ship non-trivial upgrades roughly quarterly, and each upgrade triggers re-evaluation work, regression triage, and possible re-architecture. The legacy ROI calculator does not have a line for “the value will be re-tested when the underlying engine changes,” because legacy software does not have engines that change.
The implication: AI project ROI is not a fixed annuity. It is a curve that resets at most model upgrade. Some projects come out of those resets stronger (because the new model unlocks features that were impossible at lower capability); others come out weaker (because regressions take months to fix and the eval bar drops in the interim). A static ROI number cannot price either outcome.
Regression cost
When an AI feature regresses; drops from 0.81 back to 0.74 because of a model swap, content drift, or a prompt change with unintended downstream effects; the cost of the regression is not “we lost some productivity.” The cost is “the feature is now below threshold and is producing customer-visible failures, which damages trust in the product the feature is embedded in.” That trust is a non-linear asset. Trust lost on the AI feature spills into trust lost on the surrounding product.
The implication: AI features have a tail-risk cost line that legacy ROI calculators do not represent. A feature with high mean ROI and high regression risk can be net-negative on a trust-adjusted basis, while a feature with lower mean ROI and tight regression discipline can be net-positive even before the productivity-substitution math kicks in.
Opportunity cost of trust loss
The deepest miss. AI features that ship below threshold or regress noticeably do not just fail to deliver value; they consume future option value. A team that ships one bad agent burns its credibility for the next three agents. A buyer that approves one over-promised AI project loses budget authority for the next six. None of this shows up on the productivity-substitution sheet.
The implication: AI ROI calculators that ignore credibility cost reliably approve the wrong portfolio. Projects with mediocre eval discipline and high productivity-substitution math get green-lit; projects with strong eval discipline and modest first-year productivity look expensive. The first set destroys trust; the second compounds it. Eighteen months in, the org with the second portfolio is still shipping AI; the org with the first is hosting “AI lessons learned” retrospectives.
The 4-component value model
The replacement is a 4-component model, none of which are single numbers and many of which finance teams can score on a defensible scale.
Component 1: Capability earned
What the project lets the team or the product do that was structurally impossible before. Not “do the same thing faster”; that is the productivity-substitution component, which the legacy calculator handles correctly when it applies. Capability earned is the new shape of work.
How to score: name three to five concrete capabilities the project will unlock, with the eval threshold each requires. “We will be able to triage Tier-1 customer issues with a 0.83 accuracy agent that closes 40 percent without human review” is a capability claim. “We will save 100 hours per month on customer support” is a productivity-substitution claim. The first compounds into product strategy; the second is a one-time gain.
Defensible range: each capability claim should map to a roadmap item that depends on the project shipping. If no roadmap items depend on it, the capability claim is decorative.
Component 2: Time-to-value
How long until the buyer is using the system at the eval threshold against the production workload. Not “time to demo.” Not “time to first deployment.” Time to -working, against -real traffic.
How to score: count weeks from kickoff to the first 30-day window in which the system holds at or above its locked eval threshold against production volume. Expected ranges in 2026: 8–14 weeks for narrow agentic workflows on well-scoped data, 16–24 weeks for broader agentic systems, 24–36 weeks for systems requiring substantial new data infrastructure. A project whose time-to-value claim is “month 4” should produce evidence; either an existing eval suite, a similar shipped project at the agency, or a credible plan for getting there.
Defensible range: shorter is not usually better. A 12-week time-to-value on a system that will regress badly at the first model upgrade is worse than a 20-week time-to-value on a system with the eval discipline to absorb upgrades cleanly.
Component 3: Downside-risk-reduced
What does the project prevent? Not just “save hours”; what failure modes does it cap? Customer-visible incidents reduced, compliance exposures eliminated, escalations avoided, knowledge-loss-on-attrition mitigated. These are real value lines the productivity calculator cannot represent because they are subjunctive; they price what would have happened.
How to score: name two to four specific downside risks the project addresses, with rough magnitudes. “An agent that catches Tier-1 escalations 90 percent of the time reduces our average annual escalation cost by an estimated $X.” “Automated PII detection on customer emails reduces our compliance exposure by an estimated $Y.” The numbers are imprecise; the discipline is naming them.
Defensible range: a project with no downside-risk-reduced claim is purely upside, which is rare and worth questioning. A project whose entire value is downside-risk-reduced (e.g., compliance) should be priced as insurance, not as productivity.
Component 4: Optionality created
What else does the project make possible that nobody is committing to today? Optionality is the eval library, the prompt registry, the agent skills, the observability harness; the assets the next AI project will bootstrap from. A project that ships these as byproducts has paid for the next project’s onramp. A project that does not has built a feature with no successor.
How to score: name the persistent assets the project produces beyond the launched feature. “An eval suite of 800 enterprise-domain test cases reusable on the next four projects.” “A prompt registry with version control and A/B harness.” “An observability framework that the next agent ships into in week one.” Each item is an option on a future project, not a guaranteed payoff.
Defensible range: most serious AI project should produce two to four optionality items. Projects that produce zero are pure feature work and will not compound. We make the longer compounding argument in the payback paradox spoke.
How to use the 4-component model in a budget review
Score each component on a 1–5 scale, with brief written justification. Total possible: 20. The numbers do not multiply into a single ROI; they form a profile.
| Component | Score 1 | Score 5 |
|---|---|---|
| Capability earned | One marginal capability | Three to five capabilities, each tied to a roadmap item |
| Time-to-value | 36+ weeks or undefined | 8–14 weeks with credible eval evidence |
| Downside-risk-reduced | Not addressed | Two to four named risks with rough magnitudes |
| Optionality created | Zero persistent assets | Two to four assets reusable on future projects |
A profile of (5, 4, 3, 5) describes a project that earns substantial new capability, ships fast, addresses real downside risk, and produces compounding assets; ship it. A profile of (2, 5, 1, 1) describes a project that ships fast and saves hours but earns no new capability, addresses no downside risk, and produces no persistent assets; defer it; an internal team with Claude Code Max can probably do it next quarter.
The 4-component model does not produce a single comparable number across projects. That is a feature, not a bug. Single comparable numbers across AI projects are how organizations end up funding the wrong portfolio. Profiles force the conversation about what kind of value each project produces, which is the conversation the legacy calculator was designed to avoid.
What to do when the CFO insists on a single number
Some CFOs will insist on a single comparable number per project. The model handles this. Compute a weighted blend with explicit weights:
score = 0.30 × capability + 0.20 × time_to_value + 0.20 × downside_risk + 0.30 × optionality
The weights are project-class-specific and the CFO sets them. A productivity-replacement project (the class the legacy calculator handles correctly) might weight (0.10, 0.30, 0.10, 0.10) and add a productivity term. A platform-building project might weight (0.30, 0.10, 0.20, 0.40). The discipline is that the weights are visible and arguable.
The single-number version is strictly worse than the profile but strictly better than the legacy ROI calculator, because the inputs are AI-shaped rather than productivity-substitution-shaped.
Frequently asked questions
What is the standard AI ROI calculator and why is it wrong?
It computes ROI as (hours saved times salary minus project cost) divided by project cost. A productivity-substitution model; correct for narrow automation projects with a clean human baseline, wrong for everything else. It misses eval threshold effects, model upgrade resets, regression cost, and opportunity cost of trust loss.
What is an eval threshold effect and why does ROI math miss it?
AI features ship in a binary on the eval threshold the buyer locked. A 5 percent gain from 0.76 to 0.81 against a 0.80 threshold might be the entire value of the project; a 5 percent gain from 0.85 to 0.90 might be marginal. Standard ROI treats AI quality as continuous and value as proportional, smoothing over the step function around the threshold.
How often do model upgrades reset AI project ROI?
Three to five times per year. Anthropic, OpenAI, and Google ship non-trivial upgrades roughly quarterly. Each triggers re-evaluation, regression triage, and possible re-architecture. AI project ROI is not a fixed annuity; it is a curve that resets at most model upgrade.
What is the opportunity cost of trust loss?
The credibility cost a team or buyer pays when an AI feature ships below threshold or regresses noticeably. One bad agent burns credibility for the next three. Standard ROI calculators ignore this entirely, which reliably approves projects with mediocre eval discipline and high productivity-substitution math; the projects that destroy trust and prevent the next round of AI investment.
What are the four components of the replacement value model?
Capability earned (what new shape of work the project unlocks), time-to-value (weeks until the system holds at threshold against production traffic), downside-risk-reduced (named failure modes the project caps), and optionality created (persistent assets like eval libraries, prompt registries, and skills that the next AI project bootstraps from). Each component is scored 1 to 5 with written justification, producing a profile rather than a single number.
How do you score capability earned?
Name three to five concrete capabilities the project will unlock, each with the eval threshold it requires and the roadmap item that depends on it shipping. A capability claim mapped to no downstream roadmap item is decorative. “Save 100 hours per month” is a productivity-substitution claim, not a capability claim.
Why is shorter time-to-value not usually better?
A 12-week time-to-value on a system that will regress badly at the first model upgrade is worse than a 20-week time-to-value on a system with the eval discipline to absorb upgrades cleanly. Time-to-value is meaningful only against the locked threshold and against production traffic; fast time-to-demo means nothing.
What is optionality and how does it differ from feature value?
Optionality is the persistent assets a project produces as byproducts: eval library, prompt registry, observability harness. Feature value is consumed when the feature ships; optionality compounds across the next portfolio. A project that produces zero optionality items has built a feature with no successor.
How does this relate to staged-payback ROI?
The 4-component value model is the inputs; staged payback is the gating structure on the outputs. The 90-day, 12-month, and 24-month gates score the profile at each gate, killing or doubling down based on how the four components are tracking.
Key takeaways
- Standard AI ROI calculators are productivity-substitution models. They are correct for narrow automation work and wrong for the AI projects that determine the next eighteen months.
- They miss four AI-specific dynamics: eval threshold effects, model upgrade resets, regression cost, opportunity cost of trust loss.
- The replacement is a 4-component value model; capability earned, time-to-value, downside-risk-reduced, optionality-created; scored as a profile, not a single number.
- Profiles surface what kind of value each project produces. Single-number ROI flattens that conversation, which is how organizations fund the wrong portfolio.
- If a CFO insists on a single number, weight the four components explicitly. The weights are project-class-specific and visible. Worse than the profile, better than the legacy calculator.
- Tie the value model to the staged-payback gates from the manifesto. The four components are the inputs; the gates are the kill criteria.
The math on the legacy ROI calculator is fine. The model is not. Replacing it is the price of admission for any organization that wants its AI portfolio to compound rather than churn.
Arthur Wandzel
