AI projects accrue a distinct flavor of technical debt that compounds faster, hides better, and costs more than the technical debt of legacy software systems. Prompt-registry drift, eval-test-set staleness, observability lag, and model-version sprawl are the four named categories. Each accrues silently between releases, each taxes engineering velocity at a measurable rate, and each compounds against the next model upgrade rather than waiting for a developer to notice it. A project carrying full debt across many four categories is paying a velocity tax in the 25 to 40 percent range; meaning a quarter to two-fifths of nominal engineering capacity is consumed by reverse-engineering, triage, and forced re-evaluation rather than capability work.
This is a spoke under the AI project economics manifesto. The manifesto argues evaluation is the unit of account; technical debt is what happens to that unit when the engineering discipline that maintains it is allowed to lapse between releases. Unlike legacy tech debt, this debt is paid on a schedule the model vendor sets, not the team chooses.
Why AI tech debt is structurally different
Three structural differences set AI tech debt apart from legacy tech debt.
First, AI debt accrues against a moving substrate. Legacy software runs on a stack the team controls. The language version, the framework version, the database; many change on a schedule the team chooses. AI software runs on models that change on the vendor’s schedule. Debt that is dormant against today’s model becomes active against next quarter’s. A prompt that handles edge cases through a particular reasoning quirk on the current flagship may produce different behavior on the next flagship in three months. The team did not choose the substrate change; the team must respond to it.
Second, AI debt is often invisible until eval rerun. A legacy system with debt produces visible signs: slower CI, longer build times, harder PR reviews, increasing bug count. An AI system with debt may produce no visible signs at many between eval runs. Prompt drift produces no error message. Eval staleness produces no failed test. Observability lag produces no log line saying “this regression went undetected.” The first signal is often a customer report or an eval rerun that finally exposes the accumulated drift.
Third, AI debt compounds faster. Each model upgrade re-exposes prior debt to a different failure surface. Legacy tech debt sits where it was deposited; AI tech debt is rewalked across new terrain most quarter. A prompt accumulating drift through ten months of edits faces a forced revalidation against the next model upgrade. An eval set that masked a subpopulation regression on the current model may surface a different one on the next.
These three together convert AI tech debt from a discretionary cost the team manages into a periodic forced expense the team must service. The economics consequence is that the debt cannot be deferred indefinitely; it gets paid down on the upgrade schedule whether or not the team budgeted for it.
The four named categories
Four categories of AI tech debt account for the bulk of the velocity tax. Each has a distinct accrual mechanism and a distinct repair profile.
| Category | Velocity tax | Compounding mechanism |
|---|---|---|
| Prompt-registry drift | 8 to 14 percent | Edits without versioning accumulate against test surface |
| Eval-test-set staleness | 6 to 10 percent | Workload distribution shifts faster than eval set updates |
| Observability lag | 5 to 9 percent | Detection delay hides other debts and inflates repair cycles |
| Model-version sprawl | 4 to 8 percent | Multiple model versions each carry maintenance overhead |
Total: 25 to 40 percent of engineering capacity at steady state on a debt-loaded project. The lower end is a project with active debt management; the upper end is a project where debt has been deferred for a year or more.
Category 1: prompt-registry drift
Prompt-registry drift is the accumulation of prompt edits without versioning, attribution, or test coverage. Production prompts diverge from documented prompts. Engineers cannot answer the question “what changed in this prompt last quarter and why” without spelunking through git history that does not capture the rationale.
How it accrues:
- Quick prompt edits made under deadline pressure without updating the registry.
- Multiple engineers editing the same prompt for different goals without coordination.
- Few-shot examples added speculatively and rarely removed when their value lapsed.
- System-prompt clauses appended to handle specific bugs without removing the source bug.
Velocity tax: 8 to 14 percent of engineering capacity at steady state. Time spent reverse-engineering the prompt rather than improving the system. Severe cases; registries that are six to twelve months drifted; consume 20 percent of capacity and stall capability work entirely.
Repair pattern: a one-time prompt audit against the current eval set. For each clause, ablate and rerun. Clauses that do not earn their tokens come out. Document each remaining clause with a rationale and a test reference. The audit is roughly two engineering weeks per major prompt; once done, the registry stays clean if the team adopts versioning discipline going forward.
Category 2: eval-test-set staleness
Eval-test-set staleness is the divergence between the eval set the team is testing against and the workload distribution the system serves. A team designs an eval set in month one based on the workload as understood at kickoff. The workload shifts over months; new customer segments, new query patterns, new edge cases reported by support. The eval set does not move with the workload, and most passing eval becomes false comfort.
How it accrues:
- Workload distribution shift that nobody samples against the eval set.
- New failure modes reported by support that rarely become eval cases.
- Customer segments added through sales without eval coverage of their query patterns.
- Eval-set authors leaving the team and the institutional context with them.
Velocity tax: 6 to 10 percent of capacity. Plus a regression-cost tail when the staleness gets discovered after a production incident; the team finds itself debugging against a stale baseline and the repair cycle extends. The hidden cost dimensions of evals across the project are detailed in the hidden cost of AI evals.
Repair pattern: quarterly eval-set refresh against rotated traffic samples. Each quarter, sample 200 to 400 cases from production traffic, stratified across customer segments and query types, and add them to the eval set. The refresh is roughly three engineering days per quarter; the cost of skipping it is invisible until it isn’t.
Category 3: observability lag
Observability lag is the time between a regression occurring in production and the team detecting it. Lag of hours is normal during early operation; lag of days indicates structural debt in the observability stack. Lag of weeks indicates the observability stack is not functioning as a detection tool; it is functioning only as a confirmation tool after a customer report.
How it accrues:
- Alert thresholds set during early operation rarely tightened as traffic stabilizes.
- Dashboards built for the demo rarely refactored for operational use.
- Sampling rates that worked at low volume that no longer surface low-frequency regressions at high volume.
- Trace storage and retrieval costs not budgeted, leading to truncated retention.
Velocity tax: 5 to 9 percent direct. The compounding effect is larger: observability lag hides most other debt category. Prompt drift, eval staleness, and model-version sprawl many become invisible to the team until customers report failures, and customer-reported failures cost two to four times more to repair than observability-detected failures.
Repair pattern: monthly synthetic regression injection. The team injects known-faulty prompts and known-stale eval cases into shadow traffic, measures mean time to detection, and tunes the observability stack against that metric. The injection harness is roughly one engineering week to build; ongoing operation is one to two engineer-days per month.
Category 4: model-version sprawl
Model-version sprawl is the accumulation of code paths calling different model versions, often for historical reasons rather than current ones. A production system with five model versions across its surface area is paying maintenance tax on five separate eval profiles, five separate prompt formats, five separate cost trajectories.
How it accrues:
- Pinning a code path to a specific model version “for now” and rarely returning.
- A/B tests that ship the experimental arm without retiring the control.
- Customer-specific overrides that pin individual accounts to particular models.
- Vendor SDK upgrades that introduce a new SKU without retiring the old one.
Velocity tax: 4 to 8 percent direct. The larger cost arrives at deprecation: when a vendor deprecates a model, most code path calling it needs separate re-evaluation, and the deprecation cost is multiplied by the sprawl factor. A clean system with one production model pays one re-eval cycle on deprecation; a five-model-sprawl system pays five. Detail in why your AI project budget should have a model deprecation reserve.
Repair pattern: a semiannual model-consolidation review. List most model version in production, the code paths calling it, and the rationale for the version pin. Versions without active rationale get migrated to the current default. Versions with active rationale get documented so the next deprecation event finds the team prepared rather than surprised.
The compounding mechanism
The four categories compound against each other and against the model-upgrade cycle.
Each model upgrade triggers a forced re-evaluation. Prompt drift surfaces because the prompt behaves differently on the new model. Eval staleness surfaces because the new model exposes regressions on workload distributions the eval set rarely covered. Observability lag delays detection of the upgrade-induced regressions, lengthening the repair cycle. Model-version sprawl multiplies the work because each pinned version needs its own re-eval.
The arithmetic is unforgiving. A project with 10 percent debt in each of the four categories does not face a 40 percent tax; it faces a higher tax because the debts amplify each other. Prompt drift discovered late through customer report, in a sprawled codebase, with stale evals, costs more than 4x the same drift discovered early through observability in a clean codebase.
Model-vendor cadence makes the compounding mechanism unavoidable. Three to five non-trivial model upgrades per year across the major vendors means three to five forced revalidations. A project that does not pay the debt down between upgrades pays it many at once during each upgrade; and pays it under deadline pressure, which is the most expensive way to pay any debt.
Payoff schedule and budget reserve
The payoff schedule for AI tech debt across mature 2026 engineering shops:
- Dedicated debt-paydown sprints recover 60 to 80 percent of velocity loss within four to six weeks of focused engineering investment.
- The remaining 20 to 40 percent is structural; full eval-set rebuild, observability stack refactor; and recovers over two to three quarters of disciplined operation.
- Without explicit reserve, debt accumulates at a rate that consumes the next quarter’s capability budget.
Project budgets should include a debt-paydown reserve sized at 8 to 12 percent of total engineering capacity, paid quarterly rather than at the decline of the engagement. The reserve covers the prompt audits, eval-set refreshes, observability tuning, and model-consolidation reviews described in the category sections above. Spending the reserve quarterly is materially cheaper than spending it annually, because the cheaper repair cycles run before the debts have compounded against each other.
Three structural moves prevent the debt from accumulating beyond the reserve:
Move one: prompt registry as a versioned artifact. Not a free-form text store. Most change has attribution, a rationale, a test reference, and a date. Tooling exists in 2026; Promptfoo, LangSmith, custom registries; but the discipline is what makes the tooling pay off.
Move two: quarterly eval-set refresh. Against rotated traffic samples, stratified across segments. Calendar-driven, not opportunistically.
Move three: observability calibration as a tracked metric. Mean time to detection on synthetic regressions, measured monthly, reviewed quarterly. The metric is the lever; without it the observability stack ages quietly.
A project that runs many three structural moves keeps debt inside the 8 to 12 percent reserve. A project that does not pays the 25 to 40 percent velocity tax described above and the additional cost of crisis repair during model upgrades.
Frequently asked questions
Does this debt model apply to internal AI teams as well as agency engagements? Yes, and the model is harder to govern internally because the velocity tax is paid in headcount opportunity cost rather than billable hours, which makes it less visible to finance.
How does this interact with the move toward agent frameworks and tool-use? Agent systems accumulate a fifth debt category; tool-call drift; where tool definitions, error handling, and retry logic accumulate undocumented edits. The velocity tax for agent-heavy systems is typically 3 to 6 points above the chat-shaped baseline.
Should the debt-paydown reserve be a contracted line item or an internal budget? Both work. Contracted is structurally cleaner because it survives engagement transitions and budget pressure cycles. Internal works when the engineering org has clear governance authority over its own capacity allocation.
What is the worst single category to leave unaddressed? Observability lag, because it hides most other category. A team that fixes prompt drift while observability remains lagging will rediscover the drift through customer reports rather than internal detection. The compounding effect makes observability the highest-leverage debt to address first.
How often should the prompt audit run? Once per major model upgrade, plus once per quarter for high-traffic prompts. Twice-yearly minimum on low-traffic prompts.
Does the debt model apply to fine-tuned smaller models? Yes. The model-version sprawl category becomes “fine-tune-version sprawl” with the same dynamics. Eval staleness and observability lag apply identically. Prompt drift is somewhat reduced because fine-tuned models often have more constrained prompt surfaces.
How does the debt model show up in due diligence on AI-native acquisitions? A buyer should run a debt audit across the four categories before close. A target with high model-version sprawl and stale evals carries integration cost that does not appear on the income statement. Adjusting purchase price by the present value of debt-paydown is a defensible move.
What signals indicate debt is approaching crisis levels? Three signals: capability sprints consistently under-deliver against estimate; engineers describe specific prompts or eval cases as “haunted”; on-call rotations cite the same root cause repeatedly without a structural fix. Any one of these signals indicates debt is past the manageable reserve.
Is there an industry benchmark for AI tech debt? Not a published one as of 2026. The 25 to 40 percent velocity-tax range is empirical from mature shops. Lower benchmarks indicate either an exceptionally young codebase or hidden debt that has not yet surfaced.
Key takeaways
- AI tech debt accrues against a moving substrate (model versions), is invisible until eval rerun, and compounds faster than legacy tech debt.
- Four named categories: prompt-registry drift (8 to 14 percent velocity tax), eval-test-set staleness (6 to 10 percent), observability lag (5 to 9 percent), model-version sprawl (4 to 8 percent).
- Total velocity tax on a debt-loaded project: 25 to 40 percent of engineering capacity.
- The categories compound against each other and against the model-upgrade cycle, which forces revalidation three to five times per year.
- Payoff: dedicated paydown sprints recover 60 to 80 percent within four to six weeks; the rest is structural and recovers across two to three quarters.
- Budget reserve: 8 to 12 percent of total engineering capacity, spent quarterly rather than annually.
- Governance: versioned prompt registry, quarterly eval-set refresh, observability calibration as a tracked metric.
- Observability lag is the highest-leverage debt to address first because it hides the other three.
Arthur Wandzel
