A non-technical founder who cannot speak the basic vocabulary of a 2026 AI build will be talked over in every vendor conversation. Not because vendors are dishonest. The vocabulary is where the power sits. A founder who asks “what is the eval pass rate on your golden dataset?” gets a different proposal than one who asks “how good is the AI?” The first contains numbers and a regression plan. The second contains adjectives. That gap is the gap between a product that ships and a six-figure write-off.
This glossary accompanies the idea validation playbook within the broader idea-to-product manifesto. Twenty-four terms, four groups of six. Each entry: definition, why it matters, the question to ASK in your next vendor call.
Table of Contents
- How to use this glossary
- Group 1: The building blocks
- Group 2: The product surface
- Group 3: The quality contract
- Group 4: The economics
- What to do with this vocabulary
- Frequently Asked Questions
How to use this glossary
Read it end to end — ten minutes. Bookmark each group as a checklist: Group 1 in discovery, Group 2 in the proposal, Group 3 at build kickoff, Group 4 at the unit-economics review (usually too late, which is why you want it first). The frontier (Gemini 2.5 Pro, GPT-5, Claude Opus 4.8, Llama 3.3) shifts every few months — the Artificial Analysis LLM Leaderboard is the scoreboard. The vocabulary is stable.
Group 1: The building blocks
The six terms that show up in the first ten minutes of any AI conversation.
1. LLM (Large Language Model)
A neural network trained on a very large text corpus that takes a text input and produces a text output. Frontier LLMs in 2026 include Claude Opus 4.8, GPT-5, and Gemini 2.5 Pro. Why it matters: the LLM is your product’s engine — choice drives quality, cost, and latency. ASK: which LLM are you proposing as the default, and why over the alternatives?
2. Prompt
The text input you send to an LLM. A prompt can be a single sentence or many pages of instructions, examples, retrieved documents, and user input concatenated. Why it matters: prompts are the load-bearing engineering artifact in 2026 AI products — versioned, tested, reviewed like code. ASK: where are your prompts stored, how are they versioned, and who has commit rights?
3. Context window
The maximum amount of text — measured in tokens — an LLM can process in a single call. 2026 frontier models offer windows from 200K to 2M tokens. Why it matters: the window decides whether a whole document, a long conversation, or many retrieved chunks fit into one call. A hard constraint. ASK: what is the context budget at p95 traffic, and what happens when a request exceeds it?
4. Token
The unit an LLM reads and produces. A token is roughly three quarters of an English word — “tokenization” is two tokens, “the” is one. Models are priced and rate-limited per million tokens, input and output separately. Why it matters: every cost, latency, and capacity number in your build is denominated in tokens. ASK: what is the average input and output token count per request, and how does that translate to cost-per-task?
5. Model
In an AI proposal, “model” means a specific named version — Claude Opus 4.8, GPT-5, Gemini 2.5 Pro, Llama 3.3. The named version determines behavior. Why it matters: the gap between two versions on the same vendor can be a 20–40% quality delta and a 3–10× cost delta. ASK: which exact versions are you using, and what would change if we switched to a cheaper one?
6. System prompt
The instructions a developer pre-pends to every user request, defining the assistant’s role, constraints, tone, and tool permissions. Invisible to the user, central to behavior. Why it matters: your product’s personality, safety posture, and refusal behavior are authored in the system prompt — not the LLM. ASK: can I read your current system prompt, and what is the process for proposing a change?
Group 2: The product surface
The six terms that describe what your product does when a user shows up. Vendors throw them around in proposals; you need to read those proposals critically.
7. RAG (Retrieval-Augmented Generation)
A pattern where the system retrieves relevant documents from a knowledge base, then includes them in the prompt so the LLM can answer using your content — not only its training data. The standard architecture for grounding answers in your data. Why it matters: RAG quality is dominated by retrieval, not the LLM — bad retrieval produces confident wrong answers. ASK: what is the retrieval recall on your evaluation set, and how do you know when retrieval is the bottleneck?
8. Agent
A system where an LLM decides which step to take next, often using tools and loops, rather than following a fixed script. Anthropic’s definition distinguishes workflows (fixed) from agents (dynamic). Agents are more capable and more failure-prone. Why it matters: a vendor proposing “an agent” for a job a workflow could do is selling complexity you will pay to debug. ASK: would a fixed workflow do this, and what specifically requires the agent loop?
9. Tool / function calling
The mechanism by which an LLM calls external functions — search a database, send an email, execute code. The LLM emits a structured request; your application executes it and returns the result. Why it matters: tool calls are where AI products meet the real world — and where most user-visible failures live. ASK: what tools can the system call, who approved each, and what is the failure rate per tool?
10. Multi-turn
A conversation where each user message and assistant response is added to the running context, so the assistant can reference earlier parts. The default mode for chat products. Why it matters: multi-turn introduces state — your test cases need to cover conversation paths, not just single queries. ASK: how do you test multi-turn behavior, and what is the longest conversation the system has been evaluated on?
11. Structured output
When an LLM is constrained to produce output in a specific shape — JSON matching a schema, a typed object — rather than free-form text. The plumbing that lets AI features participate in the rest of your app. Why it matters: structured output makes the LLM behave like a function rather than a free-form writer. ASK: which parts of the response are structured, what schema do they follow, and what happens when the output is invalid?
12. Guardrails
Programmatic checks layered around the LLM to prevent specific bad behaviors — blocking topics, redacting personal information, refusing out-of-scope requests, escalating to a human. Why it matters: guardrails are the last line of defense between your model and a public-relations incident. A vendor with no guardrails plan is selling the model raw. ASK: what guardrails exist, what fires when, and how are new ones added when we find a new failure mode?
Group 3: The quality contract
The group most vendor proposals skip — and the one that decides whether your product ships.
13. Eval
An automated test that grades an AI feature’s output against an expected behavior or rubric, on a fixed input set. Evals are to AI products what unit tests are to SaaS — harder, because output is non-deterministic. Why it matters: without evals, you cannot detect regressions from a model update, prompt change, or upstream shift. ASK: how many evals does the system pass, on what input set, and where are results published?
14. Rubric
The grading criteria an eval uses — typically a 1–5 scale per dimension or a binary pass/fail per requirement. A useful rubric names what good means in concrete, scoreable terms. Why it matters: a vendor who cannot show the rubric is not running real evals. ASK: can I read the rubric, and who wrote it — engineers or domain experts?
15. Hallucination
When an LLM produces a confident, well-formed statement that is factually wrong or unsupported by retrieved context. The signature failure mode of generative AI. Why it matters: hallucinations cannot be eliminated — only detected, escalated, bounded. A vendor who promises zero hallucinations is misinformed or selling. ASK: what is the measured hallucination rate on your eval set, and what is the escalation policy in production?
16. Regression
A previously-passing eval that starts failing after a change — new prompt, new model version, new retrieval index. The most important signal in any AI build. Why it matters: without a regression check on every commit, “small” changes ship as broken features. ASK: do evals run on every prompt change, and what is the policy when a regression appears in CI?
17. Golden dataset
A curated, labeled set of input-output pairs representing the behavior you want. The reference point every eval is graded against. Why it matters: what good means for your users is your knowledge, not the vendor’s. A vendor who writes it alone is writing a test the model can pass — not one that protects users. ASK: who owns it, how big is it, and what is the process for adding cases?
18. Observability
The instrumentation that shows what the AI did, why, and at what cost — every prompt, response, tool call, and token cost traced per request. Why it matters: without observability, debugging a production failure is guessing. ASK: what does the production trace for one user request look like, and how long is trace data retained?
Group 4: The economics
The six terms that decide whether the product pays for itself at scale. Most founders learn them after the inference bill arrives.
19. Inference cost
The variable cost — usually dollars per million tokens — of running an LLM call. Unlike SaaS, where marginal cost is near zero, AI products incur real per-request cost. Why it matters: inference cost decides whether unit economics work at the price your users tolerate. ASK: what is the projected inference cost per user per month at your target usage profile?
20. Latency
The wall-clock time from user request to response. AI features routinely take 1–20 seconds — an order of magnitude slower than SaaS. Why it matters: latency drives perceived quality and retention. A correct answer in 30 seconds is a worse product than a slightly worse one in 3. ASK: what is the median and p95 latency, and what would we change to halve it?
21. p50 / p95
The 50th and 95th percentile of a distribution — typically latency or cost. p50 is the median; p95 is the slow tail. Why it matters: averages lie. p95 latency is where your worst customer experience lives — usually 3–10× worse than p50. ASK: what is the p95 latency under realistic concurrent load, not single-request testing?
22. Fine-tuning vs prompting
Two ways to shape LLM behavior. Prompting changes instructions; fine-tuning changes weights using your training data. Prompting is cheap, fast, reversible. Fine-tuning is slow, expensive, locks you to a model. Why it matters: vendors propose fine-tuning more often than the workload justifies. In most 2026 builds, prompting plus retrieval is enough. ASK: have we exhausted prompting and retrieval, and what specific gap would fine-tuning close?
23. Model alias / version
A pointer like claude-opus-4-8-latest or gpt-5 that resolves to a specific model. Aliases update silently as the vendor ships; pinned versions stay stable. Why it matters: a system pinned to an alias behaves differently in three months with no code change. A pinned version misses improvements until you migrate. ASK: are we pinned to an alias or a version, and what is the migration policy when a new version ships?
24. Cost-per-task
The fully-loaded cost of one unit of user value — one resolved ticket, one drafted email, one analyzed contract. Includes input/output tokens, tool-call costs, retrieval infrastructure, and any human review. Why it matters: cost-per-task is the single number that decides whether the business model works. ASK: what is the projected cost-per-task at launch, at 10× volume, and how does it compare to user willingness to pay?
What to do with this vocabulary
In your next vendor call, deploy three questions drawn from the groups above:
- Product surface: “Is this a fixed workflow or an agent loop, and what specifically requires the loop?” (Term 8.)
- Quality contract: “Who owns the golden dataset, and what is the regression-test policy on every prompt change?” (Terms 17, 16.)
- Economics: “What is the projected cost-per-task at 10× launch volume, and what is the model-version migration policy?” (Terms 24, 23.)
Those three move the conversation from adjectives to numbers. A serious vendor answers with measurements; a non-serious one deflects. Once you can hear the shape of a serious answer, the eval-first build playbook is where that conversation becomes a scope of work an engineer can estimate. If a response sounds promising but vague, read next how to decode “production-ready” in AI agency proposals. Kill criteria and operating cadence sit downstream in the AI feasibility check and the what-is-an-AI-MVP primer.
Frequently Asked Questions
Why exactly twenty-four terms? Four groups of six is the smallest set that covers the four layers of a 2026 AI build — engine, product surface, quality contract, economics — without going unread. Founders who try to learn 80 terms learn none.
Do I need to memorize the definitions? No. Recognize the terms when a vendor uses them, and remember the question to ask. The vocabulary is a comprehension aid.
Which model should I name in my next vendor call? None specifically — let the vendor propose, then ask which exact version and why. As of mid-2026 the frontier sits around Gemini 2.5 Pro, GPT-5, and Claude Opus 4.8 (Artificial Analysis).
What is the difference between an agent and a workflow? A workflow runs a fixed sequence of steps. An agent uses an LLM to decide each step. Agents are more capable, more expensive, harder to debug. Most 2026 builds need workflows.
Is RAG always required? No. RAG is required when the system must ground answers in your specific data — internal documents, customer records, knowledge bases. If the product only needs general reasoning, you may not need retrieval.
How big should my golden dataset be? Fifty to 150 cases at validation, growing with the product. Below 50 the suite cannot distinguish a regression from a flake; above 150 you are over-investing.
Should I pay for fine-tuning? Usually not. Prompting plus retrieval covers most 2026 use cases. Fine-tuning is appropriate when both are exhausted and a specific quality gap remains.
What if the vendor cannot answer the cost-per-task question? Treat it as a signal. A serious 2026 vendor models cost-per-task before quoting fixed-price scope. A vendor who cannot answer is asking you to underwrite their unit-economics risk.
Arthur Wandzel
