Not Every AI Needs to Talk: Meet the Decision API

Image placeholder

Editorial, product-forward hero illustration — a wide horizontal composition on an off-white background. On the left, many pale-gray horizontal lines of varying lengths, like abstract paragraphs of text, fading toward the middle. On the right, a single bold emerald-green pill-shaped capsule floating alone. A thin dark-navy arrow bridges the two halves. Palette strictly off-white, pale slate grays, one emerald green accent, one navy ink. Clean editorial feel, Stripe or Linear style. No faces, no robots, no neural networks.

• Aspect ratio: 16:9
• Source size: 1600 × 900 px (sharp on retina at 760 px display width)
• Format: WEBP (preferred), PNG fallback
• Target file size: under 300 KB

Save to /public/images/blog/<slug>/, then set src and alt on this <MediaFigure /> to swap it in. Keep the ratio prop so the browser reserves the right space.

By Dr. Sanju Abraham, Fractional CTO — 180+ projects shipped, 34 funded companies. Published September 27, 2026.

Key takeaways

• A Decision API is an AI model that returns a typed, structured value — a label, a score, a routing choice — instead of a paragraph of prose.
• It’s the same underlying LLM. What changes is the output contract, enforced via JSON Schema, enums, or constrained decoding.
• Reach for a Decision API whenever the next consumer of the response is code, not a human.
• The payoff: shorter code, lower latency, measurable confidence, and real unit testability.
• Four patterns cover most production use cases: classify, extract, score, route.

A developer I spoke to last month was building something simple: a function that reads an incoming support email and routes it to the right queue. Billing goes to one place. Outages go to another. Everything else goes to a human.

She reached for the obvious tool. She sent the email to an LLM with the prompt “Which queue should this go to?” and the model dutifully returned:

“Based on the content of this email, which mentions a declined payment and a request for a refund, I would recommend routing this to the Billing queue. It appears to be a standard billing inquiry and does not indicate a system outage.”

Lovely. Thoughtful. Completely wrong tool for the job.

She then wrote twenty lines of code to parse that paragraph for the words “Billing,” “Outage,” or “Human,” handle the cases where the model editorialized (“I think it’s probably…”), handle the cases where it refused to decide, and handle the cases where it decided but spelled “biling” wrong. Then she wrote a fallback. Then she wrote a retry. Then she went to lunch.

She did not need the model to talk about the decision. She needed it to make the decision. Those are two different products.

The chatbot shadow over AI

Most people’s working mental model of AI is ChatGPT. That makes sense — ChatGPT was, for most of the world, the moment AI showed up at the door. And the interface it showed up in was conversation. You type. It talks back.

That interface was a brilliant piece of product design. It was also an accident of history. The underlying language models — the actual math under the hood — don’t care about conversation. They predict probable continuations of text. They do that whether you want prose, whether you want JSON, whether you want a single yes-or-no token, or whether you want nothing at all beyond a routing decision.

The “chat” is a wrapper. There are other wrappers. We just don’t talk about them because they don’t make for a demo video.

Builders who are only exposed to the chat wrapper tend to use the chat wrapper for everything, the way someone who only owns a hammer tends to find a lot of nails. The result is a quietly expensive mistake: production systems that ask a model to generate 200 tokens of English to communicate a single bit of information.

What is a Decision API?

A Decision API is an AI model whose contract is to return a structured, typed value from a bounded set of outcomes — a label, a score, a routing choice — not a prose response. The input is still natural language (or an image, or whatever the model consumes). But the output is a schema. It’s { "route": "billing" }. It’s "urgent" from an enum. It’s a score between 0 and 1. It’s the single boolean your business logic actually needs.

Under the hood, the mechanisms that make this reliable are things like constrained decoding, JSON-schema-conditioned sampling, enum restriction, and logit biasing — all of which force the model to emit outputs that conform to a shape you declared in advance. The model is still “the same” general-purpose model. You’ve just given it a tighter contract to operate under.

All current frontier LLMs — OpenAI’s GPT family, Anthropic’s Claude family, Google’s Gemini, and Meta’s Llama — support this mode under names like “structured output,” “JSON mode,” or “tool/function calling.” You do not need a bespoke classifier trained on your data. For most workflows, pointing a general-purpose model at a schema is enough.

Where are Decision APIs already running in production?

The pattern of “AI as decision engine” isn’t new. It’s been running production systems for a decade. You’ve just been using it without labeling it that way.

Stripe Radar scores every card transaction and returns a risk number. Gmail decides whether an email is spam, promotional, or a receipt, and returns a label. Zendesk suggests a priority. YouTube recommends what to watch next. OpenAI’s moderation endpoint — notably, a different endpoint from its chat endpoint — returns a vector of category scores and nothing else. None of these systems write you a paragraph about their reasoning in production. They decide.

What’s new is that the general-purpose LLMs that captured the public imagination can now behave this way too, with a one-line change to how you call them. The decision model used to be a bespoke classifier trained on your data. Now it can be a frontier LLM that you point at your enum and your schema, and it just works for most workflows that would have cost a quarter of engineering time to build the custom way.

The implication for builders: every “I need to decide X” problem in your backlog is now a one-day problem, not a one-quarter problem. If you’re still estimating these the old way, your roadmap is wrong.

Image placeholder

Infographic: a tidy visual map of 'invisible AI' decision models already running in common tools. Suggested layout — a 2x3 or 3x3 grid of recognizable product tiles (Stripe Radar, Gmail, Zendesk, YouTube, OpenAI Moderation, Netflix recommendations), each with a one-line label describing the decision it makes (e.g. 'risk score 0–100', 'spam / promo / receipt', 'next video'). Palette matches the article: off-white background, pale slate borders, emerald accents.

• Aspect ratio: 16:9
• Source size: 1280 × 720 px (sharp on retina at 760 px display width)
• Format: PNG
• Target file size: under 500 KB

Save to /public/images/blog/<slug>/, then set src and alt on this <MediaFigure /> to swap it in. Keep the ratio prop so the browser reserves the right space.

What changes when the AI just decides?

When the output is a decision instead of a paragraph, the code around it stops being a mess of string-handling and starts looking like ordinary software. Four specific things change for the better.

Parsing goes away. You don’t post-process natural language. You read a field off a typed object. The code becomes about ten lines shorter and five times more legible.

Uncertainty becomes a number. A chat response is “confident-sounding prose” whether the model is sure or not. A decision response carries a confidence score, token-level log-probabilities, or an explicit "unsure" value in the enum. Your downstream logic can route low-confidence cases to a human the same way it routes high-value transactions to a review queue.

Latency drops. 200 tokens of prose costs 200 tokens of generation. Three JSON fields cost three JSON fields of generation. On high-volume paths, that difference compounds into real money and real user-perceived speed.

Testing becomes possible. You can write unit tests for a function that returns { "route": "billing", "confidence": 0.9 }. You cannot meaningfully unit-test a function that returns a paragraph of varied English.

The four Decision API patterns, with examples

Four shapes cover the vast majority of practical Decision API use cases in business software: classification, extraction, scoring, and routing.

At a glance:

• Classification — one of N buckets (route tickets, tag leads, label expenses).
• Extraction — fill in a form (resume → fields, invoice → fields).
• Scoring — a number on a scale (lead score 0–100, risk 0–1).
• Routing — pick the next step (which specialist, which downstream model, which template).

Classification. “This belongs in one of N buckets.” Route tickets to queues. Tag incoming leads by segment. Mark expenses as reimbursable or not. Decide whether a user message is a complaint, a feature request, or a question. The output is one of a bounded list.

Extraction. “Pull these specific fields out of unstructured text.” Pull the vendor name, invoice total, and due date from an attached PDF. Pull the candidate’s current title, years of experience, and remote preference from a resume. The output is a filled-in form.

Scoring. “Rank or rate this item on a continuous or discrete scale.” Score a sales lead 0–100. Rate a candidate against a job description 1–5. Assess the risk of a transaction. The output is a number plus (if you ask) a one-line justification.

Routing. “Pick the next step for this item.” Choose which specialist should see this patient. Decide which of three downstream models to call based on the kind of input. Pick which template to use for a response. The output is one of a bounded set of actions, often with a confidence.

If you’ve been building AI features for a while, you’ll notice that chat completions can technically do all four — if you post-process. The difference is that a decision call does them correctly, every time, by construction. That reliability is the entire ballgame when you’re putting AI on a hot production path.

What this does to your architecture

The architectural effect of this shift is bigger than it looks. When your AI layer returns typed, scored outputs, the code around it stops being a mess of string-handling and starts looking like ordinary software.

You can type the output. TypeScript, Pydantic, Zod — your usual tools work again, because the AI is now returning a value that fits a schema. If the model deviates, you catch it the same way you catch any bad data: at the boundary, with a validation error.

You can set thresholds. “Act automatically above 0.9 confidence. Queue for human review between 0.7 and 0.9. Escalate below 0.7.” That rule is one line of code. With a prose model, the equivalent rule is unwritable.

You can test it. You can build a golden set of 200 input-output pairs and run them as a regression suite every time you change the prompt or the model. Nothing about this is novel engineering practice — it’s the same way you’d test a classifier you trained yourself. The AI just happens to be behind the function.

You can monitor it. You can log the output distribution. You can see when the model starts classifying everything as “other.” You can alert on drift. You can roll back. Chat prose offers you none of this.

Taken together, these changes mean AI stops being an opaque feature in your app and starts being an ordinary subsystem — the kind of thing you can own, operate, and improve. That is a quietly enormous shift for anyone whose job is to make software work reliably.

The one trap worth naming: teams get the structured output working, then forget to ask the model for a short reason field alongside the decision. Three months later, when the classifier is misrouting 4% of tickets and someone asks why, there’s no trail to follow — just an enum value and a confidence score. The fix is cheap: include a one-sentence reason in every schema, log it, and sample it weekly. It’s not what your production code reads, but it’s what you’ll want when the model’s behavior drifts and nobody can remember exactly which prompt version is live.

When should you use a Decision API instead of a chat completion?

Use a Decision API whenever the next consumer of the response is code, not a human. Both tools have their place, but the test is who or what reads the output.

If a human is going to read the output, you probably want a chat completion. Humans are the ones who benefit from prose that explains itself, hedges where appropriate, and sounds coherent.

If a downstream function is going to read the output, you almost certainly want a decision call. Code benefits from values, not sentences.

Two sharper heuristics that fall out of the main rule:

If the next thing that happens with the response is a string comparison, you wanted a decision call. (You’re parsing prose back into a value. Skip the step.)

If the next thing that happens is a render into a UI, you probably wanted a chat call. (A human is about to read it.)

Most AI features in production are closer to the first case than teams realize. The chat interface is familiar, so builders reach for it. But the code around it is often doing a tortured impression of a decision call that should have just been a decision call to begin with.

Image placeholder

Infographic: a timeline or S-curve showing the trajectory of 'invisible AI' adoption. Suggested treatment — a horizontal timeline from 2015 to 2030 with milestones (bespoke classifiers → LLM chat → LLM structured output → decision APIs as default infrastructure), with a secondary curve showing the share of AI calls that are structured/decision calls rising from ~10% today toward ~70% in the next five years. Palette matches the article.

• Aspect ratio: 16:9
• Source size: 1280 × 720 px (sharp on retina at 760 px display width)
• Format: PNG
• Target file size: under 500 KB

Save to /public/images/blog/<slug>/, then set src and alt on this <MediaFigure /> to swap it in. Keep the ratio prop so the browser reserves the right space.

Where this is heading

The next five years of applied AI will not be dominated by chatbots. It will be dominated by the slow, quiet absorption of decision-model APIs into ordinary software — the way SQL became invisible, the way payment APIs became invisible, the way image-resizing became invisible. Models that decide will sit behind every form, every router, every triage queue, every recommendation. Most of them will never say a word.

That’s a good thing. The best infrastructure is the kind you stop noticing. The worst outcome for AI as a technology would be if we trapped it inside the chat window forever — because then every intelligent decision in a system has to be mediated by a conversation. Decisions don’t want conversations. They want to be decisions.

If you’re building something right now that has an AI feature in it, the useful exercise this week is to walk through every call site and ask one question: does the model need to talk, or does it need to decide? Where the honest answer is “decide,” rewrite the call. Your code will get shorter, your latency will drop, your tests will start passing, and the model will start behaving like the engineering primitive it actually is.

The chatbot is not AI. The chatbot is a product. The models underneath can also be a database row, a function return value, a route, or a yes. The builders who notice this early will ship better software than the ones who don’t.

Frequently asked questions

What is a Decision API?

A Decision API is an AI model that returns a structured, typed value from a bounded set of outcomes — a label, a score, a routing choice — instead of writing a prose response. The underlying model is the same general-purpose LLM; what changes is the output contract, enforced through tools like constrained decoding, JSON Schema, or enum restriction.

How is a Decision API different from a chat completion?

A chat completion returns natural-language prose that a human reads. A decision call returns a typed value that your code reads. Both can be served by the same underlying model — GPT, Claude, Gemini, or Llama — so the difference lives in the output contract and the downstream consumer, not in the model itself.

When should I use a Decision API instead of ChatGPT?

Use a Decision API whenever the next thing to happen with the response is a function call, a database write, or a routing decision — not a human reading prose. A fast test: if the first thing your code does after receiving the response is a string comparison, you wanted a Decision API from the start.

Do GPT-5, Claude, and Gemini support Decision API-style calls?

Yes. All current frontier LLMs — OpenAI’s GPT family, Anthropic’s Claude family, Google’s Gemini, and Meta’s Llama — support structured output, JSON Schema conformance, or function and tool calling. These are the mechanisms that turn a chat model into a decision model. You rarely need a bespoke classifier; one API call with a schema is usually enough.

What are examples of Decision APIs already running in production?

Stripe Radar scores card transactions and returns a risk value. Gmail labels incoming mail as spam, promotional, or receipt. OpenAI’s Moderation endpoint — notably separate from its Chat Completions endpoint — returns category scores and nothing else. These are decision models by any other name; what LLMs did was make the same pattern available to any team that can write a schema.

Can I unit-test a Decision API call?

Yes, straightforwardly — because the output is typed and comparable with normal equality assertions. Build a golden set of 50–200 input/output pairs and run them as a regression suite on every prompt or model change. That is the kind of testability prose outputs do not permit.

Sources

Every claim about a specific product or standard in this article is linked inline to its primary source. The full list, in order of first appearance:

1. JSON Schema — the open specification for declaring and validating JSON structure, and the lingua franca for structured-output contracts across LLM providers.
2. OpenAI — Structured Outputs — the GPT family’s schema-conformant output guide, including JSON mode and response_format with a JSON Schema.
3. Anthropic — Tool Use — how Claude emits typed values via tool/function calling against a declared input schema.
4. Google Gemini — Structured Output — Gemini’s schema-constrained response configuration.
5. Meta — Llama — Meta’s open-weight model family, which supports structured output via tool/function calling in its chat template.
6. Stripe Radar — Stripe’s ML-based fraud detection product that returns a numeric risk score per transaction, cited here as a canonical production Decision API.
7. Gmail — Inbox categories — Google’s documentation on how incoming mail is classified into Primary, Social, Promotions, Updates, and Forums.
8. OpenAI — Moderation — OpenAI’s content-classification endpoint, intentionally separate from its chat endpoint, returning category scores and nothing else.
9. TypeScript — static type system used to validate structured LLM outputs at the application boundary.
10. Pydantic — Python data-validation library frequently paired with LLM structured outputs for schema-first validation.
11. Zod — TypeScript-first schema declaration and validation library used for the same purpose in JS/TS codebases.

---

If you’re designing an AI-shaped feature right now and the architecture decisions are starting to feel bigger than a single afternoon, that’s the exact gap a Fractional CTO is meant to fill — see our Fractional CTO services for how that engagement usually looks. One good conversation about whether the model should talk or decide can save a quarter of engineering time. For a broader take on how to pick AI investments that actually land, see our companion piece on an AI strategy for SMB owners that survives contact with reality.