How I Use a Decision Model in My Own Product

Image placeholder

Editorial, product-forward hero illustration — a stylized cross-section of a book-platform pipeline on an off-white background. Left side: an open book spread with a highlighted sentence drawn as a thin emerald rule. A thin pipeline runs rightward from the highlighted sentence into three small labelled boxes stacked vertically — top box 'VERIFY: quote accurate?' with a green dot; middle box 'RATE: section quality 0–100' with a blue slider mark; bottom box 'CHOOSE: collection' with one of three radio labels filled. Each box emits a tiny typed value (0.93, 78, 'Craft'). Thin pale slate separators between zones. Palette: off-white background, pale slate grays, emerald green accent (#10b981), navy ink (#0f172a), one blue accent for the slider (#2563eb). Clean, flat, geometric. Linear or Stripe editorial feel. No AI robots, no neural networks, no faces, no brains.

• Aspect ratio: 16:9
• Source size: 1600 × 900 px (sharp on retina at 760 px display width)
• Format: WEBP (preferred), PNG fallback
• Target file size: under 300 KB

Save to /public/images/blog/<slug>/, then set src and alt on this <MediaFigure /> to swap it in. Keep the ratio prop so the browser reserves the right space.

By Dr. Sanju Abraham, Fractional CTO — 180+ projects shipped, 34 funded companies. Published October 3, 2026.

Key takeaways

• I run three decision-model calls inside my book platform: a Verify call to check whether a pulled quote is accurate, a Rate call to grade a section’s craft, and a Choose call to sort a finished book into a collection.
• Each call maps cleanly to one of the three shapes — Choose, Rate, Verify — described in the three questions a decision model can answer.
• The biggest win was not speed; it was the code I got to delete. Prompt scaffolding, JSON parsers, retry logic, “please only answer yes or no I beg you” rituals — all gone. I started calling this payoff the deletion dividend.
• Tests became boring. A typed response means I can assert on values the way I assert on database rows. No more fuzzy-matching a paragraph.
• I did not rip out my chat model. Draft generation, author voice, and reader-facing explanations still run on a frontier LLM. The decision model lives silently underneath.

The short answer: I run a small book platform. Three of its AI touch points used to be chat-model calls that returned paragraphs; today they are decision-model calls that return typed values. The platform is faster, cheaper, and — the part I did not expect — much easier to test. The win I kept returning to is what I now call the deletion dividend: the lines of parsing code, prompt scaffolding, and defensive retries that quietly disappeared the moment the AI stopped writing explanations and just gave me answers.

I keep getting asked what it is actually like to run a product on top of a decision model. Not the pitch, the day-to-day. So this piece is a field report from one of my own: a book-authoring and reader-facing platform I build and maintain, which now routes three of its everyday judgments through TypeSafe AI’s Jev. I use it to decide if a quote I surfaced really appears in the source book, to rate the craft of a section I am about to recommend, and to sort finished books into reader-facing collections. Three calls. Three shapes. One pattern. For the primitives behind each, see what Jev is in plain English.

What am I actually using a decision model for?

I am using it for the small judgments my code was already making — just badly. Before, each of those judgments was either a brittle keyword rule, a chat completion I had to parse, or a human doing something tedious in a dashboard. After, each is one typed call.

The platform does three things with books end to end: it helps authors draft and revise, it packages finished books for readers, and it surfaces quotes and recommendations. Inside those three products, dozens of tiny decisions happen every minute. The ones I eventually moved to Jev all shared the same shape: a question with a bounded answer, running at scale, where the answer feeds another piece of code — not a human reading prose.

If you want the frame I used to choose which calls to move, it is the one from five everyday business tasks a decision API can do today and the mindset shift behind the move is covered in why I stopped asking AI to write and started asking it to decide. This article is the view from inside the pipe once I did.

Example one: is this quote actually in the book?

My platform surfaces quotes from a book to promote it on social, in previews, and inside the reader app. For months I had a stubborn, unfunny problem: a chat model would occasionally invent a quote, confidently cite a page number, and the quote would be nowhere in the source. The failure mode was always the same, and it was always embarrassing.

The fix was a Verify call. The input is the pulled quote plus the book’s full text; the question is a single Noul: “Does this quote appear verbatim in the book?” Jev returns a probability between 0 and 1. Above 0.9, the quote is cleared for publishing. Between 0.6 and 0.9, it goes to a human reviewer in the authoring dashboard with the suspected source paragraph highlighted. Below 0.6, the system drops the quote and asks the pipeline to pick another candidate. That three-way routing is the pattern I wrote about in why “70% sure” is more useful than “yes”.

The thing I noticed first was not the quality improvement, which was real. It was that the quote-check module stopped being a special case. Before, it had its own prompt file, its own retry logic for malformed JSON, and a parser that handled the four or five ways the chat model liked to wrap its answer in markdown. After, it is one function with one return value. One of my teammates reviewed the diff and asked where the rest of the code went. That is the deletion dividend showing up in a pull request.

Example two: how good is this section, really?

My platform recommends sections across books for readers who want a sampler. The question I needed answered was uncomfortably subjective: on a scale, how well-crafted is this particular section? Not the whole book — the section.

Before Jev, I was asking a chat model to “please rate this section from 1 to 100 and explain your reasoning in a short paragraph.” The paragraph was fine. The number was buried in the paragraph. My parser was a disaster. The scale drifted every time I touched the prompt. Half the time the model would refuse the number and give me a vibe-check instead.

I replaced that with a Score call on a 0–100 rubric with anchors — a short definition of what 20, 50, and 80 mean on this scale — baked into the prompt. The model returns a single integer and a confidence number. Sections scoring above a cutoff flow into the sampler. Sections below the cutoff get a lightweight explanation request (a separate chat-model call) only when a human is going to look at them. The Rate call runs on every section; the explanation chat call runs on maybe three a day.

The pleasant surprise was testability. I now have a small golden set of sections with known-good ratings that run every time I change the prompt or move to a new model version. If my 0–100 average drifts by more than a few points on the golden set, I know. Testing a chat model’s prose version of the same thing is a theological question, not an engineering one.

Example three: which collection does this book belong in?

At publish time, a finished book has to land in exactly one reader-facing collection. I use four at the moment — Craft, Story, Field, and Memoir — and I kept them small on purpose because every reader-facing surface in the product depends on them being mutually exclusive and legible at a glance.

The old version of this was two humans in a Google Doc arguing about taxonomy. It worked, slowly. The new version is a Choice call. The input is the book’s metadata (title, logline, chapter headings, a short synopsis I already have for other uses). The options are the four collection names. Jev returns one of them with a confidence score. Above a threshold, the system assigns the book and tells the editor it did. Between the thresholds, it proposes a collection and asks the editor to confirm with one click. Below the threshold — rare — it leaves the book uncollected and pings me.

Three details worth naming, because they generalize:

• I resisted expanding the option list. The temptation to add a fifth collection whenever one book did not quite fit was constant. Every option you add to a Choose call makes the model’s job harder and the user’s shelf browse worse. I treat the list as a product decision, not a labeling convenience.
• I did not use parallel Verify calls. I could have asked Noul-style “is this a Craft book? / is this a Memoir?” questions in parallel, as the three-shapes guide describes. But I want one collection per book, and Choose enforces that in the type system. Verify would have let overlap sneak in through the back door.
• The editor’s one-click confirm is a feature, not a workaround. The middle-confidence confirm screen is where I learned where my Choose call is wrong, and the data from it is what I use to retune the rubric. Human-in-the-loop here is the quality lever, not the embarrassment.

Image placeholder

Infographic: a before/after split comparing the old chat-based implementation of the three features to the new decision-model implementation. Left half (BEFORE, labelled in muted slate): three stacked rows each showing an AI prompt bubble, a JSON parser box, a retry-loop arrow, and a human-reading step; lots of connecting lines, a tangled feel. Right half (AFTER, labelled in emerald): the same three rows reduced to a single arrow — question → typed value → downstream code — with no parser and no retry loop. A small counter at the top says 'lines deleted' with a placeholder TODO value. Palette: off-white background, pale slate for BEFORE, emerald accent for AFTER, navy ink. No AI robots.

• Aspect ratio: 16:9
• Source size: 1280 × 720 px (sharp on retina at 760 px display width)
• Format: PNG
• Target file size: under 500 KB

Save to /public/images/blog/<slug>/, then set src and alt on this <MediaFigure /> to swap it in. Keep the ratio prop so the browser reserves the right space.

What got easier once the AI stopped writing explanations?

Four specific things, all of them boring in a good way. Boring is a compliment when you are running production software.

• Lines of code went down. The prompt files shrank, the parsers vanished, and the retry wrappers I had written to handle “the chat model returned JSON wrapped in markdown wrapped in an apology” are gone. I will not post the exact number here until I audit the diff, but it is more than I expected.
• Latency per call dropped. Short typed outputs come back fast; a decision call does not spend tokens on a paragraph it does not need. The real-user-metrics for the quote-check and the collection-choose both improved.
• Costs dropped more than I thought. Decision models are priced on short structured outputs, which my workload is. The Jev launch price of $0.042 per million input tokens with output tokens free is published in TypeSafe’s launch coverage; for a quote-check running on tens of thousands of candidates a week, that is a line item that stopped mattering.
• Tests became assertions. I write assertions like expect(call.confidence).toBeGreaterThan(0.9) and expect(call.collection).toBe("Craft"). Previously I wrote regular expressions over paragraphs and prayed. See the official OpenAI Structured Outputs guide and the Anthropic strict tool-use docs for the equivalent guarantees in those stacks — the principle is the same across vendors.

Collectively, that is the deletion dividend: the engineering value you capture not by writing new code but by deleting old scaffolding the moment the AI’s output is typed by construction. I think it is the single most under-sold reason to adopt a decision model, and I missed it in my own planning until I was midway through the migration.

What I kept exactly the same

The piece most of my friends who build in this space get wrong is thinking a decision model replaces their chat model. It does not. I still use a frontier chat model for everything a reader actually reads: draft generation when an author is unblocked, suggested rewrites on hover, longform explanations in the authoring dashboard, and the one-paragraph “why we recommended this” blurb at the top of the sampler.

The split in my head now is simple. If a human reads it, it is a chat problem. If a function reads it, it is a decision problem. For the full side-by-side of why that split works, see chatbot vs decision model and the quieter case for a whole category of models that do not talk at all in not every AI needs to talk.

What I would do differently if I started again

Three things, all of them things I did eventually but later than I should have. None of them are complicated; the mistake was assuming they would come for free.

• I would write the typed schema before any prompt. The hour I spent naming the collection enum (“Craft, Story, Field, Memoir — these four, forever”) was the single highest-leverage hour of the whole migration. Everything downstream snapped into place behind it.
• I would build the human-review surface on day one. I shipped the auto-confirm path first and then bolted on review when I discovered where the model was wrong. In hindsight I should have built the review queue before I shipped the model into production. The review surface is the sensor that keeps the model honest.
• I would audit calibration earlier. The model’s self-reported 0.9 was not always a true 90%, and I did not spot it until the quote-check started leaking two or three false positives a week. For the broader frame on this, see why “70% sure” is more useful than “yes” and the AI strategy for SMB owners piece on when to invest in that audit.

Frequently asked questions

What is a decision model and how does it fit inside a real product?

A decision model is an AI that returns a typed value — a label, a number, or a probability — instead of a paragraph. Inside a real product, it fits anywhere a small bounded judgment currently runs on either a human, a brittle keyword rule, or a chat model whose output your code has to parse. The three shapes it answers are Choose, Rate, and Verify.

Why did you move your book platform from a chat model to a decision model?

I moved the three calls where a human never read the AI’s output. The quote-accuracy check, the section-quality rating, and the collection assignment all fed downstream code, not readers. On those calls, a typed answer was always what I wanted — I was just paying a chat model to generate prose I then threw away. Moving to Jev made each call faster, cheaper, and testable.

Did you have to rebuild your whole AI pipeline?

No. I kept my chat-model calls for every reader-facing surface — draft generation, hover rewrites, longform explanations — and swapped only the three places where my code consumed the AI’s output. Each swap took a day or two: write the schema, call Jev, delete the prompt scaffolding, delete the parser. The chat parts of the pipeline did not have to change.

What is “the deletion dividend”?

It is a name I use for the engineering value you capture by deleting code when you move an AI call from prose to typed output. Specifically: the prompt scaffolding, JSON parsers, markdown-stripping, and retry logic that existed only because a chat model’s output needed to be coerced into a shape your code could use. Those lines disappear once the output is typed by construction.

Can I do this on OpenAI or Anthropic instead of Jev?

Yes. The three shapes — Choose, Rate, Verify — map directly onto OpenAI’s Structured Outputs, Anthropic’s strict tool use, and Google Gemini’s responseSchema. The vendors differ on vocabulary and pricing; the pattern is the same. I use Jev where I want decisions as the primary product and chat models where I want prose.

How did your tests change after the switch?

Tests became assertions on values. Before, I was fuzzy-matching paragraphs with regular expressions and marking test runs flaky when the chat model phrased things differently. After, I assert that the confidence is above a threshold, that the Choose returns an expected label on a golden input, and that the Rate falls inside a sanity range. The test suite stopped being an argument and became a build gate.

Should I move all my AI calls to a decision model?

No. The honest answer is: move only the ones where downstream code — not a human — consumes the output. If a user is reading the AI’s words, keep a chat model. If a function is reading the AI’s answer, a decision model will almost always be faster, cheaper, and easier to test. The split is the point.

---

If you are planning the first decision-model call inside a product you are already shipping, that call-site-by-call-site judgment is exactly what a Fractional CTO is paid to help with. One good hour on the typed schema and the review surface will save a quarter of engineering time. More pieces in this thread live in the AI Strategy archive.