Open-Weight Decision Models: Can You Run Your Own Jev?

Image placeholder

Editorial, product-forward hero illustration on an off-white background — a side-by-side comparison. On the left, a small, labeled server rack with a padlock icon over its front panel, sitting on a short pedestal labeled 'SELF-HOSTED'. On the right, a soft slate-gray cloud shape with a small dollar sign inside and a thin plug-style power cord dangling from it, labeled 'HOSTED API'. Between them, a thin emerald-green two-headed arrow with the word 'CROSSOVER' above it in small navy caps. Below the pair, a short tagline: 'open weights, real tradeoffs'. Clean, flat, geometric. Palette strictly off-white background, pale slate borders, one emerald accent, one navy ink. No AI robots, no neural networks, no faces.

• Aspect ratio: 16:9
• Source size: 1600 × 900 px (sharp on retina at 760 px display width)
• Format: WEBP (preferred), PNG fallback
• Target file size: under 300 KB

Save to /public/images/blog/<slug>/, then set src and alt on this <MediaFigure /> to swap it in. Keep the ratio prop so the browser reserves the right space.

By Dr. Sanju Abraham, Fractional CTO — 180+ projects shipped, 34 funded companies. Published October 14, 2026.

Key takeaways

• Open-weight decision models exist, and more than one is production-grade. Cloudflare’s Clef, ConvAI’s Laya, Jared Palmer’s Kev, and Together AI’s Tev1 all ship weights you can download and run on your own GPUs.
• “Can I run my own Jev?” has a precise answer: no, not Jev itself — Jev is hosted-only — but yes, you can run a Jev-shaped model from one of the open-weight projects. Kev is explicitly API-compatible with Jev.
• The real tradeoff is control and privacy versus GPU cost and operations work. A 27B open-weight decision model needs roughly 85GB of VRAM; a 9B flash variant needs about 41GB. Someone has to own that.
• License fine print still matters even when the headline says “Apache 2.0”. Most of the current wave is genuinely permissive, but OpenRAIL variants add behavioral restrictions and Llama-style community licenses add a 700M-MAU trigger — always read before you build on it.
• I call the volume math behind this decision the self-host crossover — the monthly request volume above which running your own open-weight decision model is cheaper than paying hosted per-token rates after you include a loaded engineer’s time.

The short answer: You cannot run Jev itself — TypeSafe AI only ships it as a hosted API. But you can run a decision model that behaves almost exactly like Jev, because four independent teams released open-weight alternatives in the first thirty days of the category. The honest tradeoff is control and privacy on one side; GPU cost, VRAM constraints, on-call rotations, and license fine print on the other. This piece unpacks the math, the models, and the moment self-hosting actually pays.

A CTO pings me on Slack with a short message: “We like Jev. Compliance wants us to run it in our VPC. Options?” The honest answer in October 2026 is longer than it was six weeks ago. There are at least four real open-weight choices, each with its own tradeoffs, and a license page you should read before you commit any infrastructure budget.

What is an open-weight decision model?

An open-weight decision model is a model whose trained weights you can download, inspect, and run on your own hardware — unlike a hosted API, where the weights live only on the vendor’s servers. It is still a decision model, meaning it returns a typed answer plus a confidence score, not a paragraph of prose. For the broader definition of the category, see not every AI needs to talk and the plain-English guide to Jev itself.

Three things usually travel together under the “open-weight” label:

• The weights themselves, posted on Hugging Face under a declared license.
• An inference recipe — code, a config file, or an Ollama image — so you can run the model without reverse-engineering the architecture.
• A benchmark card that reports accuracy, calibration, latency, and memory footprint on standard hardware.

Not every “open” release includes all three. Cloudflare published Clef weights but kept the training data private, which The Register noted is closer to “open-weight” than “open-source.” That distinction matters if your legal team cares about data provenance; it rarely matters if you only care about running the model.

Who is actually shipping open-weight decision models?

Four groups shipped production-ready open-weight decision models between September 18 and October 1, 2026. Named entities with verified Hugging Face pages or vendor announcements:

• Kev, from Jared Palmer. A Jev-API-compatible family in 0.6B, 4B, 9B, and 27B sizes, built on Qwen 3.5 under Apache 2.0. Code written against Jev can switch to Kev by changing a base URL and a model name. The smaller sizes are LoRA adapters with a pointer head; the 27B is a full fine-tune.
• Laya, from ConvAI Innovations. A ~421M-parameter ModernBERT-based decision model with a multilingual 322M variant on mmBERT, Apache 2.0, with vendor-reported latency around 33–40 ms per decision on a Tesla T4. Honest caveat the model card flags: zero-shot calibration is poor out of the box (reported ECE 0.466); a one-temperature-per-question-type refit on your own data pulls it down to about 0.081.
• Tev1, from Together AI. An experimental 0.8B and 4B family, fine-tuned from Qwen3.5, distributed on Hugging Face and Ollama. Marked “experimental” by the vendor; one third-party review describes it as “an earlier open experiment that took a different route” from Jev.
• Cloudflare’s Clef and Clef-flash. Apache 2.0, 27B and 9B parameters respectively, released October 1, 2026, hosted on Workers AI and downloadable from Hugging Face. 64K context, four images per request. Per secondary coverage, Clef-flash runs on any GPU with at least 41GB of VRAM; Clef requires roughly 85GB at 64K context and single concurrency.

Two further names worth mentioning for completeness: AWS Strands Decider 2B, reported Apache 2.0, and the community rebuilds SemIF and DJev — permissively licensed but less obviously production-grade. For the full category view, see 13 decision APIs in 3 weeks.

Name	License	Parameters	Context window	Verified or reported
Kev (0.6B / 4B / 9B / 27B)	Apache 2.0	0.6B – 27B	Qwen3.5 base (32K typical)	Verified via Hugging Face / OpenRouter
Laya (English)	Apache 2.0	~421M (ModernBERT)	512	Verified via Hugging Face model card
Laya-multilingual	Apache 2.0	~322M (mmBERT)	1024	Verified via Hugging Face model card
Tev1 (0.8B / 4B)	Open weights (check model card)	0.8B / 4B	Qwen3.5 base	Reported via OpenRouter / Ollama
Clef	Apache 2.0	27B	64K	Verified via Cloudflare announcement
Clef-flash	Apache 2.0	9B	64K	Verified via Cloudflare announcement
Strands Decider 2B	Apache 2.0 (reported)	2B	Not published	Reported via secondary coverage

The real tradeoff: what you gain, what you pay for

Self-hosting an open-weight decision model buys you two things — control and privacy — in exchange for two obligations — GPU cost and operations work. None of these are symmetric; a CTO’s job is to be honest about which side matters more for a given workload.

What you gain:

• Data stays on your network. No request body leaves your VPC. For healthcare, legal, defense, and some financial workloads, this is the entire reason the conversation exists.
• Price-per-token becomes price-per-GPU-hour. Above a certain volume, hosted per-million-token pricing crosses an open-weight all-in cost. That crossover point is the point of this piece.
• The model does not change under you. Hosted APIs patch weights without notice. Your own weights only change when you decide.
• Vendor lock-in gets weaker. If Kev is Jev-compatible and Clef can be adapted with a thin shim, your production code tolerates provider churn — more on that in Jev-compatible APIs: how switching costs collapsed.

What you pay for:

• GPU hardware or GPU-hour rentals. 85GB of VRAM for a 27B model is one H100 at list price or two 48GB cards. Clef-flash and the 9B tier of Kev drop that to ~41GB.
• An on-call rotation. Something has to page someone when the inference endpoint returns 503s at 3 a.m.
• Model-ops work no hosted API asks of you. Quantization, calibration refits (ask Laya), observability, autoscaling, drift monitoring.
• A security story. The model, its weights, and its logs are now your responsibility; so are the controls proving so to auditors.

What does self-hosting actually cost in 2026?

Three cost buckets, in order of surprise: GPUs, ops time, and the boring surrounding infrastructure. None of these are optional. All of them scale less steeply than hosted per-token prices once you cross a volume threshold — which is why the question is worth asking at all.

• GPUs. A single NVIDIA H100 (80GB) covers Clef-flash or Kev-9B with room for context. A 27B class model wants either an H100 with the right quantization or two 48GB cards. On-demand hourly pricing in late 2026 sits in the low single digits per hour on most clouds; one GPU running 24/7 lands near $2,000–$4,000 a month before commitments. Reserved or on-prem flips the math.
• Ops time. Call it one-tenth of a loaded engineer for a stable single-model deployment, more during ramp or incidents. At a loaded cost of ~$20,000/mo for a senior infrastructure engineer, that is another $2,000/mo baseline.
• Everything else. Observability (latency histograms, error tracking), autoscaling, model-registry hygiene, calibration refits on new data (particularly for Laya), and a security pass before anyone sees a real customer. A few hundred to a few thousand a month depending on tooling.

Compare that to a hosted baseline. Jev’s published rate in late 2026 sat near $0.042 per million input tokens. At 100M requests per month of roughly 500 tokens each, you spend close to $2,100 in token cost — no GPUs, no ops, no observability — on a hosted plan. Below that volume, hosted is almost always cheaper. Above it, the math starts leaning the other way, and privacy wins the rest of the argument.

Image placeholder

Infographic: a simple cost-crossover line chart on an off-white background. Horizontal axis labeled 'monthly request volume' with ticks at 1M, 10M, 100M, 1B. Vertical axis labeled 'monthly cost ($)'. Two lines: a blue hosted-API line that slopes steadily upward (linear-ish), and an emerald self-hosted line that starts higher (flat baseline for GPU + ops) and rises much more slowly. The two lines cross around 80M–120M requests, with a small navy dot and label 'CROSSOVER'. Shading pale blue below the hosted line on the left of the crossover (hosted wins), pale emerald below the self-hosted line on the right (self-hosted wins). A thin dashed vertical line at the crossover. Palette: off-white background, pale slate borders, one blue accent, one emerald accent, navy ink. Clean, flat, Linear-editorial feel. No AI robots.

• Aspect ratio: 16:9
• Source size: 1280 × 720 px (sharp on retina at 760 px display width)
• Format: PNG
• Target file size: under 500 KB

Save to /public/images/blog/<slug>/, then set src and alt on this <MediaFigure /> to swap it in. Keep the ratio prop so the browser reserves the right space.

Where is the self-host crossover for your workload?

The self-host crossover is the request volume at which running your own open-weight decision model is cheaper than paying hosted per-token rates, after you add a loaded engineer’s time and the GPU bill. It is a product-math question, not an infrastructure preference. Three inputs decide it:

• Your monthly request volume. Count the real number, not a projection. For most SMBs applying decision models to support routing, moderation, or lead scoring, this sits between 100K and 10M requests a month — well below any crossover.
• Your blended hosted price. The published $0.042/M for Jev is a reasonable floor. Clef’s $0.24/M, per The Register, is a reasonable ceiling for the hosted versions of the same open weights.
• Your realistic ops load. If a single engineer already runs ten services, adding one more that needs calibration refits and GPU babysitting is a bigger ask than the spreadsheet admits.

For most teams the honest answer is: the crossover is further than you think, and privacy — not price — is why you self-host first. The public pricing on Jev and OpenAI’s Decisions API is low enough that the per-token argument alone rarely clears the ops overhead until request volume is well past what most SMB and mid-market workloads generate. Where self-hosting earns its keep immediately is in regulated workloads, air-gapped environments, and anywhere a compliance officer will not sign the vendor’s data-processing agreement. For broader signal on when AI investments pay off at this layer, see an AI strategy for SMB owners.

Does “Apache 2.0” really mean “use it however you want”? License fine print

Apache 2.0 means yes, mostly — but the open-weight ecosystem also ships models under OpenRAIL variants and Llama-style community licenses, which carry restrictions that only reveal themselves in production. Read the license page, not the model card header.

• Apache 2.0 is a permissive open-source license maintained by the Apache Software Foundation. No revenue thresholds. No use restrictions. You can embed, fine-tune, redistribute, and sell. Kev, Laya, Clef, Clef-flash, and the reported AWS Strands Decider 2B all declare Apache 2.0.
• OpenRAIL-M and its variants (from the Responsible AI Licenses working group) permit commercial use but add behavioral restrictions — specific prohibited uses that travel with every derivative. The BigScience OpenRAIL-M, for example, bars using the model to produce harmful content or to “provide medical advice and medical results interpretation.” No revenue thresholds; still worth your legal team’s five-minute read.
• Llama-style community licenses (Meta’s family) add a 700-million monthly active user trigger that forces a separate commercial agreement above that scale. That is not an OpenRAIL pattern and not a revenue number — but if your product can plausibly cross it, know it exists.
• Model-specific “community” licenses sometimes do have revenue caps — most famously, some Stable Diffusion releases from Stability AI have carried annual-revenue thresholds that flip commercial use into a paid tier. None of the current decision-model wave uses this pattern, as far as I can verify, but the pattern exists in adjacent model families and teams migrating from image generation sometimes assume it is universal. It is not.

Rule of thumb: if the model card says Apache 2.0 and links to the canonical Apache 2.0 text, you are in permissive-license land. Anything else — OpenRAIL, “community,” “research-only,” “non-commercial under X revenue” — buy your lawyer coffee first.

When should your team actually self-host?

Four green lights that justify the lift, four red flags that say stick with hosted. This is not a checklist for an engineer; it is one for whoever owns the P&L.

Green lights:

• A compliance regime names your model and your weights. HIPAA, SOC 2 Type II with specific AI controls, FedRAMP, or a sector-specific framework that requires model residency.
• Your request volume sits comfortably above your hosted-vs-GPU+ops crossover, consistently. Spiky workloads do not count; sustained load does.
• You already run GPU inference for something else. The marginal cost of adding one more model to an existing MLOps footprint is dramatically lower than standing one up from scratch.
• You have a specific vendor-independence requirement — a regulated procurement rule, a sovereign-cloud mandate, or a board decision that any single AI vendor is a concentration risk.

Red flags:

• You are below 10M requests a month and nobody is asking about compliance. The crossover is not close; your engineering time is the expensive input, not tokens.
• You do not already run production GPU workloads. The first one is the most expensive one.
• Your team cannot name who gets paged when inference falls over at 3 a.m. That blank is a leading indicator.
• “We want to save money on AI” is the main reason. It almost never is, once the ops bill lands. For the broader portfolio view on how to spend AI dollars well, see our AI Strategy archive and the related piece on calibration — the feature nobody talks about.

A pragmatic middle path many teams miss: run both. Use a hosted API for the long tail of low-volume, low-risk decisions and self-host the one or two workloads that justify it. The adapter pattern described in Jev-compatible decision APIs makes this cheap. Pick one open-weight model per sensitive workload; leave the rest on the hosted plan.

Frequently asked questions

What is an open-weight decision model?

An open-weight decision model is a decision model — one that returns a typed answer plus a confidence score instead of generating prose — whose trained weights are downloadable and runnable on your own hardware. Current examples include Kev (Jared Palmer, Apache 2.0), Laya (ConvAI, Apache 2.0), Tev1 (Together AI, experimental), and Cloudflare’s Clef and Clef-flash (Apache 2.0). All ship on Hugging Face.

Can I run Jev locally?

No. Jev is TypeSafe AI’s hosted-only decision model and the weights are not publicly released. What you can do is run a Jev-compatible open-weight model — most directly Kev, which uses the same API shape, so code written against Jev switches by changing a base URL and model name. Clef, Laya, and Tev1 require thin adapters but provide the same decision-model behavior.

What are the open-weight alternatives to Jev?

The ones with public Hugging Face pages and declared licenses as of mid-October 2026 are Kev, Laya, Tev1, Clef, and Clef-flash — plus the reported AWS Strands Decider 2B. See the table above for licenses, parameter counts, and context windows. For a wider view of the whole category, see 13 decision APIs in 3 weeks.

When does self-hosting a decision model actually save money?

When your monthly request volume is high enough that hosted per-token spend exceeds the loaded cost of one GPU plus the fraction of an engineer required to operate it. For most SMB and mid-market teams that volume is further out than they expect — often north of 50–100 million requests a month. The usual reason to self-host earlier is privacy or compliance, not cost. See the self-host crossover section above.

What hardware do I need to run Clef or Kev?

Per secondary coverage of Cloudflare’s launch, Clef-flash (9B) runs on any GPU with at least ~41GB of VRAM; the 27B Clef wants about 85GB of VRAM at 64K context. Kev-9B lands in a similar footprint; Kev-4B and Kev-0.6B are substantially smaller and comfortably fit a single 24GB consumer card. Laya at ~421M parameters runs on nearly any current GPU.

What about the license fine print — can I use these commercially?

For Apache 2.0 (Kev, Laya, Clef, Clef-flash, and the reported Strands Decider 2B): yes, commercially, including derivatives — no revenue threshold and no behavioral restrictions. For OpenRAIL variants: yes, commercially, but with specific prohibited uses that travel to every derivative. For Llama-style community licenses: yes until 700M monthly active users, after which Meta’s commercial terms engage. Always read the license page linked from the model card.

Is “open-weight” the same as “open-source”?

No. Open-weight means the weights are downloadable under a declared license. Open-source, in the full OSI sense, also implies that the training data, training code, and recipe are public. Most of the current decision-model wave is open-weight; Cloudflare explicitly confirmed to The Register that Clef’s training data is private, which makes “open-weight” the honest label. For most buyers this distinction rarely blocks use; for a few — academic reproducibility, regulatory audit — it matters.

---

If “should we self-host Clef, or stay on Jev, or run both?” is the real question in a planning doc on your desk right now, that call sits exactly where a Fractional CTO earns their keep. One hour of the right conversation picks a path that saves a quarter of engineering budget. More pieces in this thread live in the AI Strategy archive and the companion pieces on choosing a decision API in 2026 and decision models in the agent stack.