Baseten's joey-1: a 2B decision model that answers in probabilities, not paragraphs

Baseten released joey-1 on October 29, 2026: a 2-billion-parameter decision model from its in-house research lab, Base Labs. Unlike a chat model, it does not write you an answer. You give it some text or an image plus a set of questions, and it returns a label with a confidence score — for example, "billing: 0.82, technical: 0.11, other: 0.07." The weights are open on Hugging Face under the MIT license, so you can download and run them yourself rather than only calling Baseten's API.

What a decision model actually does

Most everyday AI jobs are not conversations. They are decisions: where should this ticket go, should this message be flagged, should this request be approved. A chat model answers those questions by generating sentences, which costs time and tokens on every call. A decision model skips the sentences and returns the answer directly as a single token plus a calibrated probability, in one forward pass.

joey-1 supports three question types: yes/no, a choice among named options, and an ordered score against levels you define. You can ask several questions about the same input in one request, and the input is processed once and reused across them. It accepts text and images, and it was post-trained from Qwen3.5-2B on the same 6.21-million-example mixture as its larger siblings. It also has a trained "I don't know" option for cases where it is uncertain — which matters, because an uncertain answer routed to a human is far cheaper than a confident wrong one acted on automatically.

The family, and where the small one fits

joey-1 completes a three-size family that shares one API, so Baseten says switching between sizes is a one-line change:

• mammoth-1 (27B): the most accurate, for decisions where accuracy matters most.
• elephant-1 (9B): the balanced default.
• joey-1 (2B): the high-volume, latency-sensitive option.

Baseten's own example shows why size matters here. Classifying 10 million support tickets a month at about 500 tokens each totals 5 billion tokens. At the published API prices — $0.01 per million tokens for joey-1, $0.035 for elephant-1, and $0.10 for mammoth-1 — that workload costs about $50 a month on joey-1, $175 on elephant-1, and $500 on mammoth-1. Baseten's recommended pattern is joey-1 first, with the larger models as an escalation path when joey-1 is uncertain.

On speed, Baseten reports that joey-1 serves an average 2,000-token request at a median end-to-end latency of 20 milliseconds at production scale, about 2.5 times faster than elephant-1 at 49 milliseconds. That is a vendor measurement on its own infrastructure, not an independent test on your machine — but the direction is consistent with the design: a 2B model doing one forward pass instead of generating text should be fast.

How good is it, honestly

Two benchmark numbers are worth separating. On the public split of the Jev Decision Index 0.4, joey-1 scores 63.30, placing 37th of 230 models and recording the highest score among models under 4 billion parameters. That is a public, checkable result — and it also shows the trade-off plainly: elephant-1 scores 72.24 on the same split, so the small model gives up real accuracy for its speed and price.

Baseten also reports 62.9 for joey-1 on its internal replica of the Artificial Analysis Decision Index, behind mammoth-1 at 65.9, elephant-1 at 65.2, and Liquid AI's d1-ultra at 65. And its claim that joey-1 is the strongest decision model with 3 billion or fewer parameters rests on internal testing whose details Baseten says are coming soon. Treat that claim as unproven until those details arrive; the public Jev split is the number you can verify today.

What this changes for running AI yourself

The practical point is about where small, specialized models beat general chat models. If you run a support inbox, a moderation queue, or an agent that needs a yes/no gate before taking an action, a 2B decision model is the kind of thing that can live on ordinary hardware instead of renting a frontier model by the token. Its probabilities also give you something a chat reply does not: a threshold. Below, say, 0.7 confidence, send it to a human or to a bigger model; above it, act automatically. That is a concrete way to use a small model safely.

Two limits to be honest about. First, the published self-hosting route is narrow: the model card's setup installs a pinned serving runtime and requires one CUDA GPU, and requests longer than the trained length limits are refused rather than truncated. So on day one, an NVIDIA GPU owner can self-host, while Mac, CPU-only, and llama.cpp users should expect to wait for community builds — or use the API, which supports a 16,384-token context window and zero data retention by default. Second, decision models answer only the questions you define. They do not explain themselves in prose, and a badly designed set of options will produce confident, neatly scored, wrong answers. Test yours on messy real examples before wiring approvals to it.

If you are deciding whether a decision model fits a job at all, start with the workload test: is the output a label, a route, or a score? If yes, a model like joey-1 deserves a trial run against the chat model you use today — on your own data, with your own threshold. If the output needs explanation, keep the chat model.

Sources

• Decisions for a penny: introducing joey-1, our fastest decision model, Baseten (October 29, 2026)
• Baseten Releases Joey-1, a 2B Open-Weights Decision Model, Unite.AI (October 29, 2026)
• Decision models, Baseten documentation