Local AI Advisor
All posts

Ollama 0.35 can now make decisions, not just chat

Ollama's new decision models answer yes-or-no, pick-a-category or give-a-score questions in about 90 milliseconds on your own machine — no cloud, no prose.

By Ray, Your Local AI Advisor4 min readlocal-llmannouncementsbeginnersnew-models

You are paying a novelist to be a filing clerk.

That is how writer Logeshwaran put it this week: most of what people ask AI agents to do is not writing at all. Is this email billing or tech support? Should I retry this step? How urgent is this ticket on a 0-to-2 scale? None of that needs a paragraph of prose. It needs a typed answer, fast.

This week, local AI got a proper filing clerk. Ollama 0.35 shipped with decision models: small models that return a choice, a probability, or a score instead of text. They run entirely on your own machine.

🎬 Prefer to watch? Here's the video version of this article.

A robot clerk sorting glowing document cards into three trays

What a decision model actually is

A normal chat model answers everything by writing tokens, one at a time. A decision model does something narrower. You hand it a block of text (called a state) and a set of typed questions. It answers each one in a single round trip, with no streaming and no retries.

The questions come in three shapes:

  • choice — pick one label from your list, with a probability for every option and a confidence value.
  • noul — a yes-or-no question, answered as a numeric probability.
  • score — rate something against your own ordered rubric, returning a probability-weighted score.

Because the output is a small structured payload rather than a paragraph of reasoning, a single call costs about four output tokens. An independent explainer reported four output tokens on a typical triage call — that is why these models answer so quickly.

Diagram: text goes into a decision model, which answers choice, yes-or-no, and score questions as structured data

What shipped on September 28

Ollama 0.35 adds a new /v1/systemone endpoint built on TypeSafe AI's Jev API. Three open-weight decision models ship at launch:

  • nimble — 9B parameters from Bespoke Labs, a 9.5 GB download at Q8_0.
  • tev1 — 4B parameters from Together AI, a 4.5 GB download.
  • tev1:0.8b — 0.8B parameters from Together AI, an 812 MB download.

Bar chart of decision model download sizes: nimble 9.5 GB, tev1 4.5 GB, tev1:0.8b 812 MB

The headline number comes from Ollama's own library page: nimble averaged 91 milliseconds per decision running locally on a MacBook Pro M5 Max. An independent write-up measured the same model about 4.4x slower on an M5 Pro, which is a useful reminder that your machine decides the real number. The tiny 0.8B model is the interesting one for modest hardware: under a gigabyte, so it fits on almost any laptop with room to spare.

Getting one running is one command:

ollama pull nimble

The SDK is the open-source typesafe-sdk on PyPI, and it is the same SDK TypeSafe ships for its hosted Jev API — so code written against a local Ollama server can swap to the hosted API without a rewrite.

Cloudflare's Clef, released the next week

On October 1, Cloudflare released Clef and Clef-flash, the first models from its Workers AI team. They are decision models too — not chatbots — and they speak the same Jev API. Clef is a 27B model post-trained from Qwen3.8-27B; Clef-flash is 9B, from Qwen3.5-9B. Both are open-weight under Apache 2.0 with weights on Hugging Face for self-hosting, and both handle images and video in a 64K context window.

On Cloudflare's reported benchmarks, Clef beat the hosted Jev on 7 of 10 classification benchmarks from the Decision Index 0.2.1 suite — but Jev kept a clear lead on knowledge-heavy tests like GPQA Diamond. Cloudflare's own summary is honest about the gap: all numbers are vendor-reported, with no independent replication yet.

The one takeaway

If you run a local agent, most of its inner loop is classification: sorting incoming messages, picking which model should handle a step, checking whether a command looks unsafe, scoring results against a rubric. Until now, doing that locally meant coaxing a chat model into returning valid JSON and parsing it yourself. Now there is a model built for exactly that job, at roughly 90 milliseconds a decision, for free, with your data never leaving the machine.

That is the practical test the site exists to answer: does this change what you can do with AI on your own machine? If you have a local agent that classifies things, yes — the control plane no longer has to round-trip to the cloud.

The honest limits

A decision model cannot answer outside your list — but it can still answer wrong. One experiment found that renaming options from 0/1 to no/yes, with definitions unchanged, flipped about 70 of every 100 answers, with a zero type-error rate. So treat the confidence score as a signal for routing and escalation, not as proof.

Decision models are a complement to reasoning models, not a substitute. The reasoning model finds the answer; the decision model picks from the options, fast. All the benchmark numbers in this post are vendor-reported and not yet independently reproduced. And note that Ollama 0.35 shipped as a pre-release: a stable install needs the upgrade before the /v1/systemone endpoint appears.

If you want to see where a decision model fits in your own setup, run a workload through /check and see which of your tasks are really just sorting.

Sources