Lesson 8 of 10

Chat vs. coding agents

Why a model that’s great for chat can be painfully slow as a coding agent.

4 min · you’ll learn

How agents use a model
Why waits add up
When a smaller model wins

Chat: one question, one answer

Chat

You ask, it answers, you read. A short prompt, one wait, then you take your time.

Coding agent

Tools like OpenCode, Cline or Claude Code-style agents call the model dozens of times in a row, each time with thousands of tokens of files and instructions.

Every step means reading a big prompt again

Each agent step sends a big prompt (files, tool descriptions, history) that the model must read before replying. That’s prefill from the last lesson, and on laptops it’s often the slow part.

One agent step = read the prompt (prefill) + write the reply (generation)

Prefill of 5,000 new tokens Generating 300 tokens

Bigger dense model12 tok/s generation · 110 tok/s prefill

≈70 s per step× 30 steps ≈ 35 min per task

Smaller / MoE model70 tok/s generation · 650 tok/s prefill

≈12 s per step× 30 steps ≈ 6 min per task

Illustrative numbers in the range of a 48 GB laptop. In chat you wait once; an agent makes you wait every step.

Think of it like this

A 60-second wait is fine once. Thirty of them in a row is half an hour of watching a progress bar.

Same computer, same model, different verdict

Here’s the real engine rating one MacBook Pro and one model for three different jobs. Switch the model to see how an MoE changes things:

Live · uses the real engine

Same model, same computer, different workloads

MacBook Pro M4 Pro 48 GB · Qwen3.6 27B at Q4. Only the workload changes.

Casual chat

via Open WebUI

Comfortable

Comfortable for casual chat

Generation (11–14 tok/s) is adequate but not ideal for casual chat.

Generation: 11–14 tok/s
Prefill: 90–120 tok/s
Per step: ≈50 s
Per task: ≈55 s

Coding questions

via Continue

Acceptable

Acceptable for coding questions

Ingesting a prompt of 5K tokens takes ≈45 s and each step ≈75 s.

Generation: 11–14 tok/s
Prefill: 90–120 tok/s
Per step: ≈75 s
Per task: ≈2 min

Agentic coding

via OpenCode

Borderline

Borderline for agentic coding

Generation (10–13 tok/s) is slow for agentic coding — slow steps compound across many agent iterations.

Generation: 10–13 tok/s
Prefill: 90–120 tok/s
Per step: ≈75 s
Per task: ≈35 min

Switch to the MoE model: it is bigger on disk but reads only ~3B parameters per token, so every agent step is much faster. “Per task” = typical number of model calls × time per call.

Smaller can finish first

For agents, a smaller or MoE model with fast steps often finishes the whole task sooner than a bigger, smarter model that makes you wait at every step. That’s why our recommendations change with what you want to do.

Remember this

A model that’s great for chat can be painful as a coding agent. Agents need fast reading and fast writing, because they wait at every step.

Quick check

For a coding agent on a laptop, which usually matters most?

Up next · lesson 9

File formats & GPU software

GGUF, MLX, CUDA, ROCm: the names you’ll see when downloading models.

Speed: tokens per second All lessons