Local AI Advisor
All lessons
Lesson 5 of 10

Why “it fits” isn’t enough

A model can fit in memory and still make your computer miserable to use.

3 min · you’ll learn

  • Why you need spare memory
  • What “spilling” to RAM does
  • How we rate setups
1

“It fits” is the minimum, not the goal

Most calculators only ask one question: does the model fit in memory? But your operating system, browser and other apps need memory too. A model that leaves no room for them makes the whole computer stutter.

A laptop with 16 GB of memory18 / 16 GB
System
Your apps
Model
  • System · 4 GB
  • Your apps · 3 GB
  • Model · 9.5 GB
  • Chat memory · 1.5 GB

It “fits”… but your computer crawls

The model squeezes out your apps. The computer starts swapping to disk and everything, including the AI, slows to a crawl.

Think of it like this

A suitcase you can only close by sitting on it technically “fits”. But you can’t get anything else in, and good luck finding your toothbrush.

2

Fitting in the wrong kind of memory

On a PC with a graphics card there’s a second trap. If the model is slightly too big for the card’s fast VRAM, the extra layers spill into slow system RAM. It still runs, but every word now waits for the slow part.

Generation speed ceiling vs. how much of the model spilled out of VRAM

All in VRAM
≈50 tok/s
10% in RAM
≈23 tok/s
25% in RAM
≈13 tok/s
50% in RAM
≈7 tok/s
Illustrative upper bounds for a 20 GB model on a GPU with ~1,000 GB/s VRAM and a PC with ~80 GB/s dual-channel DDR5. Moving just a quarter of the model to system RAM cuts the ceiling by roughly 4×.
3

So we rate how it will feel, not just whether it fits

Every result on this site uses the same five-step scale:

  • ExcellentSubstantial headroom for this workload. Expect a highly responsive experience.
  • ComfortableA good practical setup for this workload: reasonable latency, memory headroom and context.
  • AcceptableUsable, but you will notice compromises.
  • BorderlineTechnically usable, but likely frustrating for this workload.
  • Technically RunsLoads and executes, but the experience is inappropriate for this workload.

Remember this

Leave a few GB free for your computer, and keep the model inside fast memory. That’s the difference between “runs” and “runs well”.

A 16 GB laptop runs a 15 GB model. What’s most likely?

Up next · lesson 6

Context: the model’s short-term memory

How much text a model can keep in mind at once, and what that costs.