What is local AI?
Running an AI model on your own computer instead of in someone else’s cloud.
3 min · you’ll learn
- The three pieces: app, runtime and model
- Why people run AI at home
- What your computer has to do
ChatGPT runs in a data centre. Local AI runs on your computer.
When you use ChatGPT or Claude, your message travels over the internet to a huge computer in a data centre, which writes the reply and sends it back.
Local AI means downloading an AI model and running it on your own laptop or desktop instead. The same kind of chat, but the “thinking” happens on your machine.
Three pieces work together
You only ever need these three things, and the first two are free apps:
- 1. An appWhere you type and reade.g. LM Studio, Open WebUI, a coding agent
- 2. A runtimeThe engine that runs the modele.g. Ollama, llama.cpp, MLX
- 3. A modelThe AI itself: one big filee.g. Qwen, Gemma, Llama… 2–60 GB
Think of it like this
The model is a record, the runtime is the record player, and the app is the speaker you listen through. Many apps (like LM Studio) bundle the player and speaker together.
Why people do it
- PrivateYour chats, files and code never leave your machine.
- No subscriptionOnce it’s downloaded, it’s free to use as much as you like.
- Works offlineOn a plane, on a train, or when the internet is down.
- Yours to tweakPick the model, change settings, connect your own tools.
The catch: your computer sets the limits
A data centre has machines worth hundreds of thousands of dollars. Your laptop doesn’t. So the big question in local AI is always: which models will run well on the computer I have?
The answer mostly comes down to two things you’ll learn about next: how much memory your computer has, and how fast it can read that memory.
Remember this
Quick check
You download a model and chat with it on a plane with Wi-Fi off. Does it still work?
Up next · lesson 2
Memory: where the model lives
A model has to sit in memory while it runs, and what kind of memory decides how fast it is.