Local AI Advisor
All lessons
Lesson 10 of 10

Run your first model

Install an app, download a model and chat with it, all in about ten minutes.

5 min · you’ll learn

  • The easiest app to start with
  • Picking a first model
  • Checking it runs well
1

Find out what your computer can handle

Before downloading anything, check which models will run well on your machine. It takes about a minute and saves you downloading a 20 GB file that crawls.

Check my computer

Not sure? A 9B model like Qwen3.5 9B (about 6 GB) is a great first try on most computers with 16 GB of memory or more. On 8 GB, close your other apps first and try a 3–4B model such as Llama 3.2 3B: it works, but it will feel tight.

2

Install an app

  • LM Studio

    Easiest · recommended

    A normal desktop app for Mac, Windows and Linux. Search for a model, click download, start chatting. No terminal needed.

    Get LM Studio

  • Ollama

    Great for tinkerers

    Runs quietly in the background and is controlled with short terminal commands. Many apps and coding tools connect to it.

    Get Ollama

Which one?

If you’ve never used a terminal, pick LM Studio. You can always add Ollama later; they can live side by side.

3

Download your first model

In LM Studio, open the search tab, type Qwen3.5 9B, and pick the version marked Q4_K_M (or “4-bit” MLX on a Mac). It even warns you if a model is too big for your computer.

Prefer the terminal? Copy one of these:

ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M

Downloads 5.68 GB (Q4_K_M). Ollama's default context is small: set OLLAMA_CONTEXT_LENGTH=8192 before `ollama serve` for long prompts.

4

Say hello, and check the speed

Ask it something. Most apps show the speed in tokens per second under each answer. Compare it with what you learned in the speed lesson: 10+ tok/s is fine for chat, 30+ feels fast.

Too slow? Try a smaller model or a lower quant, close heavy apps, or shorten the context in the app’s settings. Fast and comfy? Try the next size up.

Remember this

Check your computer, install LM Studio, download a Q4 model that fits comfortably, and start chatting. You’re running local AI!

Quick check

You have a laptop with 16 GB of memory and want a first model. Which is the sensible pick?

That’s the whole course!

You know more about local AI than most people. Time to put it to work.

Check my computer
File formats & GPU softwareAll lessons