File formats & GPU software
GGUF, MLX, CUDA, ROCm: the names you’ll see when downloading models.
3 min · you’ll learn
- GGUF vs. MLX
- What CUDA, ROCm, Vulkan and Metal are
- Which one you need
GGUF: the format that runs everywhere
GGUF is a single-file model format used by llama.cpp, and therefore by Ollama, LM Studio, Jan and many others. It runs on Mac, Windows and Linux, on a CPU or any GPU.
Quantization names like Q4_K_M and Q8_0 come from GGUF. If in doubt, pick GGUF.
MLX: Apple’s fast lane for Macs
MLX is Apple’s own machine-learning framework and format, built only for Apple Silicon. LM Studio can use it, and so can mlx-lm on the command line.
On a Mac it’s often 10–30% faster than GGUF, especially for reading prompts and for MoE models. MLX models are labelled “4-bit”, “8-bit” and so on.
GPU software: the driver that does the heavy lifting
Runtimes talk to your graphics chip through a low-level platform. Which one depends on your hardware:
- NVIDIA → CUDAThe most mature option. Every runtime supports it first, so things generally just work.
- AMD → ROCm or VulkanROCm works well on supported cards, especially on Linux. Vulkan runs on almost any GPU and is easy to set up.
- Mac → MetalBuilt into macOS. llama.cpp and MLX use it automatically; you don’t install anything.
- Intel → Vulkan / SYCLIntel Arc and newer laptop chips work through Vulkan or Intel’s own backends.
Good news: apps like LM Studio and Ollama pick the right one for you.
Remember this
Quick check
You have a Windows PC with an NVIDIA card. Which combo is the safe default?
Up next · lesson 10
Run your first model
Install an app, download a model and chat with it, all in about ten minutes.