Meta-Llama-3-8B-Instruct on local hardware
Estimated comfort for repository coding with Aider on a MacBook Pro M4 Pro 48GB.
Meta-Llama-3-8B-InstructOpen weights (restricted)
meta-llama · llama3 · 474,388 downloads
Gated
- Parameters
- 8.0B exact
- Context window
- 8K tokens
- Attention
- Standard (full attention)
- Layers · KV heads · head dim
- 32 · 8 · 128
- KV cache
- 128 KB/token · 4.0 GB at 32K
- Tool calling
- basic (from chat template)
- Thinking
- basic (from chat template)
- Vision
- basic (inferred — no supporting signal)
- Commercial use
- conditionalCommunity license and acceptable-use restrictions apply; large services need separate permission
- Import confidence
- medium · capabilities estimated
Weights by quantization
Q3_K_M
3.7 GB
gguf file size
Q4_K_M
4.6 GB
gguf file size
Q5_K_M
5.3 GB
gguf file size
Q6_K
6.1 GB
gguf file size
Q8_0
8.0 GB
gguf file size
- • Architecture read from unsloth/Llama-3-8B-Instruct.
- • vision: basic, inferred from missing signals (low confidence).
- • Capability ratings are estimated from model size and name — this model hasn't been assessed by hand.
- • The chat template doesn't mention tools, so tool calling is assumed to be limited.
- • Commercial use conditional: Community license and acceptable-use restrictions apply; large services need separate permission.
- • GGUF sizes come from community conversions matched by model name; equivalence and runtime support are not verified. MLX conversion availability is unverified.
Meta-Llama-3-8B-Instruct — 8-bit
Acceptable for your workload
Speed
17–25 tok/s
Adequate
Memory left
26 GB
of 48 GB
First word in
≈20 s
Comfortable
All the details
- Your setup
- MacBook Pro M4 Pro 48 GB
- Workload
- Repository coding with Aider
- Runtime → tool
- LM Studio → Aider
- Memory used
- 10 GB
- Context
- 8K · up to 8K practical
- Tool support
- Official
- Use-case fit
- Poorly suited
Why?
- Model fits with approximately 26 GB of memory headroom.
- LM Studio supports hardware acceleration on this machine (Metal (Apple GPU)).
- Aider can connect to LM Studio (officially supported).
- Ingesting a prompt of 7K tokens takes ≈20 s; prompt processing keeps up with interactive coding.
- Generation (17–25 tok/s) is adequate but not ideal for interactive coding.
- 8K window; generation drops to 17–25 tok/s when it is full. The model is limited to 8K.
- Meta-Llama-3-8B-Instruct supports at most 8K context; 32K was requested.
- Weak coding capability for interactive coding.
Why “Acceptable” rather than “Comfortable”?
- Workload suitability: Weak coding capability for interactive coding.
- Context: 8K window; generation drops to 17–25 tok/s when it is full. The model is limited to 8K.
- Generation: Expected 17–25 tok/s at ~7K tokens of context. Coding with repository context is comfortable from about 28 tok/s.
- Prompt processing: Ingesting the first prompt (7K tokens) takes ≈20 s; each of the ~4 follow-up turns takes ≈45 s (≈3 min per task).
How sure are we?
Medium confidenceEstimated from bandwidth and compute, calibrated against 2 verified benchmarks of other models on this same chip (measured with a different engine, so calibration is partial).
- Calibrated with verified benchmarks of other models on this chip.
Medium confidence