Qwen3.8-27B-NVFP4 on local hardware
Estimated comfort for repository coding with Aider on a MacBook Pro M4 Pro 48GB.
Qwen3.8-27B-NVFP4Open source
nvidia · apache-2.0 · 583,740 downloads
VisionReasoning / thinking mode
- Parameters
- 18B exact
- Context window
- 256K tokens
- Attention
- Hybrid linear + full attention
- Layers · KV heads · head dim
- 64 · 4 · 256
- KV cache
- 64 KB/token · 2.0 GB at 32K
- Tool calling
- good (from chat template)
- Thinking
- good (from chat template)
- Vision
- good (from config)
- Commercial use
- allowedRetain the required license and attribution notices
- Import confidence
- medium · capabilities estimated
Weights by quantization
Q3_K_M
8.3 GB
computed · low confidence
Q4_K_M
10 GB
computed · low confidence
Q5_K_M
12 GB
computed · low confidence
Q6_K
14 GB
computed · low confidence
Q8_0
18 GB
computed · low confidence
F16 / BF16
34 GB
computed · low confidence
- • Assumes GGUF / MLX conversions of this model exist (they do for most popular models). Sizes are computed from bits per weight.
- • vision: good, inferred from config (low confidence).
- • Capability ratings are estimated from model size and name — this model hasn't been assessed by hand.
Qwen3.8-27B-NVFP4 — 4-bit
Acceptable for your workload
Speed
15–20 tok/s
Adequate
Memory left
24 GB
of 48 GB
First word in
≈85 s
Slow
All the details
- Your setup
- MacBook Pro M4 Pro 48 GB
- Workload
- Repository coding with Aider
- Runtime → tool
- LM Studio → Aider
- Memory used
- 13 GB
- Context
- 32K · up to 128K practical
- Tool support
- Official
- Use-case fit
- Partly suited
Why?
- Model fits with approximately 24 GB of memory headroom.
- LM Studio supports hardware acceleration on this machine (Metal (Apple GPU)).
- Aider can connect to LM Studio (officially supported).
- Generation (15–20 tok/s) is adequate but not ideal for interactive coding.
- 32K window; generation drops to 14–19 tok/s when it is full.
- Weak coding capability for interactive coding.
- Ingesting a prompt of 14K tokens takes ≈85 s and each step ≈80 s; across ~4 calls a task takes ≈7 min.
Why “Acceptable” rather than “Comfortable”?
- Prompt processing: Ingesting the first prompt (14K tokens) takes ≈85 s; each of the ~4 follow-up turns takes ≈80 s (≈7 min per task).
- Workload suitability: Weak coding capability for interactive coding.
- Generation: Expected 15–20 tok/s at ~16K tokens of context. Coding with repository context is comfortable from about 28 tok/s.
- Context: 32K window; generation drops to 14–19 tok/s when it is full.
How sure are we?
Medium confidenceEstimated from bandwidth and compute, calibrated against 2 verified benchmarks of other models on this same chip (measured with a different engine, so calibration is partial).
- Calibrated with verified benchmarks of other models on this chip.
- Hybrid/sliding attention performance varies more between runtimes.
Medium confidence