Bwen-8B on local hardware
Estimated comfort for repository coding with Aider on a MacBook Pro M4 Pro 48GB.
Bwen-8BOpen source
allenai · apache-2.0 · 410 downloads
- Parameters
- 8.3B exact
- Context window
- 32K tokens
- Attention
- Standard (full attention)
- Layers · KV heads · head dim
- 36 · 8 · 128
- KV cache
- 144 KB/token · 4.5 GB at 32K
- Tool calling
- basic (inferred — no supporting signal)
- Thinking
- basic (inferred — no supporting signal)
- Vision
- basic (inferred — no supporting signal)
- Commercial use
- allowedRetain the required license and attribution notices
- Import confidence
- medium · capabilities estimated
Weights by quantization
Q4_K_M
4.7 GB
gguf file size
- • No chat template found; tool calling is inferred as limited (low confidence).
- • toolCalling: basic, inferred from missing signals (low confidence).
- • thinking: basic, inferred from missing signals (low confidence).
- • vision: basic, inferred from missing signals (low confidence).
- • Capability ratings are estimated from model size and name — this model hasn't been assessed by hand.
- • GGUF sizes come from community conversions matched by model name; equivalence and runtime support are not verified. MLX conversion availability is unverified.
Bwen-8B — 4-bit
Borderline for your workload
Speed
25–30 tok/s
Comfortable
Memory left
27 GB
of 48 GB
First word in
≈45 s
Adequate
All the details
- Your setup
- MacBook Pro M4 Pro 48 GB
- Workload
- Repository coding with Aider
- Runtime → tool
- LM Studio → Aider
- Memory used
- 10 GB
- Context
- 32K · up to 32K practical
- Tool support
- Official
- Use-case fit
- Poorly suited
Why?
- Model fits with approximately 27 GB of memory headroom.
- LM Studio supports hardware acceleration on this machine (Metal (Apple GPU)).
- Aider can connect to LM Studio (officially supported).
- Generation (25–30 tok/s) should be appropriate for interactive coding.
- Ingesting a prompt of 14K tokens takes ≈45 s, and each step ≈40 s — noticeable waits.
- 32K window; generation drops to 18–25 tok/s when it is full. KV cache is large (≈4.5 GB).
- Weak coding capability for interactive coding.
Why “Borderline” rather than “Acceptable”?
- Workload suitability: Weak coding capability for interactive coding.
- Prompt processing: Ingesting the first prompt (14K tokens) takes ≈45 s; each of the ~4 follow-up turns takes ≈40 s (≈3 min per task).
- Context: 32K window; generation drops to 18–25 tok/s when it is full. KV cache is large (≈4.5 GB).
How sure are we?
Medium confidenceEstimated from bandwidth and compute, calibrated against 2 verified benchmarks of other models on this same chip (measured with a different engine, so calibration is partial).
- Calibrated with verified benchmarks of other models on this chip.
Medium confidence