needle3 on local hardware
Estimated comfort for repository coding with Aider on a MacBook Pro M4 Pro 48GB.
needle3Open source
Cactus-Compute · apache-2.0 · 143,608 downloads
- Parameters
- 0.2B estimated
- Context window
- 8K tokens
- Attention
- Sliding-window + global layers
- Layers · KV heads · head dim
- 20 · 2 · 64
- KV cache
- 0 KB/token · 0.0 GB at 32K
- Tool calling
- basic (inferred — no supporting signal)
- Thinking
- basic (inferred — no supporting signal)
- Vision
- basic (inferred — no supporting signal)
- Commercial use
- allowedRetain the required license and attribution notices
- Import confidence
- low · capabilities estimated
Weights by quantization
Q3_K_M
0.1 GB
computed · low confidence
Q4_K_M
0.1 GB
computed · low confidence
Q5_K_M
0.1 GB
computed · low confidence
Q6_K
0.1 GB
computed · low confidence
Q8_0
0.2 GB
computed · low confidence
F16 / BF16
0.3 GB
computed · low confidence
- • Parameter count estimated from the architecture (the Hub reports none for this repo).
- • Assumes GGUF / MLX conversions of this model exist (they do for most popular models). Sizes are computed from bits per weight.
- • No chat template found; tool calling is inferred as limited (low confidence).
- • toolCalling: basic, inferred from missing signals (low confidence).
- • thinking: basic, inferred from missing signals (low confidence).
- • vision: basic, inferred from missing signals (low confidence).
- • Capability ratings are estimated from model size and name — this model hasn't been assessed by hand.
needle3 — 6-bit
Technically Runs for your workload
Speed
1130–1530 tok/s
Excellent
Memory left
36 GB
of 48 GB
First word in
under 1 s
Excellent
All the details
- Your setup
- MacBook Pro M4 Pro 48 GB
- Workload
- Repository coding with Aider
- Runtime → tool
- LM Studio → Aider
- Memory used
- 0.9 GB
- Context
- 8K · up to 8K practical
- Tool support
- Official
- Use-case fit
- Poorly suited
Why?
- Model fits with approximately 36 GB of memory headroom.
- LM Studio supports hardware acceleration on this machine (Metal (Apple GPU)).
- Aider can connect to LM Studio (officially supported).
- Generation (1130–1530 tok/s) should be appropriate for interactive coding.
- Ingesting a prompt of 7K tokens takes under 1 s; prompt processing keeps up with interactive coding.
- 8K window; generation drops to 1130–1530 tok/s when it is full. The model is limited to 8K.
- needle3 supports at most 8K context; 32K was requested.
- Weak coding capability for interactive coding.
Why “Technically Runs” rather than “Borderline”?
- Workload suitability: Weak coding capability for interactive coding.
- Context: 8K window; generation drops to 1130–1530 tok/s when it is full. The model is limited to 8K.
How sure are we?
Medium confidenceEstimated from bandwidth and compute, calibrated against 2 verified benchmarks of other models on this same chip (measured with a different engine, so calibration is partial).
- Calibrated with verified benchmarks of other models on this chip.
- Hybrid/sliding attention performance varies more between runtimes.
- Some specifications come from lower-confidence sources.
Medium confidence