Local AI Advisor
All posts

IBM's Granite Speech 5.0: transcribe audio locally with a 2B open model

IBM released Granite Speech 5.0 2B, an Apache 2.0 speech model that transcribes and translates in 6 languages — and it runs in llama.cpp.

By Ray, Your Local AI Advisor4 min readlocal-llmnew-modelsbeginnersai-news

Almost every voice transcription you use today sends your audio to someone else's server: meeting notes, lecture recordings, interview tapes. On October 22, IBM released Granite Speech 5.0 2B, an open speech-language model that does the same job on your own machine. It transcribes speech into text in 6 languages, translates some of them into English, and ships under Apache 2.0 — meaning you can use it, including commercially, without asking permission.

Sound waves traveling from a microphone into a laptop and emerging as written text, illustrating local voice transcription

What changes for you

The headline for anyone running AI locally: this model runs in llama.cpp and mlx-audio on Apple Silicon, plus Transformers and vLLM if you have a GPU. IBM recommends the Q8_0 GGUF build — for a 2B-parameter model that works out to a file of roughly 2 GB, small enough to run on a CPU-only laptop.

A few practical details from IBM's release notes:

  • Recommended audio length: about 2 minutes per segment. This is a guided workflow, not a multi-hour podcast in one pass.
  • Lower-bit quants lose accuracy on audio longer than roughly a minute, per IBM. Stick with Q8_0 for the full model unless you measure the quality hit yourself.
  • Keyword biasing: you can hand it a list of up to 2,000 terms — product names, people, acronyms — and it steers toward spelling them right. On the Earnings-22 test, keyword F1 rose from 67.2 to 90.1 with correct lists, and still reached 82.8 with 2,000-term lists mixing right and wrong terms. If your domain has jargon, this is the feature to use.
  • Language ID prompting: name the input language in the prompt to steer transcription.
  • Punctuation and capitalization on request, with capitalization F1 ranging from 90.16 to 99.51 across test sets.

The realistic use cases: lecture recordings, meeting notes, interview transcripts — anything where you would rather not upload the audio. That privacy angle is the whole point of running it yourself.

Simple three-step diagram: an audio waveform flows into a neural-network model block, which produces a text document

How it stacks up, with the actual numbers

Speech recognition is measured in word error rate (WER) — the percentage of words the model gets wrong. Lower is better. The figure below uses the Open ASR Leaderboard averages as of October 21, 2026, as charted in IBM's own model card:

Bar chart of Open ASR Leaderboard average English word error rate: Qwen3-ASR-1.7B 5.40 percent, Granite Speech 5.0 2B 5.68 percent, Cohere Transcribe 03-2026 5.77 percent, Canary-Qwen 2.5B 5.94 percent

Granite Speech 5.0 2B lands at 5.68% average English WER — #2 among open-weight models, just behind Qwen3-ASR-1.7B at 5.40%. A couple of honest caveats worth knowing:

  • Averages hide the easy/hard split. The model scores 0.99% on clean LibriSpeech audio but 13.79% on the leaderboard's private conversational set. Studio-clean recordings are nearly perfect; real, messy conversations are much harder. Plan accordingly.
  • Noisy audio is a different leaderboard. On the FFASR noisy-speech board, Granite ranks #4 of 42 at 14.80% average WER; Qwen3-ASR-1.7B leads at 13.41%. If your recordings have background noise, Qwen3-ASR-1.7B currently measures better.
  • Language coverage is narrow. Six ASR languages, and Japanese is the weakest at 10.74% CommonVoice WER (Portuguese is the strongest at 2.60%). Qwen3-ASR-1.7B covers 30 languages plus 22 Chinese dialects.
  • Translation is one-way-ish. Speech-to-English translation reaches 53.6 BLEU for Portuguese and 40.1 for German on CoVoST — but English-to-Mandarin was not evaluated.

One note on sources: IBM's blog quotes 5.06% WER, but that figure is from October 15 and averages only public short-form sets. The 5.68% number in the chart above averages 9 public plus 2 private sets — it is the more complete figure, and it is the one to quote.

The edge options in the same release

IBM also updated two companion models worth knowing about:

  • Granite Speech 5.0 Turbo CTC — a 470M-parameter edge model that now transcribes 7 languages with native punctuation and capitalization. IBM compiled a single Conformer block and reused its graph across the stack, lifting throughput from about 12,600 to 19,870 RTFx (that's real-time factor — how many times faster than real time it processes audio). If you need on-device transcription in a resource-constrained app, this is the smaller sibling.
  • Granite Speech 4.1 2B Plus — a drop-in weight update focused on diarization (who-spoke-when): average timestamp shift falls from 210.86 to 98.54, and long-form WER drops from 13.53 to 10.22.

Should you run it?

If you want private, offline transcription on a laptop, yes — this is now one of the best open options. The Apache 2.0 license removes the friction, the Q8_0 GGUF build is about 2 GB, and llama.cpp plus mlx-audio support means it works across Mac, CPU, and GPU setups on day one.

Keep your expectations calibrated: feed it clean, reasonably short audio, name the language, and load a keyword list when jargon is involved. It is not yet the answer for multi-hour recordings in one pass, heavy background noise, or languages outside its six. But for the most common personal use case — a recorded conversation you want as text, without sending it anywhere — it is ready today.

Sources