Local AI Advisor
All posts

llama.cpp v0.8.0: your local AI server can now talk back

llama.cpp v0.8.0 adds realtime voice chat and text-to-speech to llama-server, plus day-one support for the LFM3 audio and omni models.

By Ray, Your Local AI Advisor4 min readlocal-llmannouncementsapps

If you run models on your own machine, you probably already run llama.cpp, even if you never typed its name. It is the engine underneath Ollama, LM Studio, and most other local apps. Version 0.8.0, released October 28, gives that engine a new skill: spoken conversation. Your local server can now listen and talk back in real time, instead of only trading text.

A person speaking to a desktop computer, with sound waves flowing both ways between them

What just changed

The headline is two new endpoints in llama-server, the small web server that turns a model file into an API:

  • /v1/realtime: a streaming endpoint that follows the OpenAI realtime protocol. Your voice streams in, and the model's answer streams back as speech and text. It also handles live transcription.
  • POST /tts: a text-to-speech endpoint, so apps can ask the server to read text aloud instead of wiring up a separate voice tool.

Both are built for Liquid AI's LFM3 audio models, which landed in the same release. LFM3-Audio-2B listens and talks back from one 2-billion-parameter model, and LFM3-Omni-2B adds vision to the same trick, so a spoken request can also look at an image or video. The realtime endpoint supports tool calls too, which is what turns a voice chat into a voice agent: ask out loud, and the model can call a function before it answers.

Why this matters if you run models at home

Until now, a private voice assistant on your own hardware meant gluing pieces together yourself: a speech-to-text model, a chat model, and a text-to-speech model, each with its own setup and latency. This release puts the whole loop behind one server and one protocol that apps already speak. Any app that knows the OpenAI realtime protocol can now, in principle, point at a local llama-server instead of a cloud endpoint, and your voice never leaves the machine.

The models are small enough to make that realistic. At around 2 billion parameters, the LFM3 audio and omni models fit on a laptop or a modest desktop GPU, not just a server rack. That is the same family this site covered on October 28, when Ollama brought LFM3 to Linux and Windows; llama.cpp v0.8.0 is the engine-level version of the same story, available to every app that builds on it.

Diagram showing a microphone and a speaker connected both ways through a home computer running a local AI server

What else landed in 0.8.0

The release is bigger than voice. A few items worth knowing:

  • New model support: DeepSeek V4.1 Flash, Reflection's Beam, Kolibri-1, Flint, and Meta's Muse Spark Open now run in llama.cpp, along with Liquid AI's d1 decision models and a Soprano text-to-speech model. If you have been waiting for engine support before trying one of these, the wait is over.
  • Download models from the web UI: the built-in web interface can now browse and download models straight from Hugging Face, instead of making you fetch weights by hand first.
  • Longer chats stay usable: the web UI can compact a long conversation to keep it inside the context window, and search across past conversations.
  • Faster backends: the underlying ggml library moved to v0.28.0, with performance work on CUDA, SYCL, OpenCL, and other backends. You may see faster prompt processing on your GPU after upgrading, though the release notes give no measured numbers, so treat that as "try it and measure" rather than a promise.

Honest limits

Four things to know before you rebuild your setup around this. First, the streaming-audio plumbing is labelled experimental in the release notes, and realtime voice is brand new in this server. Expect rough edges, and test on your own hardware before depending on it. Second, speech quality, latency, and language coverage for the LFM3 audio models outside Liquid AI's own testing are still unproven. Third, apps have to adopt the new endpoints before you can use them; a /v1/realtime endpoint in the server does not automatically give your favorite app a voice mode. Fourth, one small breaking change: the command-line tools now require --model or -hf, so scripts that relied on a default model will need an update.

One more caution, and it matters. The release adds a --connect flag that lets the server be reached from outside your local network. That is convenient and also exactly how local servers end up exposed to the internet. If you are not deliberately setting up remote access with authentication in front of it, leave the server on your local network. The one-minute exposure check in the earlier Ollama security post applies here just as well.

The bottom line

llama.cpp v0.8.0 turns the engine behind most local AI apps into a voice-capable server. If you run llama-server yourself, upgrade, try an LFM3 audio model, and see how a spoken conversation feels when nothing leaves your machine. If you use Ollama or LM Studio instead, watch for their next releases to pick this up. And if you are still deciding what hardware to run models on, start with /check to match a model to the machine you already own.

Sources