The Problem
Real-time speech-to-speech AI typically requires either a pipeline of separate ASR, LLM, and TTS models (adding latency and complexity) or large models that demand GPU resources unavailable on edge devices. Liquid Audio addresses this with a single end-to-end audio foundation model (LFM2-Audio-1.5B) built on a lightweight 1.2B backbone, designed for low-latency conversational interaction on constrained hardware.
What This Does
The repo implements LFM2-Audio, a speech-to-speech model with two generation modes. generateinterleaved (in src/liquidaudio/model/lfm2audio.py) outputs text and audio tokens in a fixed pattern, minimizing time-to-first-audio for real-time conversation. generatesequential lets the model switch modalities via special tokens, better suited for ASR or TTS tasks. The model produces audio via 8 Mimi codebooks; a processor (src/liquidaudio/processor.py) handles conversion between waveforms, log-mel features, and tokens.
The codebase is organized into a moshi/ subpackage (adapted from the Moshi architecture) containing streaming modules, quantization (VQ), conditioning, and a server (src/liquidaudio/moshi/server.py). A ChatState helper manages multi-turn conversation formatting. The README documents a Gradio demo (liquid-audio-demo) and includes runnable examples for inference (runinference.py) and TTS (runtts.py).
How To Use It
Setup: Install via pip install liquid-audio. Optional extras: pip install "liquid-audio [demo]" for the demo interface, and pip install flash-attn --no-build-isolation for flash attention (falls back to torch SDPA). A uv.lock file exists, so uv sync is also viable. Configuration: No environment variables or config files required. Model weights are downloaded from Hugging Face at runtime. Running it: Launch the Gradio demo with liquid-audio-demo (available at http://localhost:7860). For programmatic use, instantiate LFM2AudioModel and LFM2AudioProcessor, then call generateinterleaved or generatesequential as generators yielding tensors.
pip install liquid-audio pip install "liquid-audio [demo]" liquid-audio-demo
Real-World Use
A customer-support voice agent on a modest GPU server: the system prompt is set to "Respond with interleaved text and audio." Audio from the caller is converted to log-mel features via the processor, passed to generateinterleaved, and the yielded audio tensors are detokenized back to a waveform for playback. The interleaved mode keeps the first audio response fast, and the lightweight backbone allows multiple concurrent sessions on a single GPU.
Code Health & Issues
Med – No lockfile for dependencies (only pyproject.toml), so builds are not fully reproducible despite uv.lock being present. Low – Only 2 test files (convtest.py, seanet_test.py) covering the modules subpackage; core model, generation, and processor paths are untested. Low – CI is minimal (a single check.yml); no evidence of linting, type-checking, or security scanning in the pipeline. Low – server.py exists but the README does not document it; unclear whether it is production-ready or a demo artifact. Low – The moshi/ subpackage appears to be a fork of Kyutai's Moshi code; license compatibility and provenance should be verified before commercial use.
The repo is reasonably organized with py.typed markers and type hints. The lack of a lockfile and thin test coverage are the main concerns; nothing appears critically broken.
The Bottom Line
Liquid Audio is a credible, well-documented implementation of a compact speech-to-speech model, with a clear API and practical demo. It is best suited for teams building real-time voice interfaces on constrained hardware who want a single-model alternative to multi-stage pipelines. The main caveats are the thin test coverage and the need to verify Moshi code provenance before production deployment.