The Problem

Real-time speech-to-speech AI requires a model that generates audio and text tokens with low latency, typically on resource-constrained devices. Most systems either stream text or audio separately, or rely on large backbones that cannot run interactively. Liquid Audio targets this gap with a lightweight end-to-end audio foundation model that interleaves text and audio token generation.

What This Does

Liquid Audio is a speech-to-speech model built on the LFM2 backbone. It supports two generation modes: generate_interleaved (fixed interleaved text/audio token pattern, minimal time-to-first-audio) and generate_sequential (model decides modality switches via special tokens, suited for ASR/TTS). The codebase is a PyTorch implementation with a Moshi-style architecture: streaming convolution modules, a transformer backbone, residual vector quantization for audio codecs, and conditioners for text/audio inputs.

The repo includes a server (src/liquid_audio/moshi/server.py) with a client (client.py), a Gradio demo (client_gradio.py), and inference scripts (run_inference.py, run_tts.py).

How It Is Wired

Execution starts at main in src/liquid_audio/moshi/server.py:176, which reaches 109 functions. The server runs a chat loop: handle_chat (line 74) and warmup (line 62) are the other major entry points. From main, the shortest path to an external effect is main -> from_hf_repo, which reads a config file from disk. The client path run -> _queue_loop reads audio bytes from an Opus writer.

The highest fan-in modules are src/liquid_audio/moshi/utils/compile.py (imported by 7 files, imports nothing—a leaf utility), modules/conv.py and modules/streaming.py (5 importers each). The transformer module (modules/transformer.py) is the most connected hub: 4 importers, 7 dependencies, and an instability score of 0.64, meaning it both depends on and is depended upon—changes there ripple in both directions.

The core call graph routes through to (called from 37 places) and cat (28 places)—these are tensor operations every module relies on. StreamingConv1d and StreamingConvTranspose1d are instantiated 8 times each across __init__ methods, confirming the streaming-audio architecture. from_hf_repo in models/loaders.py is the file that loads model weights and calls a model for inference.

How To Use It

Setup: Install via pip as documented in the README:

pip install liquid-audio
pip install "liquid-audio [demo]" # optional, demo dependencies
pip install flash-attn --no-build-isolation  # optional, flash attention 2

Running it: Launch the Gradio demo with liquid-audio-demo. For programmatic use, the README documents LFM2AudioModel.generate_interleaved and generate_sequential, plus LFM2AudioProcessor for token/data conversion. The server entry point is src/liquid_audio/moshi/server.py.

Real-World Use

A customer-support voice agent: the model receives a user's spoken query, the processor converts the waveform to log-mel features, generate_interleaved streams back text-then-audio responses with sub-second latency, and the detokenizer (detokenizer.py) converts audio tokens back to a waveform. The ChatState helper manages multi-turn context.

Code Health & Issues

Static analysis found 29 issues: 6 high, 23 medium. The dominant problems are deep nesting (max indentation depth 7, in utils/compile.py, conditioners/base.py, modules/conv.py) and 47 duplicated 6-line code blocks across 23 files, concentrated in model/conformer/. Three files exceed 800 lines (modules/transformer.py, models/lm.py, model/conformer/encoder.py).

  • High - Third-party GitHub Actions pinned to tags, not commit SHAs - .github/workflows/check.yml uses astral-sh/setup-uv@v3
  • Med - No Dependabot/Renovate configured - dependency updates are manual
  • Med - No dependency vulnerability scan in CI
  • Med - Test coverage is 2 test files against 56 source files (ratio 0.036)

The Bottom Line

A credible, working speech-to-speech implementation with a clean streaming architecture and a real server/client split. The code quality issues are maintenance concerns, not blockers. Use it if you need a reference implementation for low-latency audio generation or want to build on the LFM2 backbone; the duplicated conformer code and thin test suite mean you should budget for refactoring before treating it as production infrastructure.