The Problem

Transcribing audio with modern LLM‑based speech models typically requires a Python runtime, heavy GPU libraries, or a proprietary inference server. Teams that need a lightweight, self‑contained binary for edge or embedded use must either re‑implement the pipeline or depend on large toolchains, which adds latency, licensing baggage, and deployment complexity.

What This Does

voxtral.c delivers a pure‑C inference engine for Mistral’s Voxtral‑Realtime‑4B model. The repository bundles everything needed to load the model weights (*.safetensors), run the chunked audio encoder, and stream the decoder output as tokens.

  • Core logic lives in voxtral.c, voxtral_encoder.c, voxtral_decoder.c, and voxtral_kernels.c.
  • Audio I/O and resampling are handled by voxtral_audio.c/.h.
  • Tokenization and weight loading are in voxtral_tokenizer.c and voxtral_safetensors.c.
  • The public streaming API is defined by voxtral.h (type vox_stream_t).

A minimal Python reference (python_simple_implementation.py) mirrors the same flow for inspection, but the production path is the compiled binary built from main.c.

How It Is Wired

Execution starts at main.c → int main(int argc, char **argv). The function parses CLI flags (-d, -i, --stdin, --silent, --debug) and then:

  1. Calls vox_load_model() (implemented in voxtral.c) which: Opens the model directory, maps BF16 weights via load_safetensors() (voxtral_safetensors.c). Initializes the KV cache and decoder state structures.
  2. Sets up audio input: If --stdin → reads raw PCM from stdin. Otherwise → vox_load_wav() in voxtral_audio.c reads the WAV file, resamples to 16 kHz, and splits it into overlapping chunks.
  3. For each audio chunk: vox_encode_chunk() (voxtral_encoder.c) produces encoder embeddings. vox_decode_step() (voxtral_decoder.c) runs the attention kernels (voxtral_kernels.c) and updates the KV cache. * Tokens are emitted via vox_tokenizer_decode() (voxtral_tokenizer.c) and printed to stdout.
  4. When the KV cache exceeds 8192 positions, vox_cache_compact() in voxtral_decoder.c evicts the oldest entries, keeping memory bounded.

All heavy numeric work is delegated to either the Apple‑MPS path (make mps) or the OpenBLAS path (make blas). The call graph is essentially a linear pipeline: main → model load → audio load → encode → decode → tokenize → output. The biggest blast radius resides in voxtral.c (≈ 773 lines, deep nesting) and the duplicated control blocks across main.c, voxtral.c, voxtral_decoder.c, and voxtral_encoder.c.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/voxtral.c
cd voxtral.c

# Build for Apple Silicon (fastest) or generic BLAS
make mps          # or: make blas

# Fetch the 8.9 GB model files
./download_model.sh

# Transcribe a WAV file
./voxtral -d voxtral-model -i samples/test_speech.wav

# Stream from ffmpeg (any format)
ffmpeg -i audio.mp3 -f s16le -ar 16000 -ac 1 - 2>/dev/null | \
    ./voxtral -d voxtral-model --stdin

The Python helper can be run with the same model directory for inspection:

pip install torch safetensors soundfile soxr
python python_simple_implementation.py voxtral-model samples/test_speech.wav

No additional libraries, environment variables, or Docker images are required.

Real‑World Use

A SaaS edge node that receives live microphone streams can embed the compiled voxtral binary, pipe the PCM data directly from the capture process, and forward token strings to a downstream language‑understanding service. Because the KV cache rolls over automatically, the node can run indefinitely without growing memory.

Code Health & Issues

  • High – Add a test suite – No test files are present; regression detection is absent.
  • High – Add CI workflow – No .github/ or other CI configuration; builds are not automatically verified.
  • High – Deep nesting – Files inspect_weights.c, voxtral.c, voxtral_decoder.c each reach 9‑level indentation, making control flow hard to follow.
  • High – Duplicated code blocks – Repeated 6‑line snippets appear across main.c, voxtral.c, voxtral_decoder.c, voxtral_encoder.c.
  • Medium – Broad exception handling – python_simple_implementation.py uses bare except clauses that swallow errors.
  • Medium – Oversized file – voxtral.c contains 773 lines, increasing change impact.
  • Medium – High branching density – voxtral_safetensors.c and voxtral_tokenizer.c have 127 branch points over 374 lines, suggesting refactor into strategy tables.

Repository hygiene: tests – yes (2 test files found, but not exercising core code), CI – no, Dockerfile – no, licence – yes, lockfile – no, no committed secrets.

The Bottom Line

voxtral.c provides a functional, zero‑dependency C inference engine for Voxtral‑Realtime‑4B, suitable for low‑overhead deployments. The code works but suffers from deep nesting, duplicated logic, and a lack of automated testing/CI, which raises maintenance risk for production use. Teams comfortable with C refactoring can extract helpers and add a test harness to make the project robust for long‑running or large‑scale workloads.