The Problem

Voice cloning TTS systems typically cover a handful of languages, and multilingual models often degrade in speaker similarity or require long reference audio. OmniVoice targets zero-shot voice cloning across 600+ languages with fast inference, so a single model can clone a voice and speak in languages the original speaker never used.

What This Does

OmniVoice is a diffusion language-model-style TTS that generates speech from text plus a short reference clip. It supports voice cloning, attribute-based voice design (gender, age, pitch, accent, whisper), fine-grained control like [laughter], and pronunciation correction via pinyin or phonemes. The README claims a real-time factor as low as 0.025.

The package is pure Python (49 files) under omnivoice/, split into cli/, models/, training/, data/, eval/, and utils/. Training and inference configs live in examples/config/*.json, with orchestration scripts in examples/run_*.sh.

How It Is Wired

Entry points are the CLI modules: omnivoice/cli/infer.py (single inference), infer_batch.py (batch), train.py (training), merge_lora.py (LoRA merging), and demo.py. train.py routes through omnivoice/training/trainer.py, which pulls configuration from training/config.py, model construction from training/builder.py, and checkpointing from training/checkpoint.py. The data pipeline (data/dataset.py, processor.py, batching.py, collator.py) feeds the model in models/omnivoice.py, with a FlashInfer-accelerated variant in omnivoice_flashinfer.py.

A full training run reads a JSON config from examples/config/, processes audio into tokens via utils/audio.py and scripts/extract_audio_tokens.py, and writes checkpoints to disk. Inference downloads weights from Hugging Face (k2-fsa/OmniVoice) and returns a waveform. The widest blast radius sits in training/trainer.py and models/omnivoice.py — everything else is peripheral to those two. An exact internal call graph was not mapped for this briefing, so function-level tracing beyond these entry points is not available.

How To Use It

Install PyTorch for your platform, then install the package. The README shows pip install omnivoi... (truncated in the excerpt) and the repo ships uv.lock, so uv sync is the reproducible path. Configuration lives in examples/config/*.json — pick train_config_emilia.json, train_config_finetune.json, or train_config_finetune_lora.json. Run training via examples/run_emilia.sh, examples/run_finetune.sh, or examples/run_finetune_lora.sh; evaluate via examples/run_eval.sh. The exact CLI invocation for inference is not visible in the README excerpt.

Real-World Use

A media localization pipeline: ingest a 10-second reference recording of a narrator, clone the voice, and synthesize the same script in 40 languages using one model. The docs/languages.md file lists coverage; docs/voice-design.md covers attribute control. The eval suite (omnivoice/eval/) provides WER, speaker similarity, and MOS checks to gate releases.

Code Health & Issues

The single reported static-analysis finding — dependencies without a lockfile — is contradicted by the presence of uv.lock at the repo root. That finding should be discarded; uv.lock plus pyproject.toml gives reproducible builds. Other observations from the structure:

  • Low — thin test coverage: one test file (tests/test_lora.py) for a 49-file Python package; training and inference paths are untested.
  • Low — no lint/type-check CI: .github/workflows/ contains only publish.yml and stale.yml; .pre-commit-config.yaml exists but is not enforced in CI.
  • Docs are strong: 16 doc files covering training, data prep, evaluation, and LoRA fine-tuning.

The Bottom Line

A serious, well-documented TTS codebase with real breadth (600+ languages) and credible performance claims. The fork has zero stars and minimal test coverage, so treat it as an upstream snapshot for experimentation, not production infrastructure. Use it if you need multilingual voice cloning and can invest in your own validation and tests.