The Problem

Teams that need high‑quality voice cloning, dubbing, or transcription often rely on cloud APIs (e.g., ElevenLabs) that require API keys, incur per‑call cost, and expose raw audio to third‑party services. For privacy‑sensitive or offline workflows this creates compliance risk and latency.

What This Does

VoiceStudio delivers a fully‑local stack that combines 16 TTS and 11 ASR engines under a single desktop and REST interface. The backend lives in backend/ and is built on FastAPI; the UI is a Tauri‑based Electron‑style app in frontend/. Core engine selection is driven by backend/config/models.yaml, which maps logical engine names to concrete Python modules such as backend/engines/dots_tts/main.py or backend/engines/omnivoice_gguf/backend.py.

The project’s entry points (backend/engines/_asr_sidecar/main.py, backend/engines/_echo/main.py, etc.) expose a uniform run() function that each engine implements. The API routers in backend/api/routers/ (e.g., tts_stream.py, dub_generate.py) invoke these engines through backend/core/job_queue.py and backend/core/tasks.py, handling async job scheduling and result streaming.

How It Is Wired

  1. Startup – backend/main.py constructs a FastAPI app, registers all routers (backend/api/routers/*.py), and launches the server via uvicorn.run(app, ...). It also loads configuration from backend/config/models.yaml via backend/core/config.py.
  2. Request Flow – An HTTP request to /api/tts_stream (router defined in backend/api/routers/tts_stream.py) calls backend/core/tasks.py.submit_job(). This places a job record in the SQLite DB managed by backend/core/db.py and pushes a coroutine onto the job_queue (implemented in backend/core/job_queue.py).
  3. Engine Dispatch – job_queue resolves the requested engine name to a module path (e.g., backend/engines/dots_tts/main.py) and imports its run(request) coroutine. The engine’s run function streams audio bytes back through a FastAPI StreamingResponse.
  4. Side‑car Processes – Some engines (e.g., _asr_sidecar) spawn a separate subprocess via backend/core/contained_subprocess.py to isolate heavy CUDA work. The subprocess communicates over a local socket; termination is coordinated by backend/core/run_sentinel.py.
  5. Frontend Integration – The Tauri UI (frontend/src-tauri/main.rs and the bundled React app) calls the local REST API at http://127.0.0.1:<port> for all operations. UI state is persisted in the frontend/src-tauri/Cargo.toml‑defined storage directory, while voice assets reside under backend/assets/samples/.

The most “blast‑radius” component is backend/core/job_queue.py; it mediates every engine execution, so changes here affect all TTS/ASR pathways. The router layer (backend/api/routers/) is a thin façade, keeping the core logic decoupled.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/VoiceStudio.git
cd VoiceStudio

# Build the Docker image (optional, for GPU workers)
docker build -f deploy/Dockerfile -t voicestudio:latest .

# Start the backend locally (Python 3.11+ recommended)
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt   # generated from pyproject.toml
uvicorn backend.main:app --host 127.0.0.1 --port 8000

Configuration – Edit backend/config/models.yaml to set the default engine (e.g., "tts": "dots_tts"). No external API keys are required for the core workflow; optional remote workers are configured in the same file under workers:.

Run the UI – From frontend/:

pnpm install
pnpm tauri dev   # launches the desktop app that talks to the backend above

If Docker is preferred, the compose file deploy/docker-compose.yml starts both backend and a GPU‑enabled worker container; the UI can still be launched locally and will auto‑detect the Docker endpoint.

Real‑World Use

A media house can install VoiceStudio on an isolated workstation, load a proprietary voice model into backend/engines/omnivoice_gguf/, then run a batch job:

import requests, json

payload = {"text": "Welcome to the quarterly report.", "engine": "omnivoice_gguf"}
r = requests.post("http://127.0.0.1:8000/api/dub_generate", json=payload, stream=True)
with open("output.wav", "wb") as f:
    for chunk in r.iter_content(chunk_size=8192):
        f.write(chunk)

The job is queued, processed on the local GPU, and streamed back without leaving the premises.

Code Health & Issues

  • Low – No test coverage for backend/api/routers/mcp_bindings.py – present in repo but no corresponding test file.
  • Low – Missing explicit lockfile for frontend – frontend/package.json exists, but no pnpm-lock.yaml or yarn.lock is committed, risking reproducibility.
  • Low – License file present (AGPL‑3.0) but engine‑specific model licenses are not enumerated – could cause downstream compliance ambiguity.

Static analysis found no structural red flags: CI workflows (.github/workflows/*.yml) run linting, type checks, and Docker builds; tests are present (tests/ directory) and CI references them.

The Bottom Line

VoiceStudio provides a mature, self‑hosted alternative to cloud TTS/ASR services, with a clear FastAPI‑driven backend and a Tauri UI. The architecture is modular, but the central job queue is the key integration point; changes there ripple across all engine paths. It is suitable for teams needing privacy‑first audio processing and who can manage a Python‑centric stack with optional GPU resources.