local voice ai

Here's my concise, professional technical briefing for the local-voice-ai repository, written to consultant-grade standards.

The Problem

This repository provides a locally-hosted, single-container voice AI agent framework that integrates STT, LLM, and TTS services. It is architected for privacy and offline operation, but the codebase contains several production-readiness gaps that a consultant would need to address before deployment.

What This Does

The system runs as a supervised Python process (local_voice_ai serve) that manages child processes: a LiveKit Go binary for WebRTC signaling, a Llama.cpp LLM subprocess, a Nemotron or Whisper STT uvicorn server, and a Kokoro TTS uvicorn server. Communication between services occurs over 127.0.0.1 HTTP. The FastAPI supervisor exposes POST /api/connection-details for token minting and serves the Next.js frontend statically. The frontend lives in frontend/ (69 files, React/Next.js), while the Python backend is in local_voice_ai/ (14 files). Environment configuration is split between .env and frontend/.env.local — both are committed to the repository, which is a significant security concern. Docker orchestration is provided via docker-compose.yml and docker-compose.gpu.yml, with GPU support requiring the NVIDIA container toolkit.

How It Is Wired

Execution starts at local_voice_ai/__main__.py, which imports and initializes the supervisor. The supervisor (local_voice_ai/supervisor.py) orchestrates child service startup, token minting via local_voice_ai/api.py, and agent lifecycle. The internal call graph has 58 modules with 18 import edges and no circular dependencies. The most connected modules are local_voice_ai/config (5 importers, 0 exports, instability 0) and local_voice_ai/__main__ (1 importer, 3 exports, instability 0.75). The local_voice_ai/api module is imported by 2 consumers and exports 1 function, while local_voice_ai/agent receives imports from 3 sources. The local_voice_ai/wakeword module (2 importers, 0 exports, instability 0) and local_voice_ai/supervisor (3 importers, 0 exports, instability 0) are also key hubs. The tests/test_main.py and tests/test_api.py modules show instability of 1, indicating they are importers but not importers of further modules — likely terminal test entry points.

Outside the process, the system touches the filesystem via model weight caching in a local-voice-ai-models volume, writes logs to stdout/stderr, and exposes ports 8080 (web), 7880, 7881, and 7882/udp (LiveKit WebRTC). The Next.js frontend at frontend/components/livekit/ handles WebRTC rendering, agent control, and chat UI.

How To Use It

Setup: Install dependencies with pnpm install in frontend/ and uv sync or pip install -e . in local_voice_ai/. The repo uses pyproject.toml and uv.lock, so uv is the recommended Python installer.

Configuration: Required environment variables are defined in .env and frontend/.env.local — both are currently committed to the repo and must be rotated and added to .gitignore. Key variables include LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET; LLAMA_BASE_URL, LLAMA_MODEL, LLAMA_API_KEY, LLAMA_HF_REPO; STT_BASE_URL, STT_MODEL, STT_API_KEY; and TTS_BASE_URL, TTS_API_KEY. An .env.example exists in frontend/ with placeholder values.

Running it: To run via Docker: docker compose up --build. For GPU: docker compose -f docker-compose.yml -f docker-compose.gpu.yml up --build. To run bare-metal on Apple Silicon: the prebuilt image is arm64 CPU-only; for Metal inference, run locally without Docker where llama-server picks up Metal automatically. The entry point is python -m local_voice_ai serve.

Real-World Use

A privacy-conscious organization can deploy this as a single-container voice agent that processes all audio locally. STT uses either Nemotron (OpenAI-compatible uvicorn) or Whisper (faster-whisper). The LLM defaults to a 4-bit quantized Gemma model (~2.6 GB) hosted via Llama.cpp, swapable via LLAMA_HF_REPO. TTS uses Kokoro, also OpenAI-compatible. All services bind to 127.0.0.1, and the LiveKit server can be pointed at LiveKit Cloud via LIVEKIT_URL, making this suitable for internal or edge deployments where data must not leave the host.

Code Health & Issues

Static analysis of 58 files identified 10 medium-severity findings across 4 kinds:

• [MEDIUM/resource_safety] local_voice_ai/wakeword.py — File opened without context manager; open(...) not wrapped in with, handle may leak on error. Fix: use with open(...) as f:.
• [MEDIUM/resilience] Broad exception handling in local_voice_ai/api.py, local_voice_ai/agent.py, local_voice_ai/services/kokoro/server.py — Bare except swallows errors indiscriminately. Fix: catch specific exceptions; re-raise or log the rest.
• [MEDIUM/cognitive_load] Deep nesting in local_voice_ai/__main__.py, frontend/components/app/tile-layout.tsx, tests/test_main.py — Max indentation depth 6 makes control flow hard to follow. Fix: flatten with early returns/guard clauses; extract inner blocks.
• [MEDIUM/clarity] Duplicated code blocks across frontend/components/livekit/button.tsx, frontend/components/livekit/toggle.tsx, local_voice_ai/services/nemotron/server.py, local_voice_ai/services/whisper/server.py — 10 repeated 6-line blocks across 4 files. Fix: extract shared helpers; DRY the repeated logic.

Beyond the measured findings, the SDLC posture reveals several structural concerns: committed secrets (.env, frontend/.env.local tracked in git), no Dependabot or Renovate configuration, a Dockerfile using mutable node:20-slim base image without a digest, and no dependency vulnerability scanning in CI. The .env file is committed and contains live credentials — this is the highest-risk item and must be remediated before any shared deployment.

The Bottom Line

This is a functional, well-architected local voice AI framework that delivers on its promise of a single-container, offline-capable agent. The code is readable and the service wiring is clear, but the repository ships with committed secrets, no dependency governance, and basic code-quality gaps that a consultant would resolve before production use. It is suitable for teams that need a local-first voice AI pipeline and are prepared to handle the outlined hygiene and security remediation.

Word count: 528