flashvad

The Problem

Real‑time voice‑activity detection (VAD) for low‑latency calls needs a tiny, causal model that can run on CPUs, GPUs, browsers, and even macOS‑native Accelerate without pulling in heavyweight frameworks. Existing solutions are either too large, block on batch inference, or require complex deployment pipelines, making it hard to embed VAD in telephony or AI‑agent stacks.

What This Does

FlashVAD delivers a 46 k‑parameter streaming model that emits a speech‑probability every 10 ms. The repository contains four self‑contained projects:

The library can be installed from PyPI (pip install flashvad) or npm (npm install flashvad onnxruntime-web). It ships a bundled ONNX graph (models/flashvad‑v0.1/flashvad‑stream.onnx) and a pre‑compiled macOS native binary. Integration adapters for LiveKit Agents, Pipecat, and G.711 telephony are provided.

How It Is Wired

Execution starts in src/flashvad/__main__.py, which parses CLI arguments and dispatches to src/flashvad/cli.py. The CLI builds a OnnxStreamingVadModel via src/flashvad/runtime.py::_require_active_provider and OnnxStreamingVadModel.load_bundled. The model creates a per‑call stream (new_stream) whose push method runs the ONNX session (or the native macOS runtime via src/flashvad/native_runtime.py) and returns a probability array.

The import graph shows 88 internal modules and 78 edges, with no cycles. The most‑connected hub is src/flashvad/config.py (imported by 14 other modules). Changes here have the widest blast radius. src/flashvad/detector.py imports only one other module and is therefore a low‑risk leaf. The native path (native/macos/flashvad_native.c) is called only from src/flashvad/native_runtime.py, keeping the C‑side impact isolated.

Key call chain for a typical inference:

__main__ → cli.main → runtime.OnnxStreamingVadModel.load_bundled
    → runtime._provider_name / _resolved_shape
    → native_runtime.__init__ (if macOS) or ONNX Session init
    → stream.push → detector.forward (Python) → ONNX runtime exec / native C call

Modules with high instability (e.g., src/flashvad/evaluation and src/flashvad/train) import many others but are not in the hot path for inference, limiting their impact on production callers.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/flashvad
cd flashvad

# Install the Python package with all extras needed for data prep and training
uv sync --all-extras          # uv reads pyproject.toml
uv run pytest                 # run the test suite

CLI usage (see src/flashvad/cli.py):

uv run flashvad benchmark \
    --checkpoint models/flashvad-v0.1/flashvad-v0.1.pt

uv run flashvad benchmark-onnx \
    --model models/flashvad-v0.1/flashvad-stream.onnx

Programmatic usage (from src/flashvad/runtime.py):

from flashvad.runtime import OnnxStreamingVadModel

model = OnnxStreamingVadModel.load_bundled(threads=1)
stream = model.new_stream()
prob, events = stream.push(audio_float32_16khz)   # 10 ms slices
stream.reset()

For browser integration, import the bundled ES module packages/web/src/vad.mjs and feed PCM‑16kHz audio to detect.

Real‑World Use

A telephony gateway can replace its current VAD by:

from flashvad.telephony import TelephonyVadStream

vad = TelephonyVadStream()          # decodes G.711 8 kHz → 16 kHz
for chunk in incoming_pcm8k_chunks:
    prob, _ = vad.push(chunk)       # called per 10 ms packet
    if prob > 0.5:
        forward_to_asr(chunk)

The per‑call stream holds minimal state, enabling thousands of concurrent calls on a single CPU core with sub‑millisecond latency, as demonstrated in the benchmark JSON files under benchmarks/.

Code Health & Issues

• High – Pin GitHub Actions to commit SHA (.github/workflows/ci.yml).
• Medium – Add Dependabot/Renovate (.github/dependabot.yml).
• Medium – Set persist-credentials: false on checkout step.
• Low – Define job timeouts (timeout-minutes) in CI workflow.
• Low – Add repository convention files (.editorconfig, .gitattributes, formatter config).

Measured static findings (19 total):

• Medium – Cognitive load: Deep nesting (max depth 6) in src/flashvad/detector.py, src/flashvad/manifest.py, report-site/src/pages/index.astro.
• Medium – Resilience: Broad except: blocks in src/flashvad/native_runtime.py, scripts/prepare_ami_vad.py, scripts/prepare_fleurs_vad.py.
• Medium – Clarity: src/flashvad/config.py is a hub (imported by 14 modules).
• High – Clarity: Duplicated 6‑line blocks across native C headers (flashvad_native.c/h, flashvad_weights.h, src/flashvad/native_export.py).
• Medium – Resource safety: Files opened without context manager in scripts/convert_scv_manifest.py and its test.
• Medium – Cognitive load: Oversized UI files (report-site/src/pages/index.astro, VadPlayground.tsx).

All findings include concrete fix suggestions in the analysis output.

The Bottom Line

FlashVAD offers a lightweight, multi‑runtime VAD suitable for real‑time voice pipelines, with clear Python and JavaScript entry points and solid test coverage. The codebase is generally healthy but suffers from duplicated native helpers, deep nesting, and a few CI hygiene gaps that should be addressed before production hardening. It is best suited for engineers needing an embeddable VAD with optional macOS native acceleration or browser ONNX support.