The Problem

Raw ASR output is full of disfluencies, self-corrections, and conversational noise that don't match the speaker's intended written text. Most post-processing pipelines are rule-based and brittle — they can't handle context-dependent rewrites or revise earlier output when new speech arrives.

What This Does

AgenticASR is a research codebase for an agent-based ASR refiner. It turns raw speech hypotheses into clean, written-form text while preserving speaker intent. The refiner is ASR-agnostic and works with any frontend that produces text hypotheses.

The repo has three projects: pipeline/ (data generation and training pipeline), experiments/ (inference and benchmarking), and system/ (online/offline refinement system). The pipeline generates synthetic training data from seed prompts, simulates ASR errors, and produces clean text via LLM-based generators.

How It Is Wired

Execution starts at experiments/scripts/main.py:153 (main), which routes through parse_args and run. main is called from one place and reaches 182 functions — it's the primary entry point. The run function (line 103) reaches 90 functions and handles the actual workflow.

The core flow: main → run → load_rubric (reads filesystem via path.read_text) and setup_logging (writes via os.makedirs). These are the shortest traced paths from entry to external effect — two hops.

The most connected modules form a small graph: system/chunking (3 importers, 0 deps), system/refiner (2 importers, 1 dep), and pipeline/run_pipeline (1 importer, 0 deps). No circular dependencies exist among the 45 internal modules with 7 import edges.

Key files by responsibility:

  • experiments/scripts/postprocess_asr.py — batch refiner inference; called from 10 files
  • experiments/scripts/benchmark_io.py — data loading and rubric parsing
  • pipeline/src/utils/io_utils.py — file I/O helpers (read_jsonl, write_jsonl); called from 11 files
  • pipeline/src/services/llm_client.py — LLM client wrapper; generate called from 10 places
  • pipeline/src/services/vllm_lifecycle.py — VLLM process management; runs external commands

The highest blast radius: generate (10 callers), read_jsonl (9 callers), and AsyncLLMClient (7 callers). Changes to these break the most downstream code.

How To Use It

Setup:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip install -r system/requirements.txt

Refiner training requires LLaMA Factory installed separately.

Running batch inference:

python experiments/scripts/postprocess_asr.py \
  /path/to/asr_output.jsonl \
  /path/to/refined_output.jsonl \
  --model /path/to/refiner-checkpoint

Input records must contain source_record_id and output.raw_text. The refiner checkpoint is available on Hugging Face (Andrew0425/AgenticASR-Refiner) and ModelScope (MuyuanJ/AgenticASR-Refiner).

Real-World Use

A production ASR system (e.g., Whisper or Qwen3-ASR) emits raw hypotheses as JSONL. AgenticASR refines them in batch or online mode. For streaming, system/live_asr.py handles continuous refinement with bounded active spans — useful for live captioning or meeting transcription where you want clean text without waiting for the full utterance.

Code Health & Issues

Static analysis (not opinion) found 13 medium findings across 5 kinds:

  • Medium — Deep nesting (9 instances) in pipeline/run_pipeline.py, experiments/scripts/postprocess_asr.py, pipeline/scripts/03_simulate_asr.py (max depth 6)
  • Medium — File opened without context manager in pipeline/src/services/vllm_lifecycle.py
  • Medium — Duplicated code blocks (6 repeated 6-line blocks across 8 files)
  • Medium — Broad exception handling in pipeline/src/services/vllm_lifecycle.py
  • Medium — High branching density in pipeline/src/utils/keyword_utils.py (34 branch points over 113 lines)

SDLC gaps: no test suite (45 source files, 0 tests), no CI configuration, no dependency lockfile, and a 22MB binary (MediaSup/CH_demo.mp4) committed to the repo. Missing tests and CI are high-severity issues — any change ships without verification.

The Bottom Line

The architecture is sound: clear separation between data generation, inference, and system components. The lack of tests and CI is a real risk for a research codebase that others will build on. Worth using if you need a working reference implementation for agent-based ASR refinement, but budget for a test suite before relying on it in production.