insanely fast whisper

Here's a concise, professional technical briefing for insanely-fast-whisper, written to consultant standards.

The Problem

This repository provides a CLI for on-device Whisper transcription using 🤗 Transformers, Optimum, and flash-attn. It targets users who need fast, local transcription without relying on cloud APIs. The pain point it addresses is the latency gap between cloud Whisper services and local execution—particularly for long-form audio—while maintaining model quality. The project differentiates itself through Flash Attention 2 and batching optimizations, claiming 150 minutes of audio in under 2 seconds on A100 hardware.

What This Does

The codebase is small: 8 Python files, 3 notebooks, and a CLI entry point. Execution flows through src/insanely_fast_whisper/cli.py:main, which orchestrates the transcription pipeline. The core work is distributed across the utils/ subpackage: diarize.py handles audio preprocessing, diarization, and an outbound network call via requests.get (no timeout configured); result.py defines JsonTranscriptionResult and build_result; diarization_pipeline.py exposes a single diarize function; and convert_output.py defines formatters (SrtFormatter, VttFormatter, TxtFormatter) and chunk-processing utilities (format_chunk, format_seconds, preamble). The entry point main reaches 9 functions total, with the longest call chain being main -> diarize -> preprocess_inputs [network via requests.get].

How It Is Wired

Execution starts at src/insanely_fast_whisper/cli.py:111 (main). From there, the call graph resolves as follows:

• main -> convert — handles model loading and the primary inference path.
• main -> convert -> preamble / format_chunk / format_seconds — output formatting.
• main -> diarize — triggers the diarization pipeline.
• diarize -> preprocess_inputs — makes an outbound network call to fetch diarization data via requests.get (no timeout).
• diarize -> diarize_audio / post_process_segments_and_transcripts — internal audio processing.
• main -> build_result / convert — final result assembly.

The outbound network call in src/insanely_fast_whisper/utils/diarize.py is the only edge leaving the process. No CI/CD pipeline is configured, and there is no dependency lockfile, meaning builds are not reproducible across environments.

What Each File Is Responsible For

How To Use It

Setup The project uses pyproject.toml for dependencies and tooling. No lockfile is committed, so installs should be pinned explicitly. Install via pipx:

pipx install insanely-fast-whisper==0.0.15 --force

Or with pip if you need to override Python version constraints:

pip install insanely-fast-whisper --ignore-requires-python

Configuration No environment variables or config files are required for basic transcription. The only external dependency is network access for the diarization endpoint, which is invoked from src/insanely_fast_whisper/utils/diarize.py via requests.get.

Running it The CLI entry point is insanely-fast-whisper. From any path:

insanely-fast-whisper --file-name <filename or URL>

On macOS, add --device-id mps. To enable Flash Attention 2:

insanely-fast-whisper --file-name <filename or URL> --flash True

To use distil-whisper:

insanely-fast-whisper --model-name distil-whisper/large-v2 --file-name <filename or URL>

Real-World Use

A developer transcribing a 30-minute meeting recording on a workstation with an NVIDIA GPU can run:

insanely-fast-whisper --file-name meeting.wav --flash True

This loads the Whisper Large v3 model with Flash Attention 2 and batching, producing a timed transcript in under 2 minutes. The output can be formatted as SRT or VTT via the CLI flags. If the meeting requires speaker diarization, the built-in pipeline will make an outbound request to a diarization service—ensure network access is available and consider adding a timeout, as the request has none configured.

Code Health & Issues

• [MEDIUM/cognitive_load] src/insanely_fast_whisper/utils/diarize.py — Max indentation depth 6; control flow is hard to follow. Fix: flatten with early returns/guard clauses; extract inner blocks.
• [HIGH] No CI/CD pipeline — 8 source files, no automated build/test gate. Every change merges without having been built. Fix: add a workflow running the project build and test command on push and pull_request.
• [MEDIUM] No dependency lockfile — pyproject.toml declares dependencies without a lockfile, meaning builds are not reproducible. Fix: commit pdm.lock or add a requirements.txt pinned at the desired version.
• [MEDIUM] Outbound request has no timeout — src/insanely_fast_whisper/utils/diarize.py makes an unbounded requests.get call. Fix: pass timeout= on every call, or use a session with one configured.

The Bottom Line

The repository delivers on its promise: a lightweight, opinionated CLI that transcribes hours of audio in minutes using Flash Attention 2 and batching. The codebase is focused and the performance claims are credible given the optimization stack (Transformers + Optimum + flash-attn). However, the absence of CI, a lockfile, and a timeout on the diarization network call are production-grade gaps. Use this for local, GPU-accelerated transcription workflows where network access is available and the diarization endpoint is acceptable. Teams requiring reproducible builds, automated testing, or hardened network calls should add the recommended CI and timeout fixes before depending on this in a service context.

Word count: 512