The Problem

Users of Claude can query text, code, and web pages, but they cannot feed video content directly. Extracting captions, key‑frames and audio transcripts manually is error‑prone and time‑consuming, especially when the source lacks subtitles. The repository provides a self‑contained pipeline that turns any video (URL or local file) into the image‑and‑text payload Claude can ingest.

What This Does

The watch skill lives under skills/watch/scripts/.

  • watch.py defines the CLI entry point main.
  • setup.py implements the heavy‑lifting: it parses arguments, orchestrates download, frame extraction, caption fetching, and optional Whisper transcription.
  • Supporting modules – download.py, frames.py, transcribe.py, whisper.py, config.py – each encapsulate a single responsibility (e.g., download() pulls the video or captions via yt‑dlp; extract_audio() in whisper.py creates the audio clip; parse_vtt() in transcribe.py parses caption files).

The skill can be installed into Claude via the marketplace (/plugin marketplace add bradautomates/claude-video) or locally with the provided npm‑style wrapper (npx skills add bradautomates/claude-video -g).

How It Is Wired

Execution starts at skills/watch/scripts/setup.py:main (line 353). From there:

  1. Argument parsing → calls get_config() (config.py) to read environment variables (_read_env_key, _have_api_key).
  2. Download phase → download() (download.py) decides between URL and local path, then either: Calls fetch_captions() to obtain VTT subtitles via yt‑dlp, or Calls download_url() to fetch the video file. Both functions invoke external commands (yt‑dlp, ffmpeg).
  3. Frame extraction → extract_scene_or_uniform() and extract_keyframes() (frames.py) use _scale_filter, _clamp_fps, parse_time, format_time, get_metadata. This module alone holds 644 lines and is invoked by main through several helper calls, giving it the widest blast radius (20 distinct functions, external command execution, file I/O).
  4. Transcription → parse_vtt() (transcribe.py) parses subtitles; if subtitles are missing, extract_audio() (whisper.py) runs ffmpeg to produce a mono MP3, then _post_whisper() contacts the Whisper API. Broad except blocks in whisper.py swallow errors, reducing resilience.
  5. Packaging → format_transcript() and format_time() produce the final Claude‑compatible payload.

The internal call graph contains 104 resolved edges; the most‑used functions (_scale_filter, dedupe_perceptual, parse_time, extract) are each called from three‑plus locations, meaning changes to them ripple widely. No circular imports exist, keeping the module graph flat.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/claude-video
cd claude-video

# Install runtime dependencies (macOS example – Linux/Windows print commands in README)
brew install yt-dlp ffmpeg

# Install Python requirements (if a requirements.txt existed; otherwise rely on system binaries)
# pip install -r requirements.txt   # <-- not present, so skip

# Build the skill (creates the Claude‑compatible package)
bash skills/watch/scripts/build-skill.sh

# Run the watch command locally (mirrors the skill entry point)
python skills/watch/scripts/setup.py https://youtu.be/dQw4w9WgXcQ "what happens at 30 seconds?"

Environment: set WHISPER_API_KEY if you expect Whisper fallback; the key is read by _read_env_key in config.py.

Real‑World Use

A product team receives a bug‑repro video (bug.mov). Running the above command returns a JSON payload with timestamps, key‑frame thumbnails, and a transcript. The team feeds this payload to Claude via the /watch skill, receiving a concise explanation of the UI state at the failure point, without manually scrubbing the video.

Code Health & Issues

  • High – Pin GitHub Actions – .github/workflows/release.yml uses softprops/action-gh-release@v2. Replace @v2 with a fixed SHA to avoid supply‑chain risk.
  • High – CI never runs tests – The workflow defines no test step despite 10 test files. Add a step that runs pytest.
  • Low – No job timeout – release.yml lacks timeout-minutes; set a reasonable limit (e.g., timeout-minutes: 30).
  • Medium – Duplicated code – frames.py and whisper.py share a 6‑line block; extract to a common helper.
  • Medium – Broad exception handling – whisper.py catches Exception indiscriminately; replace with specific exception types and proper logging.
  • Medium – High branching density – hooks/scripts/check-setup.sh contains 20 branches in 43 lines; refactor into clearer functions or a case table.
  • Medium – Oversized file – frames.py at 644 lines is a maintenance hotspot; split by logical responsibilities (e.g., scaling, FPS clamping, metadata extraction).

No missing license, secrets, or test suite beyond the noted CI gap.

The Bottom Line

The repo delivers a functional end‑to‑end pipeline that lets Claude “watch” videos with minimal user effort. It is well‑tested and CI‑ready, but the code base suffers from a few maintainability issues (large monolithic modules, duplicated helpers, and overly broad error handling) and CI misconfiguration. Engineers comfortable with Python, ffmpeg, and yt‑dlp can adopt it quickly; teams needing tighter CI or a more modular codebase should allocate effort to refactor the highlighted hotspots.