The Problem AI agents need reliable, zero‑cost web access for research, PDF extraction and bypassing Cloudflare blocks. Existing services require API keys, incur per‑request fees, or run multiple isolated tools, which adds latency and operational overhead.

What This Does master-fetch (the Hound MCP server) runs a single local process that implements the Model‑Context‑Protocol (MCP). It fetches URLs, crawls pages, extracts text with Trafilatura, reads PDFs (including OCR), and performs a small meta‑search layer. The core code lives in src/master_fetch/ – e.g. server.py (entry point, HTTP API), actions.py, crawl.py, ocr.py, and search_metasearch.py. The CLI wrapper (src/master_fetch/__main__.py) starts the server, while the test suite under tests/ validates each tool. No external API keys are required; the only runtime dependency is a local Chromium browser installed via Playwright.

How It Is Wired

  1. Startup – Execution begins at src/master_fetch/__main__.py:main, which invokes src/master_fetch/server.py:main (line 3060). This function creates the MasterFetchServer instance and registers 62 functions and 11 classes.
  2. Request handling – An incoming MCP call reaches _make_response (server.py) which builds a ResponseModel (used in 53 places) and routes to the appropriate tool via _apply_chunking (server.py) and smart_fetch (src/master_fetch/actions.py).
  3. Core work – URL validation → validate_url (called 36 times) may raise SecurityError. Fetch → self.client.request (network call) in get/request methods of MasterFetchServer. Optional crawling → src/master_fetch/crawl.py (deeply nested logic, 8‑level indentation). PDF handling → src/master_fetch/pdf_extractor.py and ocr.py open files (currently without context managers). * Search → src/master_fetch/search_metasearch.py performs outbound HTTP calls (21 call sites).
  4. Persistence – src/master_fetch/cache.py and src/master_fetch/reranker.py read/write a SQLite DB and compute SHA‑256 hashes for caching and ranking.
  5. External effects – The server runs subprocesses (scripts/compress_tooldef.py etc.) and invokes Playwright‑managed Chromium to bypass Cloudflare.

The most‑connected hub is MasterFetchServer (83 inbound call sites); changes here propagate widely. No circular imports were detected, but many modules contain duplicated 6‑line blocks and deep nesting, raising maintenance risk.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/master-fetch
cd master-fetch

# Install Python dependencies (pyproject.toml) and optional extras
pip install "hound-mcp[all]"   # pulls all extras, matches README

# Install the bundled Chromium browser used by Playwright
playwright install chromium

# Start the MCP server (default listens on 127.0.0.1:8000)
python -m master_fetch   # runs src/master_fetch/__main__.py

Configuration is driven by environment variables (e.g., PORT, PROXY_URL) read in src/master_fetch/server.py; the repo does not ship a sample .env file, so users must define the needed vars manually.

Real‑World Use An internal chatbot can call the server via MCP:

import json, httpx

payload = {"action": "fetch", "url": "https://example.com"}
resp = httpx.post("http://127.0.0.1:8000/mcp", json=payload)
data = resp.json()
print(data["content_ok"], data["summary"])

The response contains structured fields (content_ok, page_type, relevance_score) that the agent can branch on without parsing raw HTML.

Code Health & Issues

  • High – No lockfile for package.json; transitive JS deps are not reproducible.
  • Medium – Broad except: clauses (24 sites) swallow errors; replace with specific exception handling.
  • Medium – Files opened without with context (open(...)) in 8 places; may leak handles.
  • Medium – Deep nesting (max depth 8) in actions.py, crawl.py, ocr.py makes reasoning hard.
  • Medium – Oversized server.py (≈2.6 k lines); split into routing, network, and cache modules.
  • Medium – GitHub Actions lack least‑privilege GITHUB_TOKEN permissions and a dependency‑vulnerability scan.
  • Medium – No Dependabot/Renovate configuration; vulnerable ranges (e.g., pydantic>=2.0 includes CVE‑2024‑3772).
  • Low – CI jobs have no timeout-minutes; long hangs can overlap scheduled runs.

All findings are derived from static analysis; no additional hidden issues were observed.

The Bottom Line master-fetch delivers a fully self‑hosted, keyless web‑fetch service with a rich toolset, but the codebase suffers from maintainability problems (large monolith, duplicated logic, deep nesting) and lacks reproducible JS builds and CI hardening. It is suitable for teams that can invest in refactoring and lockfile management, and that need a zero‑cost, locally controlled web accessor for AI agents.