The Problem Organizations that run DNS‑based blockers (Pi‑hole, AdGuard, Unbound, etc.) need a continuously refreshed list of phishing and scam domains. Maintaining that list manually is error‑prone and costly, especially when the threat landscape changes multiple times per day.

What This Does destroylist ships a curated blocklist of >208 k domains plus daily community contributions. The repository stores the raw data (rootlist/, community/, dns/) and a set of small Python utilities that transform the JSON sources into the various feed formats (hosts, AdBlock, RPZ, etc.). Key scripts include:

  • scripts/build_rootlist.py – reads the monthly JSON files in rootlist/ and writes the canonical list.txt/list.json.
  • scripts/json_to_txt.py – converts the master JSON into plain‑text feeds; functions load_domains, header, adblock_header, rpz_header are used by the CI pipelines.
  • scripts/smart_aggregator.py – normalises domains, removes duplicates and IPs, and aggregates the final set (is_valid_domain, normalize_domain, add_norm).
  • dns/active_domains.py – exposes the current active set via a tiny JSON API used by the “live” feeds (fetch_domains, check_domain).

The data files are version‑controlled; GitHub Actions (see .github/workflows/*.yml) invoke the scripts on each push, generate the feeds, and publish them through GitHub Pages and the jsDelivr CDN.

How It Is Wired

  1. Trigger – A push to main fires the rootlist.yml workflow.
  2. Build – scripts/build_rootlist.py loads every rootlist/YYYY/MM/*.json, calls scripts/json_to_txt.py to produce list.txt, list.json, and the per‑format files under rootlist/formats/.
  3. Aggregation – scripts/smart_aggregator.py is called next; it normalises domains (normalize_domain), strips IPs (_is_ip_addr from scripts/utils.py), and writes the deduplicated set back to the JSON feed.
  4. Live Export – dns/active_domains.py reads the generated JSON (dns/active_domains.json) and serves a read‑only API used by the CDN‑hosted hosts.txt and other formats.
  5. Deploy – The pages.yml workflow pushes the rootlist/formats/ directory to the GitHub Pages site, which jsDelivr mirrors for public consumption.

No import cycles exist; each script is self‑contained, which limits blast radius but also means shared logic is duplicated (see findings). The most‑used modules are scripts/smart_aggregator.py (17 functions) and scripts/json_to_txt.py (8 functions). All file‑level responsibilities are enumerated in the “What Each File Is Responsible For” block of the analysis.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/destroylist
cd destroylist

# Install Python runtime dependencies
pip install -r requirements.txt

# Generate the latest feeds locally (optional)
python scripts/build_rootlist.py
python scripts/json_to_txt.py
python scripts/smart_aggregator.py

The produced feeds are available under rootlist/formats/. For production use point your blocker at the CDN URL, e.g.:

https://cdn.jsdelivr.net/gh/moses-y/destroylist@main/rootlist/formats/primary_active/hosts.txt

No additional configuration files or environment variables are required; the scripts operate purely on the repository data.

Real‑World Use A corporate network running Pi‑hole can add the CDN URL to its blocklist source. On each daily sync, Pi‑hole fetches the hosts.txt file, which contains the deduplicated, normalised domain set produced by smart_aggregator.py. Because the feed is regenerated on every push, new phishing domains appear in the blocklist within minutes of being added to the upstream JSON.

Code Health & Issues

  • Med – Duplicated code – 8‑line blocks repeated across scripts/calculate_stats.py, scripts/json_to_txt.py, scripts/update_counts.py, scripts/smart_aggregator.py.
  • Med – Broad exception handling – Catch‑all except: in dns/active_domains.py, scripts/allowlist_pattern_guard.py, scripts/calculate_stats.py.
  • Med – Deep nesting – Indentation depth of 6 in scripts/smart_aggregator.py and tests/test_allowlist_pattern_guard.py.
  • Med – High branching density – 27 branches in scripts/validate_json.py.
  • Low – Missing lockfile – Dependencies are listed only in requirements.txt; reproducible builds are not guaranteed.

All findings are derived from deterministic static analysis; no additional issues were detected.

The Bottom Line destroylist provides a well‑structured, continuously updated phishing blocklist with straightforward Python tooling for feed generation. The codebase is functional but suffers from duplicated helpers and overly broad error handling, which modestly increase maintenance effort. It is suitable for teams that need a free, community‑driven threat feed and are comfortable running the simple Python scripts or consuming the CDN‑hosted outputs.