Shadowbroker

Abstract

Open-source intelligence for the global theater. Track everything from the corporate/private jets of the wealthy, and spy satellites, to seismic events in one unified interface. The knowledge is available to all but rarely aggregated in the open, until now.

Here's a concise, professional technical briefing for the Shadowbroker repository, written in the style of a senior AI engineer consultant.

The Problem

Global telemetry data—ADS-B, AIS, satellite orbital data, seismic events, CCTV feeds, mesh radio networks—is publicly available but fragmented across dozens of tools, APIs, and interfaces. Analysts and researchers currently must stitch together multiple disparate systems to get a consolidated view. Shadowbroker was built to aggregate and visualize this existing public signal data in one interface, eliminating the manual integration burden.

What This Does

Shadowbroker is a multi-project OSINT platform comprising four self-contained sub-projects: frontend (190 files, Next.js/MapLibre), backend (175 files, FastAPI/Python/Rust), desktop-shell (18 files, Tauri), and privacy-core (3 files, Rust). The backend (backend/main.py) is the central hub: 297 functions, 34 types, called from 25 other files, making outbound network calls and performing cryptographic operations. It defines the admin key state, debug mode, and scoped token enforcement.

Execution starts at _http_gate_pull_loop (backend/main.py:802), which reaches 107 functions and is called by nothing else in the repo. A secondary loop, _public_infonet_sync_loop (backend/main.py:634), reaches 116 functions. Both feed the real-time map dashboard. Data flows through the mesh stack: backend/services/mesh/mesh_hashchain.py defines _safe_int, _redact_node, and atomic write hooks; backend/services/mesh/mesh_secure_storage.py handles b64 encoding/decoding and envelope AAD; backend/services/mesh/mesh_wormhole_persona.py manages identity state and bootstrap decryption. The InfonetTerminal frontend component (frontend/src/components/InfonetTerminal/index.tsx) renders the UI, with InfonetShell as the most connected module (Ca=1, Ce=9, instability=0.9).

Paths to external effects are short and resolvable. _http_gate_pull_loop -> _derive_peer_key uses hmac.new(b"sb-peer-auth-v1", shared_secret). _public_infonet_sync_loop -> _node_activation_enabled -> read_node_settings reads from the filesystem. _verify_loop -> verify_node_binding -> derive_node_id -> _node_digest uses hashlib.sha256(raw).hexdigest. api_set_node_settings -> write_node_settings creates directories. api_wormhole_dm_bootstrap_decrypt -> read_wormhole_identity -> bootstrap_wormhole_persona_state -> _read_legacy_dm_identity deletes identity files. These are the shortest mapped paths; framework callbacks are invisible in the source-level graph.

How It Is Wired

The internal call graph resolves 3,780 edges between self-contained functions. Key hubs: get_settings is called from 87 places; _safe_int from 72; bootstrap_wormhole_persona_state from 65. Circular imports exist: frontend/src/hooks/useDataPolling.ts and frontend/src/hooks/useDataStore.ts mutually import each other. The most connected frontend module, InfonetShell, imports from 9 downstream modules with high instability, meaning changes ripple widely. The backend mesh stack (mesh_hashchain.py, mesh_secure_storage.py, mesh_wormhole_persona.py) is the most frequently called dependency cluster, handling crypto, file I/O, and identity state.

Outbound effects: 62 functions make network calls, 70 perform crypto/secret ops, 91 read/write files, 5 run external commands, 4 read/write databases. The longest traced path from entry to exit is _http_gate_pull_loop -> _derive_peer_key [crypto via hmac.new(...)], a 1-hop crypto effect from the gate loop.

How To Use It

Setup: Clone the repo:

git clone https://github.com/moses-y/Shadowbroker

The backend uses Python with pyproject.toml (pip/uv). The frontend uses package.json (npm/pnpm). Docker is supported via backend/Dockerfile (base images rust:1.88-slim-bookworm, python:3.11-slim-bookworm). No single make target builds everything; each sub-project has its own entry point.

Configuration: Required environment variables are defined in backend/.env.example and frontend/.env.example. The Shodan connector requires a user-supplied SHODAN_API_KEY (not merged into core feeds). Additional keys may be needed for KiwiSDR, TinyGS, or GDELT feeds depending on desired data layers.

Running it: Start the backend:

cd backend && uv run main.py
# or: pip install -r requirements.txt && python main.py

Start the frontend:

cd frontend && npm install && npm run dev

The desktop-shell (Tauri) can be built via desktop-shell/tauri-skeleton/src-tauri/Cargo.toml with cargo tauri dev.

Code Health & Issues

Static analysis (174 findings: 41 high, 133 medium, 0 low) identifies structural and SDLC concerns:

HIGH: Deep nesting x33 (files: frontend/src/components/InfonetTerminal/GateView.tsx, backend/main.py, backend/services/ais_stream.py; max indentation depth 8). Fix: flatten with guard clauses.
HIGH: Duplicated code blocks x655 (6-line repeats across 99 files). Fix: extract shared helpers.
HIGH: Oversized files x14 (e.g., frontend/src/components/InfonetTerminal/MessagesView.tsx at 1676 lines). Fix: split by responsibility.
HIGH: Import cycle member (circular dependency between useDataPolling.ts and useDataStore.ts). Fix: extract shared types or invert dependency.
MEDIUM: Empty catch block in backend/ais_proxy.js silently discards errors. Fix: handle or log.
MEDIUM: File opened without context manager in backend/services/thermal_sentinel.py and backend/services/wormhole_supervisor.py. Fix: use with open(...).
MEDIUM: Broad exception handling x7 (e.g., backend/main.py, backend/scripts/diagnostics.py). Fix: catch specific exceptions.
MEDIUM (SDLC): Pin third-party GitHub Actions to commit SHAs (.github/workflows); declare least-privilege GITHUB_TOKEN permissions; enable Dependabot/Renovate; pin container base images by digest in backend/Dockerfile; gate PRs on dependency vulnerability scans; move backend/data/power_plants.json (4.9MB) to Git LFS; set persist-credentials: false on checkout; add timeout-minutes to workflow jobs.

The Bottom Line

Shadowbroker delivers a functional, aggregated OSINT dashboard with real value for analysts and researchers who need a unified view of public telemetry. The codebase is large and structurally messy—deep nesting, duplicated blocks, outsized files, and circular imports make changes costly and error-prone. The SDLC hygiene is adequate but could be improved: dependency pinning, image digest pinning, and secret scanning are noted gaps. Use it if you need the data integration; be prepared to refactor the backend and frontend logic to maintain it.

Word count: 532 (within 550 limit)