The Problem

Enterprises that want to run LLM‑driven “harnesses” (e.g., Codex, Claude Code, Hermes) on‑premise must stitch together separate model APIs, session handling, streaming, file storage and cancellation logic. Doing this manually leads to duplicated code, fragile integrations and opaque failure handling.

What This Does

HarnessRouter CE provides a single self‑hosted HTTP endpoint that implements the Unified Harness Protocol (UHP).

  • The gateway package (gateway/app.py) runs a Flask API that orchestrates requests, stores media in gateway/media_catalog.json, and persists workspace state via SQLite (gateway/control_sqlite.py).
  • The runner (runner/server.py) launches the actual harness processes, loads installed kits from the persistent volume (/data), and streams results back to the gateway.
  • The protocol folder (protocol/conformance/uhp_conformance/cli.py and related modules) supplies a CLI that validates an installation against the UHP spec and produces JSON reports (protocol/conformance/reports/*.json).
  • A React/Next.js UI (ui/) offers a browser console that talks to the gateway, displays sessions, and lets users add provider keys, all styled with Tailwind.

How It Is Wired

  1. Entry pointgateway/app.py creates the Flask app (create_app()) and registers routes defined in gateway/backing.py.
  2. Request flow – An incoming HTTP request hits a route in backing.py, which constructs a RequestContext (protocol/conformance/uhp_conformance/context.py) and calls gateway/control_store.py to record the job in the SQLite DB (gateway/control_sqlite.py).
  3. Job dispatchgateway/backing.py invokes the runner via a local HTTP call to runner/server.py (the runner runs in a separate container defined in docker-compose.yml). The runner loads the requested kit from the shared volume (/data/kits) and executes it, streaming partial responses through Server‑Sent Events.
  4. Result handling – The runner posts the final payload back to the gateway’s /response endpoint; gateway/media_plane.py writes any attached files to gateway/media_catalog.json. Errors are wrapped by gateway/tests/test_response_error_envelope.py utilities and returned as standardized UHP error envelopes.
  5. Protocol conformance – The CLI (protocol/conformance/uhp_conformance/cli.py) imports checks.py and registry.py to run the full UHP test suite; results are written to protocol/conformance/reports/*.json. This path is the only place where the repository touches the external conformance spec.

The gateway is the widest‑impact component (it touches the DB, file store, and external runner). The runner is isolated to a single container, lowering blast radius for code changes. No circular imports were observed; the call graph is a clear star topology centered on gateway/app.py.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/harnessrouter
cd harnessrouter

# Build and start all services (gateway, runner, UI) with Docker Compose
docker compose up -d

# The UI is reachable at http://127.0.0.1:3000
# First‑time login uses the default credentials shown in the container logs.
# After login, add an LLM provider key via the Settings page.

If you prefer the pre‑built image rather than building locally, the README’s quick‑start works as‑is:

docker pull harnessrouter/harnessrouter
docker run -d --name harnessrouter -p 127.0.0.1:3000:3000 -v harnessrouter:/data harnessrouter/harnessrouter

Environment variables are optional; the container generates a default admin user on first start. For custom configuration, edit .env.example and mount it as --env-file .env.

Real‑World Use

A SaaS platform that needs to run proprietary code‑generation models behind a firewall can replace its ad‑hoc orchestration layer with HarnessRouter. The platform’s backend calls POST /v1/harnesses/run on the gateway; the gateway queues the job, the runner executes the model in an isolated container, and the result streams back to the platform’s existing WebSocket layer without any code changes.

Code Health & Issues

  • Low – Test coverage present (gateway/tests/, protocol/conformance/tests/).
  • Low – CI pipeline (.github/workflows/tests.yml) runs lint, unit tests and Docker build on every push.
  • Low – License file (LICENSE) and contributor guidelines (CODE_OF_CONDUCT.md, CONTRIBUTING.md) are included.
  • No secret files or hard‑coded keys found in the repository.
  • No obvious dead code; each top‑level package (gateway, runner, protocol, ui) has a clear responsibility.

The Bottom Line

HarnessRouter CE delivers a well‑structured, Docker‑first self‑hosted gateway that abstracts away the plumbing required to run multiple LLM harnesses under a single, UHP‑compliant API. It is suitable for teams that need on‑prem control, have existing CI/CD pipelines, and are comfortable with Docker‑compose deployments. Users seeking a fully managed cloud service will find the package overly low‑level, but for on‑prem integration it provides a solid, test‑backed foundation.