Technical Briefing: Cairn

The Problem

Cairn addresses the problem of general state-space search where an origin and goal are known but the path between them is unknown. Penetration testing is one validated domain, but the engine targets any problem with a clear start, success condition, and unknown path. The core architecture uses a blackboard with fact-intent-hint primitives, where workers operate on the shared board via stigmergy (no direct communication). This is the first public implementation of this general search engine.

What This Does

Cairn is a general-purpose problem-solving engine built around three task types executed by workers: Bootstrap (attempts direct solution at project start), Reason (reads the full graph to determine if goal is met or what to explore next), and Explore (claims one intent, executes exploration, reports findings). The system uses a dispatcher that schedules tasks, manages containers, and implements a protocol API. Workers run an OODA loop (Observe, Orient, Decide, Act) against the shared state board.

The codebase contains 83 files across multiple directories: cairn/src/cairn/ (65 files), container/ (4 files), docs/ (2 files), README/ (4 files), and .github/ (1 file). Python constitutes 40 of the files, with JavaScript, YAML, TOML, and Markdown making up the remainder.

How It Is Wired

Execution starts at three entry points: dispatch from cairn/src/cairn/cli.py:57 (reaches 12 functions, called by nothing else in repo), serve from cairn/src/cairn/cli.py:28 (reaches 2 functions), run from cairn/src/cairn/dispatcher/scheduler/loop.py:72 (reaches 96 functions, the primary scheduler entry), and lifespan from cairn/src/cairn/server/app.py:16 (reaches 2 functions).

The internal call graph contains 459 resolved call edges between repository functions. High-degree functions include get_conn (called from 20 places), get_project_or_404 (11 places), and utcnow (11 places). Representative edges show _try_conclude_fallback -> best_effort_release (18 calls) and run_bootstrap_task -> best_effort_release (8 calls).

Execution paths leave the process through two traced routes: serve -> configure [filesystem via _db_path.parent.mkdir] and run -> list_projects [db via conn.execute]. The dispatcher/scheduler/loop.py is the largest single file at 799 code lines and routes through 40 functions; it calls into 6 other modules. cairn/src/cairn/server/models.py defines 5 validation functions and 21 classes/types called from 4 other files. cairn/src/cairn/dispatcher/runtime/containers.py manages 23 functions handling container lifecycle and is called from 7 other files, reading/writing files on the filesystem.

Execution touches the outside world through 5 filesystem-write functions and 33 database-read/write functions. The server app configures the database path; the scheduler loop runs healthchecks that attach processes.

How To Use It

Setup: The repository uses Python with a cairn/pyproject.toml dependency file and Dockerfile at root. A docker-compose.yaml orchestrates services with a container base image ghcr.io/astral-sh/uv:python3.13-trixie. No lockfile is present in the measured analysis, meaning installs may pull latest compatible versions.

Configuration: Environment variables and keys are referenced in the code but no specific .env file or template exists in the structure. The cairn/pyproject.toml declares project configuration; container/Dockerfile sets the runtime base.

Running it: The CLI entry point is cairn/src/cairn/cli.py. Based on the dispatch and serve functions, invocation likely follows python -m cairn cli dispatch or python -m cairn cli serve from a virtual environment created via the project's dependencies. No explicit start script appears in the structure beyond the CLI module.

Real-World Use

A security team wanting to automate exploratory assessment of a target system could initialize a Cairn project with the target as origin and desired access level as goal. The Bootstrap task would attempt a direct exploit; if that fails, the Reason task analyzes the fact-intent graph to determine next exploration directions. Workers then claim intents and execute exploration steps, writing each finding back as a new Fact. The board grows toward the goal state, with human operators able to inject Hints at any time to guide direction. This workflow replaces manual reconnaissance loops with a structured search process that documents each step as a verifiable fact.

Code Health & Issues

  • HIGH - Duplicated code blocks: 87 repeated 6-line blocks across 5 files (cairn/src/cairn/dispatcher/tasks/bootstrap.py, explore.py, reason.py, and server/routers/projects.py). Fix: extract shared helpers; DRY the repeated logic.
  • HIGH - Broad exception handling x5: except clauses swallows errors indiscriminately in startup_healthcheck.py, scheduler/loop.py, and bootstrap.py. Fix: catch specific exceptions; re-raise or log the rest.
  • HIGH - Pin third-party GitHub Actions to commit SHAs: .github/workflows uses @vN tags for docker/setup-buildx-action@v3, docker/login-action@v3, docker/build-push-action@v6. Tags can move, so actions running with secrets change without notice. Fix: replace @vN with 40-character commit SHA; let Dependabot bump SHAs.
  • HIGH - No test suite: 40 source files, zero test files. Any change ships with no signal existing behavior holds. Fix: add one test per public entry point, then a CI step that runs them.
  • MEDIUM - Privileged mode and host networking in docker-compose.yaml: docker socket is mounted, privileged flag is present. Fix: grant specific capabilities with cap_add, publish ports explicitly.
  • MEDIUM - Oversized file: cairn/src/cairn/dispatcher/scheduler/loop.py at 799 code lines. Fix: split into cohesive units by responsibility.
  • MEDIUM - No Dependabot/Renovate configured: 1 manifest, no update bot. Fix: commit .github/dependabot.yml covering repo ecosystems plus github-actions.
  • LOW - No timeout-minutes on workflow jobs: .github/workflows/build-container-ghcr.yml declares no job timeout. Fix: add timeout-minutes with realistic bound.

The Bottom Line

Cairn presents a credible general-state-search engine with a novel blackboard architecture that cleanly separates Fact, Intent, and Hint primitives. The codebase is functional but shows signs of rapid prototyping: duplicated logic, bare exception catches, and an oversized scheduler module create maintenance risk. The SDLC hygiene gaps (no lockfile, no tests, pinned Action tags using semver) are standard for early-stage projects but should be addressed before production use. Teams comfortable with Python/Django-style async patterns and Docker-based workflows can adopt this for structured exploration problems, but should budget time to refactor the duplicated task logic and add test coverage before extending the platform.