The Problem

Organizations deploying autonomous LLM‑driven agents have no systematic way to verify that the agents resist prompt injection, tool abuse, or configuration drift before they are released. The result is costly post‑deployment breaches that could have been caught in a pre‑flight test.

What This Does

SuperClaw is a scenario‑driven red‑team framework for AI agents. Attack techniques live under src/superclaw/attacks/ (e.g., jailbreaks.py, prompt_injection.py) and are executed against a target adapter defined in src/superclaw/adapters/ (the default mock adapter is mock.py; the OpenClaw integration is openclaw.py). Security‑related behaviours – such as protocol_security.py or sandbox_isolation.py – are implemented in src/superclaw/behaviors/ and can be composed into a test run.

The CLI (src/superclaw/cli.py) orchestrates configuration (src/superclaw/config/), selects attacks, runs them through the chosen adapter, evaluates results with the codeoptix engine, and emits reports via src/superclaw/reporting/ in HTML, JSON, or SARIF.

How It Is Wired

Entry point – src/superclaw/cli.py parses command‑line options, loads settings from src/superclaw/config/settings.py, and builds a SuperClawRunner (conceptually) that:

  1. Adapter selection – imports the adapter module (adapters/openclaw.py or adapters/mock.py) and instantiates its class.
  2. Attack dispatch – iterates over the attack package. The most‑imported module is attacks/base.py (Ca = 5), which defines the Attack base class and common helpers. Specific attacks (encoding.py, jailbreaks.py, etc.) subclass it and are invoked via a simple factory in attacks/__init__.py.
  3. Behavior enforcement – each attack optionally registers behaviour checks (behaviors/*). The behaviours are lightweight; the most‑connected behaviour file is behaviors/protocol_security.py, which contains deep nesting (indentation depth 8) and thus high cognitive load.
  4. Evaluation – the codeoptix/engine.py runs the agent’s output through the evaluator.py to extract evidence. This module shares duplicated helper blocks with attacks/__init__.py (identified as a high‑severity DRY violation).
  5. Reporting – final results are handed to reporting/html.py, json_report.py, or sarif.py. All reporting modules write only to the filesystem; no external services are contacted.

The static import graph shows 43 internal modules with only 5 import edges and no circular dependencies, indicating a very low coupling surface. The hub is attacks/base.py, which is imported by five other modules but imports none itself, limiting blast radius when changes are made there.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/superclaw
cd superclaw

# Install dependencies (uv is declared in pyproject.toml)
uv pip install -e .

# Run a quick sanity check (default mock adapter + all attacks)
python -m superclaw.cli --adapter mock --attacks all --report html

Configuration – edit src/superclaw/config/settings.py or supply a YAML file referenced with --config path/to/file.yml. The OpenClaw adapter requires the environment variable SUPERCLAW_AUTH_TOKEN (checked in adapters/openclaw.py). No other secret files are present.

Reporting – output files are written to the current working directory; choose format with --report {html,json,sarif}.

If any of the above files are missing (e.g., a Dockerfile), the repository does not support container‑based execution out of the box.

Real‑World Use

A CI pipeline can embed SuperClaw as a gate before releasing a new version of an autonomous coding assistant:

- name: Run security tests
  run: |
    python -m superclaw.cli \
      --adapter openclaw \
      --attacks jailbreaks,prompt_injection \
      --report sarif > superclaw.sarif
- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: superclaw.sarif

The SARIF report can be consumed by GitHub’s security tab, giving developers immediate feedback on newly introduced agent vulnerabilities.

Code Health & Issues

Measured audit (7 findings)

  • HIGH – Pin third‑party GitHub Actions to a commit SHA (.github/workflows/*.yml).
  • HIGH – No test suite; 43 source files lack corresponding tests.
  • MEDIUM – Declare least‑privilege permissions for GITHUB_TOKEN (.github/workflows/ci.yml).
  • MEDIUM – Enable Dependabot or Renovate (.github/dependabot.yml missing).
  • MEDIUM – Add a dependency‑vulnerability scan step in CI.
  • MEDIUM – Set persist-credentials: false on actions/checkout.
  • LOW – Add timeout-minutes to workflow jobs.

Additional static findings

  • HIGH – Duplicated 6‑line code blocks across 21 files (e.g., attacks/__init__.py, codeoptix/engine.py).
  • HIGH – Deep nesting (max depth 8) in several modules, making logic hard to follow.
  • MEDIUM – Broad except: clauses in adapters/openclaw.py, attacks/__init__.py, codeoptix/engine.py.
  • MEDIUM – Oversized CLI file (src/superclaw/cli.py, 791 lines).
  • MEDIUM – High branching density in attacks/multi_turn.py (57 branches/202 lines).

No lockfile is present (pyproject.toml only), so reproducible builds depend on the current state of PyPI.

The Bottom Line

SuperClaw provides a concrete, extensible framework for pre‑deployment security testing of AI agents, with a clear separation of adapters, attacks, behaviours, and reporting. The codebase is lightweight and low‑coupled but suffers from a lack of automated tests, duplicated logic, and deep nesting that will hinder maintenance. It is suitable for teams that can invest in adding a test suite and refactoring hot spots, and who need a programmable red‑team harness integrated into CI pipelines.