The Problem

Developers who interact with Claude Code or Codex generate conversation logs that contain personal identifiers and API keys. Extracting those logs, sanitising them, and publishing them as a reusable dataset is tedious and error‑prone, especially when the same tooling must be repeated across multiple machines.

What This Does

dataclaw is a small Python CLI that turns Claude Code / Codex session histories into a Hugging‑Face‑ready JSONL dataset.

  • Parsing – dataclaw/parser.py walks the local ~/.cache/claude (or Codex) directory tree, discovers projects and extracts per‑session JSONL records (parse_project_sessions).
  • Redaction – dataclaw/secrets.py scans each text line for high‑entropy strings, mixed‑character tokens, and user‑defined patterns, then replaces them via redact_text / redact_custom_strings.
  • Anonymisation – dataclaw/anonymizer.py hashes usernames (_hash_username) and rewrites file‑system paths (anonymize_path).
  • Export – dataclaw/cli.py builds the final dataset, writes a local file (export_to_jsonl) and optionally pushes it to Hugging Face (session.get inside redact_session).

All configuration (source selection, exclusions, custom redaction lists) lives in dataclaw/config.py (load_config / save_config).

How It Is Wired

Execution starts at dataclaw/cli.py::main (line 1149). The command dispatcher routes to sub‑commands:

  • prep → dataclaw/cli.py::prep (line 1076) – discovers projects via parser.discover_projects and writes a config file (save_config).
  • list → dataclaw/cli.py::list_projects (line 271) – reads the same config and prints project metadata.
  • export → dataclaw/cli.py::_run_export (line 1291) – iterates selected sessions, calls redact_session (via secrets.scan_text → secrets.redact_text), then writes JSONL (export_to_jsonl). The export path touches the filesystem (CONFIG_DIR.mkdir, dest.parent.mkdir) and makes a network request (session.get) to Hugging Face if --publish is used.
  • confirm → dataclaw/cli.py::confirm (line 906) – recomputes redaction stats (_scan_pii) and writes an attestation file.

Key internal hot‑spots:

  • scan_text (in secrets.py) is invoked from 37 locations, making it the most widely used function.
  • _hash_username (in anonymizer.py) is called from 12 places, handling all user‑identifier hashing.

The import graph shows dataclaw/cli.py as the central hub (imports 4 modules, is imported by 1). No circular imports exist, so refactoring a single module has predictable ripple effects.

External interactions:

  • Filesystem – config read/write, project discovery, dataset output (26 distinct functions).
  • Network – urllib.request.urlopen in cli.update_skill, session.get in secrets.redact_session.
  • Cryptography – SHA‑256 hashing in anonymizer._hash_username.

How To Use It

# Install from PyPI (or the repo if that fails)
pip install dataclaw
# Or:
git clone https://github.com/moses-y/dataclaw.git /tmp/dataclaw && pip install /tmp/dataclaw

# Initialise config and pick a source
dataclaw prep
dataclaw config --source "claude|codex|both"

# Review discovered projects
dataclaw list --source both

# Optional: exclude or add custom redactions
dataclaw config --exclude "personal,scratch"
dataclaw config --redact "my-secret.com,api_key"

# Export locally first
dataclaw export --no-push --output /tmp/dataclaw_export.jsonl

# Review the PII report, then attest and publish
dataclaw confirm \
  --full-name "YOUR NAME" \
  --attest-full-name "Scanned export for full name." \
  --attest-sensitive "No sensitive data found." \
  --attest-manual-scan "Manual review completed."

dataclaw export --publish-attestation "User approved publishing."

All commands are defined in dataclaw/cli.py; the --help flag lists available options.

Real‑World Use

A data‑science team that archives internal code‑review sessions can run the above workflow nightly on a CI runner. The resulting Hugging‑Face dataset can be consumed by downstream models without exposing proprietary identifiers, because dataclaw guarantees deterministic hashing (_hash_username) and entropy‑based redaction (scan_text).

Code Health & Issues

  • HIGH – Pin GitHub Actions – .github/workflows/*.yml uses pypa/gh-action-pypi-publish@release/v1. Replace with a commit SHA.
  • HIGH – Missing lockfile – pyproject.toml declares dependencies but no poetry.lock/requirements.txt. Generate and commit a lockfile.
  • MEDIUM – Unrestricted GITHUB_TOKEN – .github/workflows/test.yml lacks a permissions: block. Add contents: read.
  • MEDIUM – No Dependabot/Renovate – No automated update config. Add .github/dependabot.yml.
  • MEDIUM – No dependency‑vulnerability scan – CI lacks a review step. Add dependency-review-action or osv-scanner.
  • MEDIUM – Checkout persists credentials – .github/workflows/publish.yml should set persist-credentials: false.
  • LOW – No job timeout – Add timeout-minutes: to each workflow job.

Additional observations from the static audit:

  • Two files (dataclaw/cli.py, tests/test_cli.py) exceed 1 300 lines, creating high cognitive load. Splitting by responsibility (e.g., separate command modules) would reduce risk.
  • Deep nesting (max depth 6) appears in parser.py and related tests; refactoring to early returns will improve readability.
  • secrets.py contains 62 branch points over 170 lines; a strategy‑pattern dispatch could simplify the decision tree.

The Bottom Line

dataclaw delivers a focused, reproducible pipeline for sanitising and publishing Claude/Codex conversation logs, but the codebase suffers from oversized, deeply nested modules and a few CI‑security gaps. It is suitable for teams that need an immediate, auditable export tool and are prepared to invest in modest refactoring and CI hardening.