The Problem

Training and evaluating agentic models requires replaying multi-turn agent-environment interactions at scale. Research groups end up rebuilding the same sandboxing substrate—container orchestration, file I/O, exec, git-patch handling—for every new recipe. Orchard packages that layer as a stable service so trajectories, training recipes, and evaluation protocols stay portable across harnesses and domains.

What This Does

Orchard is a Kubernetes-native sandbox service plus Python SDK. The core lives in orchard_env/: a client (client/sandbox_client.py, client/process.py) that talks to an orchestrator (orchestrator/api.py, orchestrator/k8s_client.py, orchestrator/sandbox_manager.py) which spins up containers on demand. A lightweight agent (agent/server.py) runs inside each sandbox and handles exec, file transfer, and PTY sessions over HTTP.

The repo also contains a vendored RL training stack under trainer/slime/ (a forked slime with Orchard rollout code), plus Kubernetes manifests in k8s/ and deployment scripts in scripts/. The README ties this to a paper (arXiv:2605.15040) and a Hugging Face dataset with SWE and GUI trajectories.

How It Is Wired

Execution starts in one of two places. The orchestrator entry point is main in orchard_env/orchard_env/orchestrator/main.py, reaching 94 functions. The agent-side server exposes HTTP handlers from agent/server.py: exec_command (reaches 58 functions), upload_file, download_file, list_files. The agent's main at server.py:567 is the only entry that reaches a wide slice of the program.

The call graph shows a clear hub-and-spoke structure. exec is called from 29 places, create_sandbox from 20, _request from 14. The highest-risk functions are in client/sandbox_client.py (67 functions, called from 15 files) and orchestrator/k8s_client.py (32 functions, 5 callers). k8s_client.py is the Kubernetes API gateway—change it and everything downstream breaks.

Traced paths to external effects are short. From main → run_harness → upload_content → _request, the code hits the network in four hops. From exec_command → submit_exec → get_sandbox, three. Filesystem I/O lives in agent/server.py and agent/pty_runner.py; database access in client/process.py; network calls in the client, orchestrator, and pod_watcher.py.

How To Use It

  • Setup: cd orchard_env && pip install -e . (pyproject.toml present; no lockfile committed)
  • Configuration: Kubernetes manifests in k8s/—configmap.yaml, secret.example.yaml, deployment.yaml. The README documents deployment via scripts/deploy_aks.sh and scripts/deploy_k8s.sh.
  • Running it: Start the orchestrator via orchestrator/main.py, then use the SDK from examples/getting_started.py. The agent server runs inside each sandbox via agent/server.py.
cd orchard_env
pip install -e .
python -m orchard_env.orchestrator.main

The README references a full deployment flow; the exact commands are in scripts/ rather than the top-level docs.

Real-World Use

A training pipeline launches thousands of rollouts. The orchestrator provisions a Kubernetes pod per rollout, the agent inside exposes exec/file endpoints, and the SDK drives multi-turn interaction—exec a command, read output, apply a git patch, capture the trajectory. The agent_harness.py example shows this pattern; tests/integration/bench_concurrent.py benchmarks it under load.

Code Health & Issues

Static analysis found 27 findings (5 high, 22 medium). The high-severity items:

  • High - Duplicated code blocks: 25 repeated 6-line blocks across 8 files, including orchestrator/job_store.py and orchestrator/redis_job_store.py. Extract shared helpers.
  • High - Deep nesting: 13 files with max indentation depth 8, notably orchestrator/k8s_client.py and orchestrator/sandbox_manager.py. Flatten with early returns.
  • High - Oversized files: 5 files over 1,000 lines, including client/sandbox_client.py (1,493 lines) and orchestrator/api.py. Split by responsibility.
  • Medium - Broad exception handling: 6 files catching Exception indiscriminately, including agent/server.py.
  • Medium - High branching density: 2 shell scripts with 14 branch points over 47 lines.

SDLC gaps: no CI/CD pipeline, no lockfile for reproducible builds, no Dependabot/Renovate. The Dockerfile uses python:3.11-slim unpinned by digest. Tests exist (9 files) but nothing runs them automatically.

The Bottom Line

Orchard is a serious attempt at a reusable agentic sandboxing substrate with real research backing. The architecture is sound—client/orchestrator/agent separation is clean—but the codebase needs refactoring before it's maintainable at scale. The missing CI and lockfiles are the first things to fix. Best suited for research teams already running Kubernetes who want a proven sandbox layer rather than building their own.