The Problem

Current open‑ended AI research lacks a reproducible framework for groups of agents to evolve together and share improvements. Teams that want to experiment with self‑improving code‑generation agents must stitch together ad‑hoc scripts, manual Docker builds, and separate benchmark suites (SWE‑bench, Polyglot). The result is fragile pipelines, duplicated logic, and a high barrier to systematic experimentation.

What This Does

Group Evolving Agents (GEA) provides a ready‑to‑run research stack that treats a population of coding agents as the evolutionary unit.

  • The core loop lives in GEA_outer.py, which orchestrates generations, loads prompts from prompts/, and invokes the initial agent implementation in coding_agent.py (or its polyglot variant coding_agent_polyglot.py).
  • Interaction with the SWE‑bench evaluation suite is encapsulated in swe_bench/, while cross‑language benchmarking is handled by polyglot/.
  • Shared utilities such as logging, file handling, and common data structures are in utils/ (e.g., utils/common_utils.py is the most‑imported module, with seven inbound imports and no outbound dependencies).

The repository ships with a Dockerfile that bundles the Python environment, the SWE‑bench checkout, and the Polyglot dataset preparation script (polyglot/prepare_polyglot_dataset). All experimental artifacts (logs, diffs, reports) are written under initial/ and initial_polyglot/.

How It Is Wired

  1. Entry point – python GEA_outer.py starts execution.
  2. GEA_outer.py imports the evolution driver self_improve_step (the module with the highest outward coupling: 10 imports, 1 inbound). This driver runs the self‑improvement loop, calling: coding_agent.run() – the primary LLM‑driven code generation routine (found in coding_agent.py). llm_withtools.py – wraps OpenAI/Anthropic calls and provides tool‑use helpers; it is also a hotspot for broad except: clauses.
  3. After each generation, GEA_outer.py writes results to the appropriate sub‑directory in initial/ or initial_polyglot/.
  4. For SWE‑bench evaluation, GEA_outer.py invokes the swe_bench package, which in turn calls the benchmark harness in polyglot/harness.py. The harness imports utils/common_utils.py (the zero‑outbound, seven‑inbound hub), making it the stability anchor of the codebase.
  5. Docker builds are driven by Dockerfile → polyglot/docker_build.py (2 inbound, 3 outbound imports). The build script assembles the container, installs requirements.txt, and copies the repository source. No USER directive or image digest pinning is present (see health section).

The import graph contains 35 internal modules and 51 edges, with no circular dependencies, meaning a change to a leaf module (e.g., prompts/self_improvement_prompt.py) has limited blast radius, whereas modifications to hub modules like utils/common_utils.py affect many downstream components.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/GEA
cd GEA

# Create a virtualenv and install Python deps
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Build the Docker image (optional but recommended for reproducibility)
docker build -t gea:latest .

# Prepare the Polyglot benchmark data
python -m polyglot.prepare_polyglot_dataset

# Pull and install SWE‑bench (required for evaluation)
cd swe_bench
git clone https://github.com/princeton-nlp/SWE-bench.git
cd SWE-bench
git checkout dc4c087c2b9e4cefebf2e3d201d27e36
pip install -e .
cd ../../

# Export required API keys (see README)
export OPENAI_API_KEY=...
export ANTHROPIC_API_KEY=...

# Run the evolution loop
python GEA_outer.py

The above steps follow the exact commands documented in README.md. No additional configuration files are required; the code reads environment variables directly.

Real‑World Use

A research team can spin up the Docker image on a GPU node, point OPENAI_API_KEY at their own account, and run GEA_outer.py to generate a series of agent generations. After each run, the initial/ directory contains JSON reports (report.json) and diff patches that can be fed into a CI pipeline for regression testing or further analysis.

Code Health & Issues

  • High – Add CI workflow – 35 source files, no .github/workflows/ present.
  • High – Add Docker build gate – Dockerfile exists but is never validated automatically.
  • Medium – Enable Dependabot – Only requirements.txt is present; no automated version updates.
  • Medium – Pin base image by digest – Dockerfile uses python:3.10-slim without a SHA.
  • Medium – Add non‑root USER – Container runs as root; no USER directive.

Additional static findings:

  • Broad except: blocks in llm_withtools.py, polyglot/docker_build.py, GEA_outer.py.
  • Deep nesting (up to 6 levels) in llm.py, llm_withtools.py, prompts/self_improvement_prompt.py.
  • Repeated code blocks across coding_agent*.py and many evaluation scripts.
  • Oversized files (llm_withtools.py, prompts/self_improvement_prompt.py, polyglot/benchmark.py).
  • File opened without a context manager in prompts/tooluse_prompt.py.

These findings are deterministic outputs of the repository‑wide static analysis; no speculative issues are added.

The Bottom Line

GEA delivers a functional, Docker‑based research platform for group‑level self‑improving agents, with clear entry points and a modest dependency footprint. However, the codebase suffers from maintainability problems (deep nesting, duplicated logic) and lacks essential production safeguards (CI, locked dependencies, hardened container). It is suitable for experimental research groups that can tolerate manual testing but should be refactored before any production‑grade deployment.