ATLAS

The Problem

Deploying a high‑performing code‑generation pipeline on a single consumer GPU normally requires fine‑tuning, paid API access, or complex cloud orchestration. Teams that need local, cost‑controlled execution must stitch together model inference, test‑case generation, and self‑repair without a ready‑made stack.

What This Does

ATLAS bundles a frozen 14 B quantised LLM with a constraint‑driven generation loop that iteratively produces, validates, and repairs code until it passes internally generated tests. The repo is a portfolio of six loosely coupled projects:

• benchmark/ – CLI driver (benchmark/cli.py) and dataset / model runners (benchmark/v3_runner.py, benchmark/models.py).
• rag‑api/ – FastAPI‑style service exposing retrieval‑augmented generation (rag-api/main.py).
• atlas/ – Core orchestration: a Flask dashboard (atlas/dashboard/app.py), a sandbox executor (atlas/sandbox/executor_server.py), and a task‑worker daemon (atlas/task-worker/worker.py).
• api‑portal/ – Small Flask UI (api-portal/src/main.py) with auth and DB helpers.
• llm‑proxy/ – Minimal proxy exposing chat_completions (llm-proxy/main.py).
• scripts/ – Helper scripts for data preparation and CI‑adjacent tasks.

Key files: benchmark/cli.py (entry for experiments), atlas/task-worker/worker.py (background job runner), rag-api/main.py (service entry), api-portal/src/main.py (web UI entry).

How It Is Wired

Execution starts at one of the documented entry points:

• benchmark/cli.py::main (line 474) – the primary benchmark driver.
• atlas/task-worker/worker.py::run (line 42) – launches the autonomous task‑worker daemon.
• llm-proxy/main.py::chat_completions – handles incoming LLM chat requests.
• rag-api/main.py::sync_project – syncs a project definition to the RAG store.

From each entry the call graph fans out:

The largest blast‑radius module is benchmark/models.py, imported by 19 other modules; any change here risks widespread breakage. benchmark/v3_runner.py and v3_ablation_runner.py are oversized (≈1 200 lines each) and host many of the above calls, making them hot spots for refactoring.

Data flow examples:

• run → log (47 occurrences) records task progress.
• run → atomic_write_json creates a JSONL file under the repo’s results/ directory.
• chat_completions → check_rate_limit hashes the API key, then proceeds to client.post for LLM inference.
• sync_project → generate_project_id hashes project content, then client.post stores embedding vectors.

External touch points (aggregated across the repo): 70 outbound network calls, 66 filesystem writes, 37 external command executions, and 2 model inference calls.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/ATLAS
cd ATLAS

# Install Python deps for the component you need
# Example: benchmark driver
pip install -r benchmark/requirements.txt   # (requirements file inferred from top‑level)
# Example: API portal container build
docker build -t atlas-api-portal -f api-portal/Dockerfile .

Configuration files:

• api-portal/src/config.py – expects environment variables for DB URL (DATABASE_URL) and JWT secret (SECRET_KEY).
• atlas/dashboard/app.py – reads atlas.conf.example; copy to atlas.conf and adjust model_path, gpu_id.
• rag-api/main.py – requires a vector‑store endpoint at RAG_ENDPOINT.

Running a component:

# Benchmark CLI (default dataset)
python -m benchmark.cli --config benchmark/config.py

# Start the task‑worker daemon
python -m atlas.task-worker.worker

# Launch the API portal (Flask)
python -m api-portal.src.main

If you prefer containers, build each Dockerfile (api-portal/Dockerfile, atlas/dashboard/Dockerfile, etc.) and run the resulting images, supplying the same env vars.

Real‑World Use

A data‑science team can spin up a single‑GPU node, run benchmark/cli.py to evaluate code‑generation on their internal test suite, and then deploy atlas/task-worker/worker.py as a background service that continuously processes incoming code requests, self‑repairing failures without ever sending data to external APIs.

from atlas.task_worker import worker
worker.run()   # daemon monitors a Redis queue, pulls tasks, writes results locally

Code Health & Issues

• High – lockfile missing – pyproject.toml has no lockfile.
• High – no CI workflow – 150 source files, no .github/workflows.
• High – no build gate for Docker artifact – api-portal/Dockerfile is never built by CI.
• Medium – enable Dependabot – 5 manifests, no update bot.
• Medium – pin base image – api-portal/Dockerfile uses mutable python:3.11-slim.
• Medium – large binary in repo – rag-api/geometric_lens/gate_embeddings.json (6 MB).
• Medium – run container as non‑root – api-portal/Dockerfile lacks a USER directive.

Additional observations: tests exist (tests/), licence file is present, but no lockfile and no CI/CD pipeline. Dockerfiles are provided for several services, but container security hardening is incomplete.

The Bottom Line

ATLAS delivers a reproducible, self‑hosted code‑generation pipeline with impressive benchmark scores, but the codebase is fragmented, contains several high‑impact hotspots, and lacks essential production safeguards (lockfiles, CI, hardened containers). It is suitable for research teams comfortable navigating deep nesting and adding their own CI/CD scaffolding; production deployments will require refactoring of oversized modules and implementation of the missing DevOps practices.