The Problem
Running large‑language‑model (LLM) pipelines locally typically requires fine‑tuning or costly API calls to reach competitive coding performance. Teams that need data‑privacy, predictable cost, or single‑GPU deployment lack a ready‑to‑run stack that combines structured generation, verification, and self‑repair.
What This Does
ATLAS supplies a self‑contained inference and refinement pipeline built around a frozen 14 B quantised model. The core orchestration lives in atlas/dashboard/app.py (FastAPI server) and the worker processes in atlas/task‑worker/worker.py. Benchmark utilities (benchmark/cli.py, benchmark/v3/) implement the “PlanSearch → Lens routing → self‑verified repair” flow that lifts pass@1 from ~40 % to 74.6 % on LiveCodeBench.
The repository also ships a Retrieval‑Augmented Generation API (rag‑api/main.py) and a lightweight web UI (api‑portal/src/main.py with api‑portal/templates/.html). Dockerfiles for each component (api‑portal/Dockerfile, atlas/dashboard/Dockerfile, atlas/task‑worker/Dockerfile, llama‑server/Dockerfile, llm‑proxy/Dockerfile) enable Kubernetes deployment via the manifests in manifests/.
How To Use It
Setup
Build all containers (script uses the Dockerfiles above) ./scripts/build-containers.sh
Pull the quantised Qwen3‑14B model (script placeholder)
./scripts/download-models.sh
If you prefer a local Python environment, install the top‑level dependencies: pip install -r api-portal/requirements.txt pip install -r atlas/task-worker/requirements.txt pip install -r rag-api/requirements.txt pip install -r llm-proxy/requirements.txt
(Dependency versions are not locked; consider generating a requirements‑lock.txt with pip freeze.)
Configuration
atlas.conf.example must be copied to atlas.conf and populated with paths to the model checkpoint and any GPU settings. api-portal/src/config.py reads environment variables DBURL and SECRETKEY; set them before launch. Kubernetes manifests reference ConfigMaps named atlas-config and rag-api-config; create them from the corresponding *.conf files.
Running
Start the dashboard (FastAPI) locally python atlas/dashboard/app.py # listens on 0.0.0.0:8000
Start a task‑worker process
python atlas/task-worker/worker.py --queue redis://localhost:6379/0
For end‑to‑end evaluation: python benchmark/cli.py --pipeline v3 --tasks benchmark/custom/tasks.json
The CLI prints pass@k‑v(k=3) scores and stores logs under benchmark/.
Real‑World Use
A software‑testing team can deploy the Docker stack on an on‑prem K3s cluster, point their CI jobs at http://atlas-dashboard:8000/generate, and receive self‑validated code snippets without exposing proprietary code to external APIs. Example client call (Python): import requests payload = {"prompt": "def fib(n):", "max_tokens": 256} resp = requests.post("http://atlas-dashboard:8000/v3/generate", json=payload) print(resp.json()["code"])
Code Health & Issues
Medium – No CI/CD – No .github/workflows or other pipeline definitions; changes are not automatically linted or tested. Low – Missing lockfiles – requirements.txt files are not version‑pinned, risking non‑reproducible builds. Medium – Inconsistent entry points – Multiple main.py files (api-portal/src/main.py, rag-api/main.py, llm-proxy/main.py) each start a FastAPI server; documentation does not clarify which should be primary in a production deployment. Low – Sparse error handling – Workers (atlas/task-worker/worker.py) assume successful Redis connections; no retry logic is present. Low – Documentation gaps – README describes benchmark results but does not detail required environment variables beyond the example config file. Medium – Potential race condition – atlas/task-worker/metrics.py updates shared in‑memory counters without locks; could corrupt metrics under high concurrency.
The Bottom Line
ATLAS delivers a functional, containerised pipeline that pushes a frozen 14 B model to competitive coding performance without external APIs. The codebase is sizable and includes benchmarks, but it lacks CI, version‑locked dependencies, and robust production‑grade safeguards. It is suitable for teams comfortable managing Docker/Kubernetes deployments and willing to add missing CI and hardened error handling. Teams seeking an out‑of‑the‑box, fully vetted SaaS alternative should look elsewhere.