The Problem
Training LLMs with reinforcement learning requires environments that package a task dataset, a model harness, and a reward function into one coherent unit. Teams typically hand-roll these per project, leading to duplicated infrastructure, inconsistent evaluation logic, and no clean way to share or reuse environments across experiments. Verifiers standardizes this packaging so environments are portable across RL training, evals, and synthetic data generation.
What This Does
Verifiers is a library for building and running LLM environments. Each environment under environments/ is a self-contained package with its own pyproject.toml, a Python module defining the task (e.g., environments/gsm8k/gsm8k.py, environments/mathpython/mathpython.py), and a README. The library also includes a browser-based agent template under assets/templates/browserbase/cua/ with TypeScript source (index.ts, server.ts) and Docker build files, and a set of RL/eval configs under configs/ (e.g., configs/rl/gsm8k.toml, configs/eval/minimal.toml).
The project integrates with Prime Intellect's hosted training platform and the prime-rl trainer. Release notes under assets/release/ show active versioning (v0.1.9 as of Jan 2026) with regular feature additions like trajectory-based rollout tracking and the vf.RLTrainer nano trainer. CI is configured via GitHub Actions (.github/workflows/), and docs live under docs/.
How To Use It
The README gives the setup path:
install uv curl -LsSf https://astral.sh/uv/install.sh | sh install the prime CLI uv tool install prime log in to the Prime Intellect platform prime login set up a new workspace prime lab setup
That command creates a Python project, installs verifiers, and downloads starter configs. For individual environments, each has its own pyproject.toml, so you can install one directly with uv add or pip install -e environments/gsm8k/. Running an environment requires the prime CLI and a valid login; the repo does not document a standalone CLI entry point for a single environment. Config files like configs/rl/gsm8k.toml define training runs, but the exact invocation command is not present in the repo.
Real-World Use
A practical scenario: you want to train a model on math word problems. You install the gsm8k environment, point the trainer at configs/rl/gsm8k.toml, and the environment handles dataset loading, model harness setup, and reward scoring. The same environment can later be reused in an eval config (configs/eval/multi-env.toml) to benchmark the trained model against other tasks without rewriting evaluation logic. The browser CUA template (assets/templates/browserbase/cua/) shows how to package an agent harness with Docker for sandboxed execution.
Code Health & Issues
Med - No lockfiles for Python or Node dependencies - pyproject.toml files and package.json declare dependencies without lockfiles, making builds non-reproducible. This is a real risk for a library meant to be shared. Low - Minimal test coverage - only 4 test files exist across 200 total files. CI is configured (.github/workflows/test.yml), but the test surface is thin for a library with this many environment types. Low - Heuristic issue flagged - assets/templates/browserbase/cua/package.json has no lockfile; the Docker build (Dockerfile.build) may pull non-pinned versions. Low - No license file found - the repo has a LICENSE file, so this is fine; the heuristic analysis did not flag it as missing. Low - .env.example present but no .env - good hygiene; secrets are not committed.
The repo is otherwise organized: clear directory separation, documented release process, and CI for style and tests.
The Bottom Line
Verifiers is a well-structured library for packaging LLM RL environments, with a clear integration path to Prime Intellect's platform and prime-rl. It is best suited to teams already using that ecosystem; standalone use requires the prime CLI and platform login. The lack of lockfiles and thin test coverage are the main concerns. If you are building RL environments for Prime Intellect, this is the right starting point; otherwise, evaluate whether the platform coupling fits your workflow.