litgpt - Moses Yebei

The Problem

Training or fine-tuning LLMs typically means assembling a fragile stack: model code, tokenizer, data pipeline, distributed strategy, and checkpoint conversion scripts that must all agree. LitGPT removes that glue by shipping from-scratch implementations of 20+ models with recipes for pretraining, fine-tuning (LoRA, QLoRA, Adapter), and deployment in one package.

What This Does

This is a fork of Lightning-AI/litgpt (13,621 stars upstream). The core package lives in litgpt/ with model definitions in model.py, fine-tuning variants in adapter.py, adapter_v2.py, and lora.py, and a high-level LLM API in api.py. The config_hub/ directory holds 47 YAML recipes covering model-specific fine-tuning configurations.

The repo also includes extensions/ for Thunder and XLA backends, plus a deploy/serve.py for running an inference server. Tutorials and 64 test files round out the package.

How It Is Wired

Execution starts at main in litgpt/__main__.py:65, which reaches 192 functions. The call graph shows fit is the workhorse, calling gradient_accumulation_iters 23 times and validate 21 times. The LLM class in api.py is the primary user-facing entry point, with from_name called from 88 places and GPT from 70.

The dependency graph is hub-and-spoke: litgpt/utils.py is imported by 58 modules and participates in a circular import cycle. litgpt/data/__init__.py (32 importers) and litgpt/__init__.py (39 importers) are also in cycles. This makes changes to utils.py high-blast-radius—touching it risks breaking a large portion of the codebase.

A run leaves the process through setup (filesystem via out_dir.mkdir), predict (model inference via self.llm.generate), and run_server (subprocess via subprocess.Popen). The system reads/writes files in 48 functions and makes 3 outbound network calls.

How To Use It

git clone https://github.com/moses-y/litgpt
cd litgpt
pip install -e .

Basic inference:

from litgpt import LLM

llm = LLM.load("microsoft/phi-2")
text = llm.generate("Fix the spelling: Every fall, the family goes to the mountains.")

Fine-tuning recipes are YAML files in config_hub/finetune/—for example, config_hub/finetune/llama-3-8b/lora.yaml. The litgpt/__main__.py CLI dispatches to fine-tune, pretrain, and generate subcommands.

Real-World Use

A typical workflow: download a pretrained checkpoint, apply LoRA fine-tuning on a domain dataset using a recipe from config_hub/finetune/, then deploy via litgpt/deploy/serve.py. The LLM API abstracts model loading and generation, while the underlying lora.py and adapter.py modules handle parameter-efficient fine-tuning.

Code Health & Issues

Static analysis found 68 issues: 38 high, 28 medium, 2 low. Key findings:

High — Import cycles: 27 modules participate in circular dependencies (litgpt/utils.py, litgpt/__init__.py, litgpt/data/__init__.py).
High — Deep nesting: max indentation depth of 7 in litgpt/utils.py, litgpt/tokenizer.py, litgpt/prompts.py.
High — Oversized files: utils.py has 858 code lines; config.py and model.py are similarly large.
High — Hub modules: 58 modules depend on utils.py, making it high-blast-radius for changes.
High — Duplicated code: 1,464 repeated 6-line blocks across 74 files.
Medium — Broad exception handling in litgpt/utils.py, data/prepare_starcoder.py, and tests/test_cli.py.
Medium — Files opened without context managers in eval/evaluate.py and data/prepare_slimpajama.py.

Security audit found one critical issue: .github/workflows/cpu-tests.yml exposes HF_TOKEN to contributor-triggered workflows—a PR could exfiltrate credentials. No lockfile exists (pyproject.toml only), so builds aren't reproducible. CI lacks dependency vulnerability scanning.

The Bottom Line

A solid, well-tested LLM training/inference toolkit with good model coverage and recipes. The codebase works but has real maintainability debt: circular imports, oversized hub modules, and no lockfile. Forking from a mature upstream project means the core is proven; the fork itself adds no visible changes. Suitable for teams needing a self-contained LLM fine-tuning stack who can tolerate the coupling in utils.py.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

252 files

Python137
YAML55
Markdown31
JSON5
TOML1

DockerPyTorchTransformers

Architecture

Top-level areas of the codebase, sized by module count. Arrows show how many imports cross from one area into another.

graph TD g0["litgpt<br/>60 modules"] g1["tests<br/>57 modules"] g2["extensions<br/>17 modules"] g3["tutorials<br/>3 modules"] g1 -->|145| g0 g2 -->|28| g0 g3 -->|9| g0 g1 -->|4| g2

Module dependencies

The 10 most depended-upon modules of 137, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_litgpt_utils["utils"] n1_litgpt___init__["__init__"] n2_litgpt_data___init__["__init__"] n3_litgpt_tokenizer["tokenizer"] n4_litgpt_prompts["prompts"] n5_litgpt_constants["constants"] n6_litgpt_model["model"] n7_litgpt_config["config"] n8_litgpt_generate_base["base"] n9_litgpt_args["args"] n1_litgpt___init__ --> n7_litgpt_config n1_litgpt___init__ --> n6_litgpt_model n1_litgpt___init__ --> n4_litgpt_prompts n1_litgpt___init__ --> n3_litgpt_tokenizer n1_litgpt___init__ --> n0_litgpt_utils n7_litgpt_config --> n6_litgpt_model n8_litgpt_generate_base --> n7_litgpt_config n8_litgpt_generate_base --> n5_litgpt_constants n8_litgpt_generate_base --> n6_litgpt_model n8_litgpt_generate_base --> n4_litgpt_prompts n8_litgpt_generate_base --> n3_litgpt_tokenizer n8_litgpt_generate_base --> n0_litgpt_utils n6_litgpt_model --> n7_litgpt_config n4_litgpt_prompts --> n7_litgpt_config n4_litgpt_prompts --> n1_litgpt___init__ n3_litgpt_tokenizer --> n0_litgpt_utils n0_litgpt_utils --> n5_litgpt_constants n0_litgpt_utils --> n1_litgpt___init__ classDef cyc stroke:#ef4444,stroke-width:2px; class n0_litgpt_utils,n1_litgpt___init__,n2_litgpt_data___init__,n3_litgpt_tokenizer,n4_litgpt_prompts,n6_litgpt_model,n7_litgpt_config,n8_litgpt_generate_base cyc;

Dependencies

1 of 8 declared dependencies are a major version behind. Resolved 8 of 8 against the registry.

Package	Declared	Latest	Behind
`huggingface-hub`	>=0.30,<1.4	1.27.0	1 major

Findings

LowDependencies declared without a lockfile - non-reproducible builds pyproject.toml

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

critical1

high1

medium5

low1

Keep secrets out of workflows a fork can triggercritical

.github/workflows/cpu-tests.yml

HF_TOKEN (publish or cloud scope)

Contributor-controlled input reaches a job holding a publish token or cloud credential, so a pull request becomes credential exfiltration.

Fix: Move secret-using steps into a workflow_run job that never checks out pull request code, or gate on an environment with required reviewers.

Commit a lockfile beside the manifesthigh

pyproject.toml

manifest with no lockfile in scope

An unlocked range means the artifact you tested and the artifact you ship can contain different transitive code, so a malicious patch release reaches production with no diff.

Fix: Run the package manager once and commit the generated lockfile.

Declare least-privilege permissions for GITHUB_TOKENmedium

.github/workflows/check-links.yml

1 workflow(s) declare no permissions

With no declaration the token inherits the repository default, so any injected step can push commits or mint releases from inside your own CI.

Fix: Add permissions: contents: read at the top of the workflow and widen per job only where needed.

Pin the container base image by digestmedium

.devcontainer/Dockerfile

mcr.microsoft.com/devcontainers/python:1-${VARIANT}

An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.

Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Set persist-credentials: false on checkoutmedium

.github/workflows/check-links.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Pin the resolution so a vulnerable version cannot installmedium15 occurrences

no lockfile, and the declared ranges reach torch@2.7 CVE-2025-55551 (high) +14 more

The range includes versions with published advisories and nothing in the repository records which one was resolved, so two installs a month apart can differ in whether they are vulnerable and neither is reproducible from what is committed.

Fix: Commit the lockfile, which fixes the resolution and makes this answerable.

Set timeout-minutes on the workflow jobslow2 occurrences

.github/workflows/check-links.yml

2 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.