The Problem

Training a GPT model typically means pulling in PyTorch, CUDA, and a multi-gigabyte framework stack. For anyone who wants to understand what a transformer actually does under the hood—or needs a minimal inference engine for an embedded or resource-constrained target—the dependency overhead is the real barrier. microgpt-c removes that barrier entirely: a complete character-level transformer, from forward pass to backprop to sampling, in a single C file with nothing beyond libc.

What This Does

src/microgpt.c is the entire implementation—1228 lines of C that train a 4192-parameter character-level GPT on a text corpus and generate new text from it. The Makefile handles the build, selecting NEON (ARM64) or AVX2 (x86-64) vectorization flags for the host. The data/names.txt file is the default training corpus (~32k names), and docs/PERFORMANCE.md documents the inference optimizations and measured throughput.

Training runs in seconds on a laptop and the model demonstrably generalizes rather than memorizes: at 2.2054 nats/char on held-out data, it beats an interpolated trigram with nearly five times the parameters.

How It Is Wired

Execution starts at main in src/microgpt.c. The flow is straightforward: read the corpus, train for 20,000 steps (Adam optimizer, backprop through the transformer), then run inference to sample new names. Training uses gpt_forward which stores activations for backprop; inference uses the specialized gpt_forward_infer single-token path designed for throughput. The two forward passes produce logits that match to within fp32 rounding.

There is no module graph to speak of—one file, one translation unit, zero internal dependencies. The only external touch is libc for file I/O (fopen, fread) and printf. The entire program is self-contained; there is no database, network, or external service involved.

File-by-file responsibility:

  • src/microgpt.c — the whole model: tokenizer, transformer, training loop, inference, sampling, CLI.
  • Makefile — build orchestration and host-specific vectorization flags.
  • data/names.txt — default training corpus.
  • docs/PERFORMANCE.md — inference-path documentation and benchmark numbers.
  • README.md — usage, build instructions, and training results.

How To Use It

Setup: Requires only a C compiler and make. No dependencies to install.

Build and run (verbatim from the README):

make run

Or run directly on any corpus with one item per line:

./microgpt data/names.txt

Configuration: None. The model architecture, training steps, and hyperparameters are hardcoded in src/microgpt.c. Corpus is passed as the single CLI argument.

Real-World Use

This fits where a full ML stack is overkill: teaching transformer internals, generating names or short tokens in a mobile or embedded app, or as a reference implementation to port to another language. The single-file design means you can drop microgpt.c into any C project and have a working character-level generator with no external dependencies.

Code Health & Issues

Static analysis of src/microgpt.c reports two high-severity findings:

  • High — Deep nesting: Max indentation depth is 9 in src/microgpt.c, making control flow hard to follow. Fix: early returns and guard clauses.
  • High — Oversized file: 1228 code lines in one file. Fix: split into cohesive units by responsibility.

SDLC observations from the repo structure:

  • Med — No tests: No test files detected; the training/inference paths are unverified beyond the README's reported results.
  • Med — No CI: No CI/CD configuration; no automated build or test gate.
  • Low — Missing lockfile: No dependency lockfile, though with zero dependencies this is largely moot.
  • Good: License present, no committed secrets detected.

The Bottom Line

This is a clean, well-documented reference implementation that delivers exactly what it promises: a dependency-free GPT in one C file. The single-file design is the point—it maximizes readability and portability at the cost of testability and maintainability. Use it to learn transformer internals, as a porting reference, or for a minimal embedded generator. Don't use it as the foundation of a production NLP system.