transformer explainer

The Problem

Understanding how transformer LLMs like GPT work internally is genuinely hard—the math (attention, embeddings, layer norm) is abstract and buried behind framework APIs. This tool makes it concrete by running a live GPT-2 model in the browser and letting you watch each internal operation as it predicts the next token.

What This Does

An interactive visualization tool that runs a real GPT-2 model in-browser via ONNX. You type text, and it shows, step by step, how embeddings, attention, MLP layers, and softmax produce the next-token probability distribution.

The UI is SvelteKit with Tailwind. Each transformer stage has its own component in src/components/: Attention.svelte, Embedding.svelte, QKV.svelte, Mlp.svelte, LinearSoftmax.svelte, Sampling.svelte, Temperature.svelte. Popover components in src/components/Popovers/ explain each operation, and src/components/textbook/ holds guided tutorial pages. It also ships a research paper (arXiv 2408.04619, IEEE VIS 2024).

How It Is Wired

Execution starts in src/routes/+page.svelte. All state flows through src/store/index.ts, the central Svelte store that coordinates the model, inputs, and UI. The model itself is a chunked ONNX GPT-2 in static/model-v2/gpt2.onnx.part0 through .part62, fetched by src/utils/fetchChunks.js. The Python scripts in src/utils/model/ (model.py, export_to_onnx.py, quantize.py, chunk.py) generated and split that model.

The module graph is small and clean: 25 internal modules, 6 import edges, zero circular dependencies. The most-connected modules are tailwind.config (imported by 3 files) and src/utils/animation.ts (imported by 2, itself imports 1)—animation is the shared utility most components depend on for transitions. No module carries a wide blast radius; the risk is concentrated in the two oversized components instead.

How To Use It

Per the README, prerequisites are Node.js 20+ and NPM 10+. No environment variables or config files are required.

git clone https://github.com/moses-y/transformer-explainer
cd transformer-explainer
npm install
npm run dev

Then open http://localhost:5173.

Real-World Use

This fits in teaching and onboarding: ML courses, conference demos, or a team getting familiar with transformer internals. A lecturer can fork it, add custom example inputs in src/constants/examples/, and demonstrate attention patterns live. It is also a reference implementation for anyone building similar model-visualization tools.

Code Health & Issues

Measured static analysis (not opinion) found 32 findings: 12 high, 20 medium. The high findings are deep nesting (max indentation depth 9 across Attention.svelte, Embedding.svelte, InputForm.svelte) and 243 duplicated 6-line code blocks across 23 files. The medium findings are two oversized files—Sankey.svelte and article/Article.svelte at 645 lines each.

SDLC gaps from the structure: no test files anywhere, no CI configuration, and no dependency-update bot. The repository also carries 63 blobs over 5MB—the ONNX model parts are 10MB each—which bloats every clone and CI checkout. 19 of 39 declared dependencies are behind their current major version, with @sveltejs/adapter-auto and @sveltejs/vite-plugin-svelte four majors behind.

The Bottom Line

A solid, well-documented educational tool that works as described and is easy to run. It is not production code: add tests, a CI build step, and move the ONNX model files to Git LFS before treating it as anything but a demo or teaching aid.

What the analyser found

Deployment readiness

4/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

166 files

Svelte50
TypeScript15
JavaScript10
Python4
JSON3
SCSS2

ReactSvelteTailwind

Module dependencies

The 10 most depended-upon modules of 25, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_tailwind_config["tailwind.config"] n1_src_utils_animation["animation"] n2_src_utils_array["array"] n3_src_utils_data["data"] n4_src_constants_gradient["gradient"] n5_src_store_index["index"] n6_src_utils_textbookPages["textbookPages"] n7__eslintrc[".eslintrc"] n8_postcss_config["postcss.config"] n9_src_constants_examples_index["index"] n4_src_constants_gradient --> n0_tailwind_config n5_src_store_index --> n0_tailwind_config n1_src_utils_animation --> n0_tailwind_config n3_src_utils_data --> n2_src_utils_array n3_src_utils_data --> n1_src_utils_animation n6_src_utils_textbookPages --> n1_src_utils_animation

Dependencies

19 of 39 declared dependencies are a major version behind. Resolved 39 of 39 against the registry.

Package	Declared	Latest	Behind
`@sveltejs/adapter-auto`	^3.2.0	7.0.1	4 major
`@sveltejs/vite-plugin-svelte`	^3.0.0	7.3.0	4 major
`vite`	^5.0.3	8.2.1	3 major
`bignumber.js`	^9.1.2	11.1.5	2 major
`eslint`	^8.56.0	10.8.1	2 major
`flowbite`	^2.3.0	4.0.2	2 major
`flowbite-svelte-icons`	^1.6.1	3.1.0	2 major
`typescript`	^5.0.0	7.0.2	2 major

Findings

MediumNo test files detected - untested code paths repository-wide
MediumNo CI/CD pipeline detected - no automated build/test gate .github/ or CI config

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high2

medium2

Add a test suite; this repository has nonehigh74 occurrences

74 source files, no test files

Any change ships with no signal that existing behaviour still holds, so a regression reaches production undetected.

Fix: Add one test per public entry point, then a CI step that runs them.

Add a workflow that builds and tests this repositoryhigh

29 source files, no CI configuration

Every change merges with nobody having run the build once, which across this estate is 499 repositories.

Fix: Add a workflow running the project build and test command on push and pull_request.

Enable Dependabot or Renovatemedium

1 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Move large binaries to Git LFS or out of the repositorymedium63 occurrences

static/model-v2/gpt2.onnx.part0

63 blobs over 5MB: gpt2.onnx.part0 10.0MB, gpt2.onnx.part1 10.0MB, gpt2.onnx.part10 10.0MB

One repository here carries twenty blobs over 5MB including a 10.9MB spreadsheet, so every clone and every CI checkout pays for data nobody diffs.

Fix: Track those extensions with LFS, or move datasets to object storage and fetch them in a setup step.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.