The Problem

Deploying a 135 M‑parameter transformer on a machine without a heavy runtime is hard. Existing solutions require Python, PyTorch, or a compiled library, all of which add binaries, dependencies, and security surface. A user who has only the SQLite CLI and wants to generate text locally would need to write or port an entire inference engine.

What This Does

smollm-135m-sql implements the full inference pipeline inside SQLite. Tokenization, embedding, transformer layers, KV cache, and greedy decoding are expressed as SQL views, triggers, and statements. The repo contains a database schema and a sequence of SQL files that build the model from HuggingFace safetensors and run inference without any host language code.

Key files:

  • outputs/setup.sql creates tables and loads the safetensors/JSON files.
  • outputs/accelerate.sql converts 32‑bit float weights to SQLite REAL values and indexes them for fast access.
  • outputs/stream.sql drives a greedy generation loop, emitting decoded text via SELECT statements that the SQLite CLI prints line‑by‑line.

A lightweight web UI is also bundled: the dashboard/ folder contains SQLPage templates that expose the same inference logic through a browser.

How It Is Wired

Entry Point – sqlite3 outputs/smollm.db < outputs/setup.sql The setup.sql script creates the smollm schema, loads the model.safetensors, tokenizer.json, and config.json files, and builds helper tables such as tokenizer and kv_cache.

Weight Conversion – sqlite3 outputs/smollm.db < outputs/accelerate.sql This file reads the raw weight blobs from 01_weights.sql, splits them into REAL columns, and creates indexes on each layer’s weight matrix.

Inference Loop – sqlite3 outputs/smollm.db < outputs/stream.sql The loop iterates through the following sequence:

  1. 10_prompt.sql – embeds the prompt token IDs.
  2. 07_forward.sql – runs the transformer layers, updating kv_cache.
  3. 09_decode.sql – applies softmax and selects the next token greedily.
  4. 12_generated_text.sql – decodes the token to Unicode using 03a_tokenizer_asset.sql and writes it to the output.

Each step is a single SQL statement or view that touches only the database; no external calls are made. The entire flow is linear and stateless across sessions, except for the KV cache which is reset by 15_reset.sql.

Dashboard Wiring – dashboard/index.sql and related files expose the same views to SQLPage. When a user submits a prompt, SQLPage invokes run.sql which internally calls the same 07_forward and 12_generated_text logic, updating the web page in real time.

How To Use It

Setup

git clone https://github.com/moses-y/smollm-135m-sql
cd smollm-135m-sql
curl -L --fail --output outputs/model.safetensors \
  https://huggingface.co/HuggingFaceTB/SmolLM-135M/resolve/main/model.safetensors
curl -L --fail --output outputs/tokenizer.json \
  https://huggingface.co/HuggingFaceTB/SmolLM-135M/resolve/main/tokenizer.json
curl -L --fail --output outputs/config.json \
  https://huggingface.co/HuggingFaceTB/SmolLM-135M/resolve/main/config.json
curl -L --fail --output outputs/UnicodeData.txt \
  https://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt

Build the database:

sqlite3 outputs/smollm.db < outputs/setup.sql
sqlite3 outputs/smollm.db < outputs/accelerate.sql

Run a Generation

sqlite3 outputs/smollm.db < outputs/stream.sql

Edit the prompt inside outputs/stream.sql to change input.

Launch the Web UI

brew install sqlpage        # or download the binary
sqlpage --config-dir dashboard_config

Open http://127.0.0.1:8080.

Real‑World Use

A CI/CD pipeline can pre‑build the smollm.db artifact and ship it to edge devices that only run sqlite3. The device can then stream text from the database directly, using a web UI (via SQLPage) or a custom front‑end that executes the same SQL scripts.

Code Health & Issues

  • Medium / SDLC – No tests – no test_*.sql or similar.
  • Medium / SDLC – No CI – repository lacks .github or other CI config.
  • Medium / SDLC – No license – unclear redistribution rights.
  • Low – No documentation beyond README – minimal inline comments in SQL.

The Bottom Line

smollm-135m-sql demonstrates that a medium‑size transformer can run entirely inside SQLite, eliminating external runtimes. The approach is straightforward to deploy on any machine with the SQLite CLI, but the lack of tests, CI, and license limits reuse in production. Engineers who need a lightweight, container‑free inference engine will find this a useful reference, provided they are comfortable debugging raw SQL.