The Problem
Deploying a 135 M‑parameter transformer on a machine without a heavy runtime is hard. Existing solutions require Python, PyTorch, or a compiled library, all of which add binaries, dependencies, and security surface. A user who has only the SQLite CLI and wants to generate text locally would need to write or port an entire inference engine.
What This Does
smollm-135m-sql implements the full inference pipeline inside SQLite. Tokenization, embedding, transformer layers, KV cache, and greedy decoding are expressed as SQL views, triggers, and statements. The repo contains a database schema and a sequence of SQL files that build the model from HuggingFace safetensors and run inference without any host language code.
Key files:
outputs/setup.sqlcreates tables and loads the safetensors/JSON files.outputs/accelerate.sqlconverts 32‑bit float weights to SQLiteREALvalues and indexes them for fast access.outputs/stream.sqldrives a greedy generation loop, emitting decoded text viaSELECTstatements that the SQLite CLI prints line‑by‑line.
A lightweight web UI is also bundled: the dashboard/ folder contains SQLPage templates that expose the same inference logic through a browser.
How It Is Wired
Entry Point – sqlite3 outputs/smollm.db < outputs/setup.sql The setup.sql script creates the smollm schema, loads the model.safetensors, tokenizer.json, and config.json files, and builds helper tables such as tokenizer and kv_cache.
Weight Conversion – sqlite3 outputs/smollm.db < outputs/accelerate.sql This file reads the raw weight blobs from 01_weights.sql, splits them into REAL columns, and creates indexes on each layer’s weight matrix.
Inference Loop – sqlite3 outputs/smollm.db < outputs/stream.sql The loop iterates through the following sequence:
10_prompt.sql– embeds the prompt token IDs.07_forward.sql– runs the transformer layers, updatingkv_cache.09_decode.sql– applies softmax and selects the next token greedily.12_generated_text.sql– decodes the token to Unicode using03a_tokenizer_asset.sqland writes it to the output.
Each step is a single SQL statement or view that touches only the database; no external calls are made. The entire flow is linear and stateless across sessions, except for the KV cache which is reset by 15_reset.sql.
Dashboard Wiring – dashboard/index.sql and related files expose the same views to SQLPage. When a user submits a prompt, SQLPage invokes run.sql which internally calls the same 07_forward and 12_generated_text logic, updating the web page in real time.
How To Use It
Setup
git clone https://github.com/moses-y/smollm-135m-sql
cd smollm-135m-sql
curl -L --fail --output outputs/model.safetensors \
https://huggingface.co/HuggingFaceTB/SmolLM-135M/resolve/main/model.safetensors
curl -L --fail --output outputs/tokenizer.json \
https://huggingface.co/HuggingFaceTB/SmolLM-135M/resolve/main/tokenizer.json
curl -L --fail --output outputs/config.json \
https://huggingface.co/HuggingFaceTB/SmolLM-135M/resolve/main/config.json
curl -L --fail --output outputs/UnicodeData.txt \
https://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt
Build the database:
sqlite3 outputs/smollm.db < outputs/setup.sql
sqlite3 outputs/smollm.db < outputs/accelerate.sql
Run a Generation
sqlite3 outputs/smollm.db < outputs/stream.sql
Edit the prompt inside outputs/stream.sql to change input.
Launch the Web UI
brew install sqlpage # or download the binary
sqlpage --config-dir dashboard_config
Open http://127.0.0.1:8080.
Real‑World Use
A CI/CD pipeline can pre‑build the smollm.db artifact and ship it to edge devices that only run sqlite3. The device can then stream text from the database directly, using a web UI (via SQLPage) or a custom front‑end that executes the same SQL scripts.
Code Health & Issues
- Medium / SDLC – No tests – no
test_*.sqlor similar. - Medium / SDLC – No CI – repository lacks
.githubor other CI config. - Medium / SDLC – No license – unclear redistribution rights.
- Low – No documentation beyond README – minimal inline comments in SQL.
The Bottom Line
smollm-135m-sql demonstrates that a medium‑size transformer can run entirely inside SQLite, eliminating external runtimes. The approach is straightforward to deploy on any machine with the SQLite CLI, but the lack of tests, CI, and license limits reuse in production. Engineers who need a lightweight, container‑free inference engine will find this a useful reference, provided they are comfortable debugging raw SQL.