llm wiki

The Problem

Most document-based AI tools use RAG: retrieve relevant chunks, then answer from scratch on every query. That approach re-derives knowledge repeatedly, wastes tokens, and produces inconsistent answers. LLM Wiki instead compiles documents into a persistent, interlinked wiki once, then keeps it current incrementally—so queries run against a maintained knowledge base rather than raw source material.

What This Does

LLM Wiki is a cross-platform desktop app (Tauri + React) that ingests documents, uses an LLM to generate structured wiki pages with [[wikilink]] cross-references, and maintains that wiki over time. Key features include a two-step chain-of-thought ingest pipeline with caching, a knowledge graph with community detection, vector search via LanceDB, and a Chrome web clipper.

The architecture follows Karpathy's LLM Wiki pattern: raw sources (immutable) → wiki (LLM-generated) → schema (rules/config). The wiki directory is Obsidian-compatible. The frontend lives in src/ (React/TypeScript), the backend in src-tauri/ (Rust commands), and the browser extension in extension/.

How It Is Wired

Execution starts in src/App.tsx at init (line 85), which reaches 35 functions. From there, handleOpenProject reaches 25 functions. The core file operations route through src/commands/fs.ts, which defines readFile, writeFile, listDirectory, and is called from 30 files—the widest blast radius in the repo. src/lib/path-utils.ts follows with normalizePath, called from 59 places.

The critical path for file I/O is short: init → openProject → ensureProjectId → readFile (filesystem via fs.readFile). Project switching flows through handleSwitchProject → resetProjectState → pauseQueue → saveQueue → writeFile (filesystem via fs.mkdir). The ingest pipeline lives in src/lib/ingest.ts (14 functions, called from 5 files), with autoIngestImpl calling tryReadFile 5 times. The queue system in src/lib/ingest-queue.ts handles serial processing with crash recovery.

The module graph shows no circular dependencies across 166 internal modules. However, src/lib/llm-client.ts (Ca 9) and src/lib/ingest.ts (Ca 9) are hubs—changes to their interfaces ripple widely. src/components/settings/settings-view.tsx imports 7 modules but is imported by none, making it a leaf with no downstream risk.

How To Use It

Setup:

git clone https://github.com/moses-y/llm_wiki
cd llm_wiki
npm install
npm run tauri dev

Configuration: LLM provider settings live in src/components/settings/sections/llm-provider-section.tsx and src/lib/llm-providers.ts. The app supports OpenAI-compatible endpoints and Claude Code CLI. Embedding configuration is in src/lib/embedding.ts. No environment variables are required; settings are stored per-project.

Running: The Tauri app builds from src-tauri/. The Chrome extension in extension/ loads unpacked from extension/manifest.json.

Real-World Use

A researcher imports a folder of PDFs and markdown notes. The ingest pipeline (src/lib/ingest.ts) reads each file, the LLM generates wiki pages with source traceability, and src/lib/enrich-wikilinks.ts adds cross-references. The knowledge graph (src/components/graph/graph-view.tsx) reveals clusters via Louvain detection. When the researcher adds new documents, the incremental cache in src/lib/ingest-cache.ts ensures only changed content is re-processed.

Code Health & Issues

Static analysis found 33 findings (5 high, 27 medium, 1 low). Key issues:

• High - Oversized files - src/lib/ingest.ts (769 lines), extension/Readability.js, src/components/chat/chat-message.tsx. Split by responsibility.
• High - Deep nesting - src-tauri/src/clip_server.rs, src-tauri/src/commands/fs.rs, src/components/graph/graph-view.tsx (max depth 10). Use early returns.
• High - Duplicated code - 90 repeated 6-line blocks across 33 files, including chat-message.tsx and file-preview.tsx. Extract shared helpers.
• Medium - Empty catch blocks - extension/Turndown.js silently discards errors.
• Medium - High branching - src/lib/sweep-reviews.ts (125 branches over 321 lines). Consider strategy dispatch.

SDLC observations: CI uses unpinned GitHub Actions (dtolnay/rust-toolchain@stable, tauri-apps/tauri-action@v0)—pin to commit SHAs. Tokio and zip have published advisories; upgrade. No Dependabot configured. Three PDFium binaries (6.7–7.2MB each) bloat the repo; use LFS. CI runs npm install instead of npm ci.

The Bottom Line

LLM Wiki is a well-architected implementation of a genuinely different approach to knowledge management—compiling rather than retrieving. The test suite (64 files) and clean module graph suggest disciplined development. The main risks are maintainability (oversized files, duplication) and supply-chain hygiene (unpinned actions, outdated dependencies). Worth using if you want a self-hosted, Obsidian-compatible alternative to RAG-based tools.