freellmapi - Moses Yebei

Abstract

OpenAI-compatible proxy that stacks the free tiers of 16 LLM providers (~1.7B tokens/month) behind one /v1 endpoint — plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.

The Problem Sixteen different free‑tier LLM providers each require their own SDK, rate‑limit handling, and error‑mapping. Managing keys, fall‑over, and per‑provider budgets is tedious and error‑prone. FreeLLMAPI collapses all of them behind a single OpenAI‑compatible /v1/chat/completions endpoint so any client library can speak one URL.

What This Does FreeLLMAPI is a proxy built on Express (server) and React/Vite (client). It stores provider keys encrypted in a SQLite database (server/src/db/index.ts) and uses a router (server/src/services/router.ts) to pick the best available model per request, fall‑over on rate‑limit, and track usage against each provider’s free‑tier cap. The router consults the hub module server/src/db/index.ts (45 importers, instability 0.02) for key lookup and usage stats. An oversized proxy handler lives in server/src/routes/proxy.ts (708 lines) and the migration script in server/src/db/migrations.ts (708 lines) – both are high‑churn, high‑blast‑radius files that would benefit from splitting into smaller units. The call graph shows cn (43 call sites), getDb (38), and apply (23) as the most‑touched functions; a change to the database schema ripples through many modules.

How It Is Wired

Entry point: main in server/src/index.ts:12 reaches 87 functions, called from 1 place.
Database hub: initDb → migrateDbSchema → createTables (server/src/db/index.ts) sets up the SQLite schema and performs crypto‑key initialization (server/src/lib/crypto.ts).
Routing flow: routeRequest → isGatedApiPath → getSessionKey → getStickyModel → provider‑specific HTTP call (fetchWithTimeout).
Key operations: getUnifiedApiKey, regenerateUnifiedKey, decrypt all touch the DB (server/src/db/index.ts) and crypto (server/src/lib/crypto.ts).
Blast‑radius modules: server/src/db/index.ts (45 dependents), server/src/app.ts (20 dependents, instability 0.39), and server/src/lib/crypto.ts (13 dependents, instability 0).
External touches: 60 functions read/write the DB; 1 function calls a model for inference; 8 perform crypto/secret ops. No model inference occurs inside the repo; calls exit the process via the provider HTTP layer.

How To Use It

Step	Command / File
Clone	`git clone https://github.com/moses-y/freellmapi`
Install	`cd freellmapi && npm ci` (lockfile‑driven install)
Env	Copy `.env.example` → `.env`; set `UNIFIED_DB_PATH`, provider API keys (e.g., `GOOGLE_API_KEY`, `GROQ_API_KEY`). The unified key is stored encrypted in `server/src/db/index.ts`.
Build/Docker	`docker build -t freellmapi .` (Dockerfile present) or `docker-compose up --build`
Start server	`npm run start` (runs `server/src/index.ts` → `main`)
Start client	`npm run dev` in `client/` (Vite dev server, connects to `http://localhost:3000` by default)
Desktop	`npm run build` then `npm run start` in `desktop/` (Electron, entry `desktop/src/main.ts`)

Real‑World Use A team wants to prototype a chatbot using multiple free models without managing sixteen SDKs. They add keys for Google Gemini, Groq Llama 3, and OpenRouter, point their OpenAI‑compatible client (openai npm package) at http://localhost:3000/v1/chat/completions, and the router automatically selects the cheapest‑available model, falls back on the next provider when rate‑limited, and logs per‑key token usage to stay under each free‑tier cap.

Code Health & Issues (measured findings, not opinion)

[HIGH] Pin GitHub Actions to commit SHAs – .github/workflows/ci.yml uses @v3/@v5 tags for docker/setup-buildx-action, docker/login-action, docker/metadata-action, docker/build-push-action. Tags can move, risking secret exposure. Fix: replace with 40‑character SHAs and let Dependabot bump them.
[MEDIUM] Declare least‑privilege GITHUB_TOKEN permissions – one workflow declares no permissions. Fix: add permissions: contents: read at top and widen per‑job as needed.
[MEDIUM] Enable Dependabot or Renovate – 5 manifests, no update bot configured. Fix: commit .github/dependabot.yml covering npm and GitHub Actions.
[MEDIUM] Install from lockfile in CI – npm install without --ci. Fix: use npm ci (or yarn install --immutable).
[MEDIUM] Gate PRs on dependency vulnerability scan – no scan in CI. Fix: add dependency-review-action on pull_request or osv-scanner on push.
[MEDIUM] Gate checkout on persist-credentials: false – token remains in .git/config for later steps. Fix: add with: persist-credentials: false and pass explicit token only to push steps.
[LOW] Set timeout-minutes on workflow jobs – 2 jobs have no timeout. Fix: add realistic timeout-minutes per job.
[LOW] Add convention files – missing .editorconfig, .gitattributes, formatter config. Fix: add those files per project conventions.

The Bottom Line FreeLLMAPI delivers a pragmatic solution for experimenting with many free LLM tiers behind a single OpenAI‑compatible API. The architecture is clear: a hub DB module, a router, and encrypted key storage. However, the codebase shows several high‑impact maintainability risks—oversized proxy/migration files, untagged GitHub Actions, and missing Dependabot. It’s well‑suited for personal or small‑team prototyping where the free‑tier aggregation outweighs the need for rock‑solid production hardening; for larger deployments, invest in the listed hygiene fixes and consider splitting the proxy and migration logic into smaller, testable units.

What the analyser found

Deployment readiness

7/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

186 files

TypeScript99
TSX25
JSON13
YAML4
Markdown4
HTML3

ReactExpressDockerCUDA

Architecture

Top-level areas of the codebase, sized by module count. Arrows show how many imports cross from one area into another.

graph TD g0["server<br/>85 modules"] g1["client<br/>30 modules"] g2["desktop<br/>11 modules"] g2 -->|5| g0

Module dependencies

The 10 most depended-upon modules of 127, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_server_src_db_index["index"] n1_server_src_app["app"] n2_server_src_lib_crypto["crypto"] n3_r_src___tests___helpers_auth["auth"] n4_server_src_services_router["router"] n5_server_src_providers_index["index"] n6_server_src_providers_base["base"] n7_erver_src_services_ratelimit["ratelimit"] n8_server_src_lib_content["content"] n9_server_src_routes_proxy["proxy"] n1_server_src_app --> n9_server_src_routes_proxy n5_server_src_providers_index --> n6_server_src_providers_base n9_server_src_routes_proxy --> n4_server_src_services_router n9_server_src_routes_proxy --> n7_erver_src_services_ratelimit n9_server_src_routes_proxy --> n0_server_src_db_index n9_server_src_routes_proxy --> n8_server_src_lib_content n7_erver_src_services_ratelimit --> n0_server_src_db_index n4_server_src_services_router --> n0_server_src_db_index n4_server_src_services_router --> n5_server_src_providers_index n4_server_src_services_router --> n2_server_src_lib_crypto n4_server_src_services_router --> n7_erver_src_services_ratelimit n4_server_src_services_router --> n6_server_src_providers_base

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high1

medium5

low2

Pin third-party GitHub Actions to a commit SHAhigh4 occurrences

.github/workflows

docker/setup-buildx-action@v3, docker/login-action@v3, docker/metadata-action@v5, docker/build-push-action@v6

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Declare least-privilege permissions for GITHUB_TOKENmedium

.github/workflows/ci.yml

1 workflow(s) declare no permissions

With no declaration the token inherits the repository default, so any injected step can push commits or mint releases from inside your own CI.

Fix: Add permissions: contents: read at the top of the workflow and widen per job only where needed.

Enable Dependabot or Renovatemedium

5 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Install from the lockfile in CImedium

.github/workflows

npm install with a committed package-lock.json

A fresh resolution in CI means the tested dependency set is not the locked one, so the failure only appears after merge.

Fix: Use npm ci, yarn install --immutable, or pnpm install --frozen-lockfile.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Set persist-credentials: false on checkoutmedium

.github/workflows/ci.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Set timeout-minutes on the workflow jobslow2 occurrences

.github/workflows/ci.yml

2 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Add the repository convention files this project lackslow3 occurrences

missing .editorconfig, .gitattributes, a formatter config

Without them one contributor's editor writes tabs into a Python file, a shell script commits with CRLF and fails in the container, and a notebook diff is unreviewable.

Fix: Add .editorconfig, .gitattributes with text=auto eol=lf, and a formatter config.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.