llms txt hub - Moses Yebei

The Problem

The llms.txt standard is meant to standardize how AI tools discover and use project documentation, but there is no central, community-maintained registry of implementations. Teams building LLM-powered tools have no single place to find which projects support llms.txt, how they implement it, or what tooling exists to validate and generate these files. Discovery is fragmented across individual READMEs and blog posts.

What This Does

This repository is a directory and tooling hub for llms.txt implementations. The core is apps/web, a Next.js site that lists projects supporting the standard, organized by primary categories (AI/ML, Developer Tools, Data & Analytics) and secondary ones (personal, agency, e-commerce). The list is auto-generated: scripts/ and .github/workflows/update-llms-index.yml crawl registered sites and refresh the markdown between  markers.

Alongside the directory, packages/ contains five self-contained projects: a CLI (packages/cli) that installs llms.txt docs as skills into 35+ AI coding agents, a design system (packages/design-system), analytics, API utilities, and a submission-trust module that scores incoming site registrations. The apps/e2e Playwright suite covers the web app's core flows.

How It Is Wired

Execution starts at the web app's entry points: apps/web/components/mdx/index.tsx renders the markdown list, and apps/web/lib/content-loader.ts fetches the generated data. The packages/design-system/lib/utils.ts is the module hub—40 modules import it, making it the highest-blast-radius file in the graph; any change there ripples across the UI. The CLI's init and install commands are the heaviest importers (12 and 9 imports respectively), with packages/cli/src/lib/lockfile and registry managing state.

Seven modules sit in a circular import dependency, including apps/web/lib/submissions/submission-state.ts and analytics.ts, which complicates refactoring. The scripts/pr-backfill-dry-run.ts file (2,359 lines) handles the backfill of the project list and is the largest single unit.

The GitHub Actions workflows (.github/workflows/) handle list updates, link checking, PR intake, and release automation. The postinstall script in package.json runs on install, which is a supply-chain consideration.

How To Use It

git clone https://github.com/moses-y/llms-txt-hub
cd llms-txt-hub
pnpm install
pnpm dev

Configuration lives in apps/web/.env.example; copy it to .env for local development. The CLI is in packages/cli and can be run with pnpm --filter cli run <command>. The e2e tests require apps/e2e/.env and run via Playwright.

Real-World Use

A team building an AI documentation assistant wants to verify a site supports llms.txt before integrating it. They use the CLI to install the standard as a skill into their coding agent, then check the hub's web app to see which vendors in their stack already have implementations. The submission-trust module scores new registrations, and the auto-generated list keeps the directory current without manual editing.

Code Health & Issues

Static analysis found 66 findings (11 high, 55 medium). Key issues:

High – apps/web/.env.test is tracked despite being in .gitignore; untrack and rotate credentials.
High – Third-party GitHub Actions pinned to tags (lycheeverse/lychee-action@v2.7.0, peter-evans/create-issue-from-file@v6) instead of commit SHAs; a moved tag can leak secrets.
High – CI workflows contain no test command despite 106 test files existing; the green check doesn't execute assertions.
High – 573 duplicated 6-line blocks across 126 files; extract shared helpers.
Medium – No Dependabot/Renovate config across 23 manifests; no dependency vulnerability scan in CI; persist-credentials: false missing on checkout; no pre-commit secret gate.

The repo has a license, lockfile, and CI, but no Dockerfile.

The Bottom Line

A functional directory and tooling suite for the llms.txt ecosystem, with solid test coverage and automation. The main risks are the committed .env.test file and unpinned CI actions—both need immediate attention. Suitable for teams adopting llms.txt who want both a reference index and CLI tooling, but the security hygiene should be fixed before production use.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

3,411 files

TypeScript306
TSX272
JSON56
Markdown22
YAML19
JavaScript19

ReactNext.jsExpressSpringTransformersLangChainVector store

Architecture

Top-level areas of the codebase, sized by module count. Arrows show how many imports cross from one area into another.

graph TD g0["apps<br/>394 modules"] g1["packages<br/>179 modules"] g2["scripts<br/>20 modules"] g3["configs<br/>2 modules"] g2 -->|1| g0 g2 -->|1| g1

Module dependencies

The 10 most depended-upon modules of 597, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_ages_design_system_lib_utils["utils"] n1_packages_cli_src_lib_logger["logger"] n2_packages_cli_src_types_index["index"] n3_ackages_cli_src_lib_lockfile["lockfile"] n4_ackages_cli_src_lib_registry["registry"] n5_ts_forms_submit_form_schemas["submit-form-schemas"] n6_s_submission_trust_src_types["types"] n7_packages_cli_src_lib_storage["storage"] n8_uth_src_client_context_index["index"] n9_tem_components_shadcn_button["button"] n3_ackages_cli_src_lib_lockfile --> n2_packages_cli_src_types_index n3_ackages_cli_src_lib_lockfile --> n1_packages_cli_src_lib_logger n4_ackages_cli_src_lib_registry --> n2_packages_cli_src_types_index n7_packages_cli_src_lib_storage --> n2_packages_cli_src_types_index n9_tem_components_shadcn_button --> n0_ages_design_system_lib_utils

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high4

medium5

low1

Untrack the file your own .gitignore says to ignorehigh

apps/web/.env.test

tracked although .gitignore excludes it

The author already decided this file must never be committed, so its presence is an accident nobody noticed, which means the credentials in it are the ones actually in use.

Fix: git rm --cached the path and rotate; the ignore rule is already correct.

Pin third-party GitHub Actions to a commit SHAhigh3 occurrences

.github/workflows

lycheeverse/lychee-action@v2.7.0, peter-evans/create-issue-from-file@v6, changesets/action@v1

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Remove the committed .env and rotate what it holdshigh

apps/web/.env.test

A tracked .env is the most common route for a working key to reach a public clone, and it is the file the app actually loads, so the value is usually live.

Fix: git rm --cached the file, add it to .gitignore, rotate every credential it names, commit a .env.example with empty values.

Make CI invoke the test suite it hashigh106 occurrences

.github/workflows

106 test files, no test command in the 4 workflows read

A green check that never executed an assertion is worse than no check, because reviewers trust it.

Fix: Add a test step to the existing workflow rather than a new workflow.

Enable Dependabot or Renovatemedium

23 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Add a pre-commit secret gatemedium

a secret-shaped file is tracked and no repo-level gate is visible

Without a gate the same class of leak recurs on the next commit, so a leak finding is the symptom and this is the cause.

Fix: Add a pre-commit hook scanning staged content, and enable push protection.

Set persist-credentials: false on checkoutmedium

.github/workflows/pr-intake.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Review the install lifecycle script and disable scripts in CImedium

package.json

postinstall

Install scripts are the execution vector for every npm compromise from event-stream onward, and a postinstall that fetches a binary makes the build depend on a URL nobody reviews.

Fix: Move the work into an explicit build step, or set ignore-scripts in CI and run it by name.

Set timeout-minutes on the workflow jobslow4 occurrences

.github/workflows/link-checker.yml

4 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.