sourcebot - Moses Yebei

The Problem Large, heterogeneous codebases make it hard for engineers and AI assistants to locate definitions, understand cross‑repo dependencies, and answer ad‑hoc questions reliably. Manual search and scattered documentation increase onboarding time and error rates.

What This Does sourcebot is a self‑hosted service that indexes any Git host, provides AI‑driven Q&A, full‑text code search, and IDE‑style navigation. The UI lives in packages/web, the indexing engine in packages/backend, and the query language compiler in packages/queryLanguage. Configuration is JSON‑based (configs/*.json) and the whole stack can be launched with Docker Compose.

How It Is Wired

Entry point – packages/backend/src/index.ts starts an Express server, loads packages/backend/src/utils.ts (a hub module used by 15 other packages), and registers routes for indexing and query handling.
Indexing – packages/db/src/index.ts creates a PostgreSQL connection, then packages/db/tools/scriptRunner runs migration scripts. The backend calls packages/backend/src/bitbucket.ts (the largest file, 655 LOC) to fetch repo data.
Search & Query – HTTP requests hit /api/search → packages/web/src/proto/webserver (imports 46 other modules). Queries are parsed by packages/queryLanguage/src/index.ts, which builds a protobuf request sent to the web‑server proto.
Chat / Ask – UI components in packages/web/src/app/(app)/@sidebar/components/defaultSidebar invoke /api/ask, which routes through packages/web/src/features/chat/types.ts (a “hub” module with 16 dependents) into the backend’s LLM provider defined in configs/auth.json.
Circular dependencies – The import graph shows 28 cycles, notably between packages/backend/src/utils.ts and packages/web/src/proto/zoekt/webserver/v1/Q.ts. These cycles increase blast radius when changing shared types.

All external effects (DB writes, network calls to code hosts, LLM API calls) originate from the backend utils and the query‑language compiler; the web layer is thin and only forwards HTTP payloads.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/sourcebot
cd sourcebot

# Install dependencies (yarn workspace)
yarn install

# Build containers
docker compose -f docker-compose.yml up -d

# Create a config (example)
cat > config.json <<'EOF'
{
  "$schema": "https://raw.githubusercontent.com/sourcebot-dev/sourcebot/main/schemas/v3/index.json",
  "repos": [{ "url": "https://github.com/your-org/your-repo.git" }],
  "llmProvider": { "type": "openai", "apiKey": "<redacted>" }
}
EOF

# Start the service (backend + web)
yarn workspace @sourcebot/backend start
yarn workspace @sourcebot/web dev

The server listens on http://localhost:3000; the UI is reachable at the same address. Adjust docker-compose-dev.yml if you need custom ports or a separate DB instance.

Real‑World Use A team adds a new microservice repository to config.json. After restarting the containers, Sourcebot automatically indexes the code, making its types and routes searchable. A developer can ask, “How does the createUser handler validate email?” and receives a citation‑rich answer pulling directly from packages/backend/src/utils.ts and the OpenAPI spec in docs/api-reference/sourcebot-public.openapi.json.

Code Health & Issues

Static analysis (411 findings):

High / cognitive_load – deep nesting (e.g., packages/backend/src/utils.ts, max depth 8).
High / soundness – import cycles (e.g., packages/backend/src/utils.ts ↔ packages/web/src/proto/zoekt/webserver/v1/Q.ts).
Medium / cognitive_load – oversized file (packages/backend/src/bitbucket.ts, 655 lines).
Medium / cognitive_load – high branching density (packages/web/src/ee/features/chat/mcp/externalMcpError.ts).
Medium / clarity – hub modules (packages/web/src/features/chat/types.ts, packages/backend/src/utils.ts).

Code‑health audit (8 findings, already prioritized):

HIGH – Pin GitHub Actions to commit SHAs (.github/workflows/*).
HIGH – Remove committed .env.development and rotate credentials.
HIGH – Add a test step to the existing CI workflows.
MEDIUM – Enable Dependabot (.github/dependabot.yml).
MEDIUM – Pin dev‑container base image by digest.
MEDIUM – Add a dependency‑vulnerability scan gate in CI.
MEDIUM – Install a pre‑commit secret‑scan hook.
LOW – Set explicit timeout-minutes on workflow jobs.

The Bottom Line Sourcebot delivers a functional, Docker‑ready code‑search and AI‑Q&A platform with a clear separation between backend indexing and a React/Next UI. The codebase is large and contains several high‑impact cycles and deep nesting that will require refactoring for long‑term maintainability. Teams comfortable managing Docker, Yarn workspaces, and secret rotation can adopt it quickly, but should address the listed security and architectural risks before production use.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

1,547 files

TypeScript619
TSX439
SQL87
JSON46
YAML30
Markdown23

ReactNext.jsExpressTailwindDocker

Module dependencies

The 10 most depended-upon modules of 1,068, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0__web_src_features_chat_types["types"] n1_packages_backend_src_utils["utils"] n2_app__app__browse_hooks_utils["utils"] n3_pp__components_banners_types["types"] n4_components_settingsContainer["settingsContainer"] n5_web_src_features_tools_types["types"] n6_c_proto_zoekt_webserver_v1_Q["Q"] n7_s_web_src_features_git_utils["utils"] n8_packages_backend_src_types["types"] n9_ckages_setupWizard_src_utils["utils"] n1_packages_backend_src_utils --> n8_packages_backend_src_types classDef cyc stroke:#ef4444,stroke-width:2px; class n1_packages_backend_src_utils,n6_c_proto_zoekt_webserver_v1_Q cyc;

Dependencies

20 of 48 declared dependencies are a major version behind. Resolved 45 of 48 against the registry.

Package	Declared	Latest	Behind
`@types/node`	^22.7.5	26.2.0	4 major
`cross-env`	^7.0.3	10.1.0	3 major
`dotenv-cli`	^8.0.0	11.0.0	3 major
`@gitbeaker/rest`	^40.5.1	43.8.0	3 major
`azure-devops-node-api`	^15.1.1	17.0.0	2 major
`glob`	^11.1.0	13.0.6	2 major
`typescript`	^5.6.2	7.0.2	2 major
`concurrently`	^9.2.1	10.0.5	1 major

Findings

LowSecret-shaped paths present; the code-health audit confirms or clears them .env.development, public.pem

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high3

medium4

low1

Pin third-party GitHub Actions to a commit SHAhigh7 occurrences

.github/workflows

aws-actions/configure-aws-credentials@v4, aws-actions/amazon-ecr-login@v2, docker/metadata-action@v5, docker/setup-buildx-action@v3

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Remove the committed .env and rotate what it holdshigh

.env.development

A tracked .env is the most common route for a working key to reach a public clone, and it is the file the app actually loads, so the value is usually live.

Fix: git rm --cached the file, add it to .gitignore, rotate every credential it names, commit a .env.example with empty values.

Make CI invoke the test suite it hashigh131 occurrences

.github/workflows

131 test files, no test command in the 2 workflows read

A green check that never executed an assertion is worse than no check, because reviewers trust it.

Fix: Add a test step to the existing workflow rather than a new workflow.

Enable Dependabot or Renovatemedium

8 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Pin the container base image by digestmedium

.devcontainer/Dockerfile

mcr.microsoft.com/devcontainers/typescript-node:24

An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.

Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Add a pre-commit secret gatemedium

a secret-shaped file is tracked and no repo-level gate is visible

Without a gate the same class of leak recurs on the next commit, so a leak finding is the symptom and this is the cause.

Fix: Add a pre-commit hook scanning staged content, and enable push protection.

Set timeout-minutes on the workflow jobslow2 occurrences

.github/workflows/_build-cloud.yml

2 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.