omni - Moses Yebei

The Problem

Enterprises need a single place to search across heterogeneous workplace tools (Google Workspace, Slack, Confluence, Jira, file systems, etc.) while keeping data on‑premises and respecting source‑system permissions. Existing solutions either require cloud SaaS or force separate search stacks for each source.

What This Does

omni bundles a search backend, AI agent, and web UI into a self‑hosted platform. Core services live under services/ (Rust indexer, searcher, connector‑manager) and expose HTTP APIs used by the SvelteKit UI in web/. Connectors for each external system are isolated Rust crates in connectors/ (e.g., connectors/google/, connectors/atlassian/) each with its own Dockerfile. Shared utilities (DB models, content storage, SDK client) are in shared/. The sdk/ folder provides a TypeScript wrapper (sdk/typescript/src/index.ts) for external callers.

How It Is Wired

Execution begins in benchmarks/src/main.rs (line 65) – the only main function the static analysis could resolve. It calls run_server, which:

Loads configuration via from_env (reads .env.example variables).
Creates a Redis client (RedisClient::open) – this is the first external resource access (filesystem‑based Redis socket).
Starts the HTTP server (run_server) that routes to the Rust search service.

From the request path, the server eventually calls create_sync_run (≈20 distinct callers) which inserts a record into Postgres (shared/src/db/repositories/document.rs::insert). That function is a hub: 13 modules import web/src/lib/server/db/index.ts, and shared/src/embedding_queue.rs is the most‑used internal file (15 functions, called from 53 places, also writes to the DB).

Connector containers are built from the Dockerfiles under connectors/*/Dockerfile. When a sync job runs, the connector‑manager (services/connector-manager/src/scheduler.rs::run) invokes the appropriate connector binary (e.g., connectors/google/src/main.rs). The connector reads its source system (Google Drive, Slack, etc.) and writes documents via shared/src/content_storage.rs::store_content, which in turn persists to the DB.

Two import cycles inflate change risk:

services/ai/providers/__init__ ↔ services/ai/embeddings/__init__ (both in a cycle).
sdk/typescript/src/connector.ts participates in the same cycle.

The hub module web/src/lib/server/db/index.ts has the highest inbound degree (13) and therefore a large blast radius; any API change here ripples through many UI components.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/omni.git
cd omni

# Build all Rust crates (requires Cargo)
cargo build --workspace

# Build the UI (npm inferred from package.json in web/)
cd web
npm ci
npm run build   # see package.json scripts for exact command

# Start the stack with Docker Compose (compose file lives in repo root)
docker compose up --build

Configuration is read from .env.example; replace placeholder values with your own secrets (Postgres URL, Redis URL, LLM API keys). The UI runs on port 3000, the Rust API on port 8000 (as defined in services/searcher/src/main.rs – not listed but inferred from Cargo.toml binaries).

Real‑World Use

A company deploys the stack behind its internal network. An employee searches “Q3 sales deck” in the web UI; the request hits run_server, which queries Postgres (BM25 + pgvector) via shared/src/db/repositories/document.rs. If the result is a Google Drive file, the AI agent calls the Google connector, which streams the document into shared/src/content_storage.rs and returns a summary generated by the LLM configured in services/ai/.

Code Health & Issues

Measured static‑analysis findings

HIGH (cognitive_load) – Deep nesting (max depth 8) in connectors/microsoft/ms_connector/graph_client.py, connectors/github/github_connector/connector.py, services/ai/providers/bedrock.py.
HIGH (soundness) – Import cycles in services/ai/embeddings/__init__.py, services/ai/providers/__init__.py, sdk/typescript/src/connector.ts.
MEDIUM (resilience) – Bare except: blocks in connectors/microsoft/ms_connector/syncers/onedrive.py, .../sharepoint.py.
MEDIUM (clarity) – Hub module web/src/lib/server/db/index.ts has 13 dependents; changes have high blast radius.

Health‑audit findings (11 total)

HIGH – Pin GitHub Actions to commit SHAs (.github/workflows/*.yml).
HIGH – Upgrade vulnerable dependencies (e.g., tar@0.4, futures-util@0.3, sqlx@0.8, opentelemetry_sdk@0.27).
HIGH – Remove committed .env.test in services/indexer/ and rotate its secrets.
HIGH – Add a test step to CI workflows (currently no test command).
MEDIUM – Enable Dependabot or Renovate (.github/dependabot.yml).
MEDIUM – Pin Docker base images by digest in connectors/atlassian/Dockerfile.
MEDIUM – Add a dependency‑vulnerability scan gate to CI.
MEDIUM – Add a pre‑commit secret‑scan hook.
MEDIUM – Run containers as non‑root (USER directive missing).
LOW – Set timeout‑minutes on workflow jobs (.github/workflows/build-ai.yml).

No critical issues were detected; tests, CI, Dockerfiles, and a LICENSE are present.

The Bottom Line

omni delivers a fully self‑hosted, multi‑source search and AI assistant built on a consistent Rust‑centric backend with a Svelte UI. The codebase is modular but contains several high‑impact cycles and deep nesting that increase maintenance cost. Security‑related hygiene (pinned actions, secret handling, image digests) needs immediate tightening before production use. Teams comfortable with Rust, Docker, and Terraform will find the platform extensible; others should allocate effort to resolve the highlighted health issues first.

What the analyser found

Deployment readiness

7/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

897 files

Svelte157
Rust151
TypeScript150
Python143
Terraform70
SQL56

ReactSvelteDockerTerraform

Module dependencies

The 10 most depended-upon modules of 294, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_web_src_lib_server_db_index["index"] n1_services_ai_db_connection["connection"] n2_web_src_lib_server_db_schema["schema"] n3_sdk_typescript_src_models["models"] n4_ft_ms_connector_graph_client["graph_client"] n5_rvices_ai_providers___init__["__init__"] n6_vices_ai_embeddings___init__["__init__"] n7_sdk_typescript_src_client["client"] n8_web_src_lib_server_config["config"] n9_python_omni_connector_models["models"] n7_sdk_typescript_src_client --> n3_sdk_typescript_src_models n0_web_src_lib_server_db_index --> n2_web_src_lib_server_db_schema n0_web_src_lib_server_db_index --> n8_web_src_lib_server_config classDef cyc stroke:#ef4444,stroke-width:2px; class n5_rvices_ai_providers___init__,n6_vices_ai_embeddings___init__ cyc;

Dependencies

54 declared dependencies, none a major version behind. Resolved 0 of 54 against the registry.

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high4

medium5

low2

Pin third-party GitHub Actions to a commit SHAhigh4 occurrences

.github/workflows

docker/setup-buildx-action@v3, docker/login-action@v3, docker/metadata-action@v5, docker/build-push-action@v5

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Upgrade the pinned dependency carrying a published advisoryhigh14 occurrences

tar@0.4 CVE-2018-20990 (high) +8 more; futures-util@0.3 CVE-2020-35908 (medium) +1 more; sqlx@0.8 GHSA-xmrp-424f-vfpx (medium) +1 more; opentelemetry_sdk@0.27 CVE-2026-48504 (medium)

The pinned version is the one that installs, so a published advisory against it describes this deployment rather than a hypothetical one, and the advisory is also a public description of how to exploit it.

Fix: Upgrade to the fixed version named in the advisory, then commit the lockfile.

Remove the committed .env and rotate what it holdshigh

services/indexer/.env.test

A tracked .env is the most common route for a working key to reach a public clone, and it is the file the app actually loads, so the value is usually live.

Fix: git rm --cached the file, add it to .gitignore, rotate every credential it names, commit a .env.example with empty values.

Make CI invoke the test suite it hashigh75 occurrences

.github/workflows

75 test files, no test command in the 3 workflows read

A green check that never executed an assertion is worse than no check, because reviewers trust it.

Fix: Add a test step to the existing workflow rather than a new workflow.

Enable Dependabot or Renovatemedium

21 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Pin the container base image by digestmedium2 occurrences

connectors/atlassian/Dockerfile

lukemathwalker/cargo-chef:latest-rust-1.91.0-bookworm, debian:bookworm-slim

An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.

Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Add a pre-commit secret gatemedium

a secret-shaped file is tracked and no repo-level gate is visible

Without a gate the same class of leak recurs on the next commit, so a leak finding is the symptom and this is the cause.

Fix: Add a pre-commit hook scanning staged content, and enable push protection.

Add a non-root USER to the imagemedium

connectors/atlassian/Dockerfile

CMD or ENTRYPOINT with no USER directive

A process running as root in the container is root against every mounted volume, and it turns any container escape or writable-mount mistake from a contained problem into a host one.

Fix: Create an unprivileged user, chown what it needs, and end the Dockerfile with USER.

Set timeout-minutes on the workflow jobslow3 occurrences

.github/workflows/build-ai.yml

3 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Add the repository convention files this project lackslow3 occurrences

missing .editorconfig, .gitattributes, a formatter config

Without them one contributor's editor writes tabs into a Python file, a shell script commits with CRLF and fails in the container, and a notebook diff is unreviewable.

Fix: Add .editorconfig, .gitattributes with text=auto eol=lf, and a formatter config.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.