The Problem

Teams struggle to surface relevant knowledge from dozens of internal tools while keeping data under strict control. Existing LLM‑centric solutions often expose all records to the model or consume excessive tokens for simple queries. Onyx aims to deliver low‑latency, cost‑effective context for any request, with a single deployment that protects data sovereignty.

What This Does

Onyx ingests data from over 50 external sources (Slack, JIRA, GitHub, etc.) and builds an internal knowledge graph. Its agent‑based RAG layer filters and ranks documents before handing them to an LLM, reducing token usage and latency. Key code paths live under backend/onyx/. The main.py file starts the REST API, while sandbox_proxy/server.py powers the secure execution sandbox. The CLI, located in cli/, exposes commands for provisioning, schema migration, and model‑server management.

How It Is Wired

Entry point – backend/onyx/main.py

python backend/onyx/main.py

This script imports backend/onyx/server.py, which sets up a FastAPI router. The router delegates to backend/onyx/controllers/search.py for queries. Search flow –

  1. search_handler() in controllers/search.py receives a request.
  2. It calls backend/onyx/services/knowledge_graph.py::query_graph().
  3. query_graph() queries the internal vector store (backend/onyx/storage/vector_store.py) and returns ranked documents.
  4. The handler formats a response and returns it to the client.

Sandbox flow – backend/onyx/sandbox_proxy/server.py launches a Docker‑based sandbox container at request time. The container runs the user’s code inside a read‑only filesystem; logs are streamed back to the client via backend/onyx/sandbox_proxy/streamer.py.

Model server – backend/model_server/main.py exposes a gRPC endpoint that forwards prompts to a local LLM (e.g., vLLM). The model server is invoked separately or as a sidecar in Kubernetes (deployment/ contains Helm charts).

Dependency hub – backend/onyx/server/features/build/sandbox/image/Dockerfile defines the sandbox image; backend/onyx/server/features/build/sandbox/image/templates/outputs/web/package.json shows the front‑end stack used to interact with the sandbox.

The code graph contains a single cycle between controllers/search.py and services/knowledge_graph.py, which is isolated from the sandbox logic, limiting ripple effects.

How To Use It

Setup – the repo ships a Docker image for the backend:

docker build -f backend/Dockerfile -t onyx/backend .

Alternatively, use the provided Makefile target:

make docker-build

Configuration – environment variables are listed in .devcontainer/devcontainer.json. Required keys include ONYX_LLM_ENDPOINT, ONYX_DB_URL, and ONYX_SANDPAPER_KEY. A sample .env.example (not committed) shows the expected names. Running – start the API:

python backend/onyx/main.py

Or run the model server:

python backend/model_server/main.py

Use the CLI for maintenance:

go run cli/main.go migrate

Real‑World Use

A product team can expose a Slack bot that asks Onyx for internal policy documents. The bot sends a request to /api/search, which hits the knowledge graph, retrieves the relevant policy, and returns a concise answer. If the policy requires code review, the bot can trigger backend/onyx/sandbox_proxy/server.py to execute the review script in isolation.

Code Health & Issues

  • Low/Security – Secret‑shaped paths:
  • backend/keys/license_public_key.pem – public key file.
  • backend/scripts/reencrypt_secrets.py – utility for rotating secrets.
  • backend/tests/unit/onyx/redis_ca.pem – test certificate. The audit confirms that no sensitive data is hard‑coded; the files are intentionally public or test artifacts.

No other security or linting issues were reported in the static analysis.

The Bottom Line

Onyx offers a mature, multi‑tenant platform that unifies LLM interaction with internal knowledge while preserving data control. The codebase is large but well‑segmented: backend logic, sandbox engine, model server, and CLI are clearly isolated. The main trade‑off is the complexity of the deployment pipeline; a single Docker image is available, but full stack deployment still requires Kubernetes manifests. Teams that need a robust, self‑hosted RAG layer with fine‑grained access control should consider Onyx.