The Problem
AI agents often run in production without observability, cost tracking, or guardrails. Teams struggle to trace every LLM call, evaluate safety, and adjust prompts without redeploying. The result is opaque failures, hidden costs, and brittle guardrails.
What This Does
OpenLIT supplies an end‑to‑end harness for agent observability and cost management.
- Tracing –
cli/internal/otlp/exporter.gopushes OpenTelemetry spans for every LLM and tool call into the OpenTelemetry collector (opentelemetry-gpu-collector/). - Guardrails –
cli/internal/coding/hook/opencode/privacy.goenforces privacy rules at runtime. - Pricing –
cli/internal/coding/pricing/pricing.gocalculates per‑token cost using local pricing data (docs/assets/pricing.json). - CLI –
cli/cmd/openlit/main.goexposes commands to launch agents, install hooks, and run diagnostics.
How It Is Wired
- Entry Point –
cli/cmd/openlit/main.goparses flags and callscmd.Run. - Command Routing –
cli/internal/coding/cmd.godispatches to sub‑commands (install, uninstall, doctor). - Hook Installation –
cli/internal/coding/install/install_opencode.gowrites the OpEncode plugin (plugins/opencode/openlit.ts) into the harness. - Runtime Tracing – During a call, the agent emits a span that flows to
cli/internal/otlp/exporter.go, which serialises to OTLP and forwards to the collector defined inopentelemetry-gpu-collector/otel-collector-config.yaml. - Guard Enforcement – The collector forwards the span to
cli/internal/coding/hook/opencode/privacy.go, which inspects the prompt and rejects disallowed content. - Pricing –
cli/internal/coding/pricing/pricing.goconsumesdocs/assets/pricing.jsonand attaches cost attributes to the span.
The wiring is a single path from CLI → internal command → hook installation → OTLP exporter → collector → guard & pricing. There are no hidden cycles; each file owns a clear slice of responsibility (e.g., otlp/exporter.go only handles OTLP payloads, privacy.go only inspects prompt text).
How To Use It
# 1. Clone
git clone https://github.com/moses-y/openlit
# 2. Build the CLI
cd openlit
go build ./cli/cmd/openlit
# 3. Install hooks into your harness
./openlit install opencode
# 4. Run your agent (example: Anthropic chat)
export OPENAI_API_KEY=sk-...
./openlit run examples/anthropic-chat-app/app.py
Configuration – Set OPENAI_API_KEY or the relevant model key. The collector is pre‑configured in opentelemetry-gpu-collector/otel-collector-config.yaml; no extra env vars needed unless you change the endpoint.
Running – The agent script (examples/anthropic-chat-app/app.py) is the entry point for your harness; OpenLIT injects the hook automatically.
Real‑World Use
A SaaS platform runs a large fleet of Cognition agents. By installing the OpenLIT hook, every LLM call is traced, cost is attached, and any prompt that violates the privacy policy is automatically blocked. The platform can then run CI gates on the trace data to prevent regressions before deployment.
Code Health & Issues
- High – Commit a lockfile beside the manifest –
sdk/python/pyproject.toml - Medium – Pin the container base image by digest –
cli/Dockerfile - Medium – Gate pull requests on a dependency vulnerability scan –
.github/workflows/ - Medium – Move large binaries to Git LFS or out of the repository –
src/opamp-server/opamp-server - Medium – Add a non‑root USER to the image –
cli/Dockerfile - Low – Set timeout‑minutes on the workflow jobs –
.github/workflows/release-cli.yml - Low – Add the repository convention files this project lacks – missing
.editorconfig,.gitattributes, formatter config
The repo contains a license, lockfiles (go.sum, docs/package-lock.json), tests, and CI. Secret files are present but audited.
The Bottom Line
OpenLIT provides a lightweight, OpenTelemetry‑native harness that adds tracing, guardrails, and cost tracking to any AI agent without changing the agent code. The CLI and hook system are well‑structured, and the project has a solid CI pipeline. However, the code health audit highlights missing lockfiles, mutable base images, and missing vulnerability scans. Teams that already use OpenTelemetry and need runtime guardrails and cost attribution will benefit most; those requiring zero‑trust deployments should address the highlighted security gaps.