The Problem

AI agents often run in production without observability, cost tracking, or guardrails. Teams struggle to trace every LLM call, evaluate safety, and adjust prompts without redeploying. The result is opaque failures, hidden costs, and brittle guardrails.

What This Does

OpenLIT supplies an end‑to‑end harness for agent observability and cost management.

  • Tracing – cli/internal/otlp/exporter.go pushes OpenTelemetry spans for every LLM and tool call into the OpenTelemetry collector (opentelemetry-gpu-collector/).
  • Guardrails – cli/internal/coding/hook/opencode/privacy.go enforces privacy rules at runtime.
  • Pricing – cli/internal/coding/pricing/pricing.go calculates per‑token cost using local pricing data (docs/assets/pricing.json).
  • CLI – cli/cmd/openlit/main.go exposes commands to launch agents, install hooks, and run diagnostics.

How It Is Wired

  1. Entry Point – cli/cmd/openlit/main.go parses flags and calls cmd.Run.
  2. Command Routing – cli/internal/coding/cmd.go dispatches to sub‑commands (install, uninstall, doctor).
  3. Hook Installation – cli/internal/coding/install/install_opencode.go writes the OpEncode plugin (plugins/opencode/openlit.ts) into the harness.
  4. Runtime Tracing – During a call, the agent emits a span that flows to cli/internal/otlp/exporter.go, which serialises to OTLP and forwards to the collector defined in opentelemetry-gpu-collector/otel-collector-config.yaml.
  5. Guard Enforcement – The collector forwards the span to cli/internal/coding/hook/opencode/privacy.go, which inspects the prompt and rejects disallowed content.
  6. Pricing – cli/internal/coding/pricing/pricing.go consumes docs/assets/pricing.json and attaches cost attributes to the span.

The wiring is a single path from CLI → internal command → hook installation → OTLP exporter → collector → guard & pricing. There are no hidden cycles; each file owns a clear slice of responsibility (e.g., otlp/exporter.go only handles OTLP payloads, privacy.go only inspects prompt text).

How To Use It

# 1. Clone
git clone https://github.com/moses-y/openlit

# 2. Build the CLI
cd openlit
go build ./cli/cmd/openlit

# 3. Install hooks into your harness
./openlit install opencode

# 4. Run your agent (example: Anthropic chat)
export OPENAI_API_KEY=sk-...
./openlit run examples/anthropic-chat-app/app.py

Configuration – Set OPENAI_API_KEY or the relevant model key. The collector is pre‑configured in opentelemetry-gpu-collector/otel-collector-config.yaml; no extra env vars needed unless you change the endpoint.

Running – The agent script (examples/anthropic-chat-app/app.py) is the entry point for your harness; OpenLIT injects the hook automatically.

Real‑World Use

A SaaS platform runs a large fleet of Cognition agents. By installing the OpenLIT hook, every LLM call is traced, cost is attached, and any prompt that violates the privacy policy is automatically blocked. The platform can then run CI gates on the trace data to prevent regressions before deployment.

Code Health & Issues

  • High – Commit a lockfile beside the manifest – sdk/python/pyproject.toml
  • Medium – Pin the container base image by digest – cli/Dockerfile
  • Medium – Gate pull requests on a dependency vulnerability scan – .github/workflows/
  • Medium – Move large binaries to Git LFS or out of the repository – src/opamp-server/opamp-server
  • Medium – Add a non‑root USER to the image – cli/Dockerfile
  • Low – Set timeout‑minutes on the workflow jobs – .github/workflows/release-cli.yml
  • Low – Add the repository convention files this project lacks – missing .editorconfig, .gitattributes, formatter config

The repo contains a license, lockfiles (go.sum, docs/package-lock.json), tests, and CI. Secret files are present but audited.

The Bottom Line

OpenLIT provides a lightweight, OpenTelemetry‑native harness that adds tracing, guardrails, and cost tracking to any AI agent without changing the agent code. The CLI and hook system are well‑structured, and the project has a solid CI pipeline. However, the code health audit highlights missing lockfiles, mutable base images, and missing vulnerability scans. Teams that already use OpenTelemetry and need runtime guardrails and cost attribution will benefit most; those requiring zero‑trust deployments should address the highlighted security gaps.