The Problem

AI‑agent platforms (n8n, Flowise, Make.com, LangChain, LlamaIndex) call Anthropic’s Claude API without prompt‑caching. Every request retransmits the full system prompt and tool definitions, inflating token usage, cost, and latency—especially for agents with 1 k‑5 k token system prompts or dozens of tool specs.

What This Does

autocache runs as a transparent HTTP proxy that inspects inbound Claude requests, computes optimal cache breakpoints, and injects the required cache-control fields before forwarding the call to Anthropic. The proxy also adds ROI analytics (cost‑saved, latency‑saved) to response headers.

  • Core logic lives in internal/cache/injector.go, which parses the JSON payload, determines token boundaries using the tokenizer in internal/tokenizer/, and inserts the cache_control object.
  • Network I/O is performed by internal/client/anthropic.go, a thin wrapper around Go’s http.Client that forwards the mutated request and returns the raw response.
  • internal/server/handler.go exposes a single /v1/complete endpoint (mirroring Anthropic’s API) and orchestrates config loading (internal/config/config.go), request injection, and response forwarding.

The repository ships with a full suite of scenario‑driven tests under test_data/ and unit tests in internal/*_test.go, confirming correct token counting and cache header insertion.

How It Is Wired

  1. Entry point – cmd/autocache/main.go reads environment variables via internal/config.Load(), constructs a Server struct, and calls server.Start().
  2. Server start – internal/server/handler.go registers the HTTP handler (http.HandleFunc("/", s.handleRequest)) and listens on the port defined in .env.example (default 8080).
  3. Request handling – handleRequest parses the incoming JSON, then calls injector.InjectCacheControl(reqBody) (in internal/cache/injector.go).
  4. Token analysis – injector delegates to tokenizer.New() (implemented in internal/tokenizer/real_tokenizer.go for live token counts or offline_tokenizer.go for deterministic tests).
  5. Cache header injection – after token boundaries are known, InjectCacheControl mutates the request payload to include the cache_control object.
  6. Forwarding – the mutated payload is passed to client.Do(req) in internal/client/anthropic.go, which contacts https://api.anthropic.com/v1/complete using the API key from the env.
  7. Response handling – the raw response is streamed back to the caller; handler adds custom ROI headers (X-Autocache-Cost-Saved, X-Autocache-Latency-Improvement) derived from token counts and pricing logic in internal/pricing/pricing.go.

The only side‑effects are outbound HTTP calls and header mutation; no persistent storage is used. The hub of the call graph is handler.handleRequest, which touches config, injector, client, and pricing modules. No cycles are present, making the flow straightforward to modify.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/autocache
cd autocache

# Build the binary (Makefile provides a target)
make build          # produces ./bin/autocache (or similar)

# Copy example env and add your Anthropic key
cp .env.example .env
# edit .env → ANTHROPIC_API_KEY=your_key

# Run via Docker (Dockerfile is present)
docker build -t autocache .
docker run -p 8080:8080 --env-file .env autocache

Alternatively, start the binary directly:

go run cmd/autocache/main.go

The proxy listens on http://localhost:8080. Point any Claude client (n8n, Flowise, etc.) to this base URL instead of Anthropic’s official endpoint. No code changes are required in the upstream workflow.

Real‑World Use

A n8n workflow that previously called https://api.anthropic.com/v1/complete can be retargeted to http://localhost:8080/v1/complete. The workflow continues to send the same JSON payload; autocache injects cache_control automatically, and the response headers now report savings, e.g.:

X-Autocache-Cost-Saved: $0.0405
X-Autocache-Latency-Improvement: 83%

Code Health & Issues

  • Tests – Low – 58 test files (unit, integration, scenario) run via make test; coverage appears comprehensive.
  • CI – Low – GitHub Actions workflows (.github/workflows/ci.yml, pr-video.yml, release.yml) execute lint, test, and build steps.
  • License – Low – MIT license present (LICENSE).
  • Configuration – Medium – .env.example exists, but the exact variable names are not documented beyond the example file. Adding a README section with required env keys would improve onboarding.
  • Docker – Low – Dockerfile builds a minimal image; no multi‑stage build analysis needed.

No structural red flags detected; the code follows a clear separation of concerns.

The Bottom Line

autocache delivers a ready‑to‑run Go proxy that automatically adds Claude prompt‑caching, delivering measurable cost and latency reductions without altering existing agent pipelines. It is well‑tested, containerized, and straightforward to extend, but documentation of required environment variables could be clearer. Ideal for teams that already use Anthropic and want immediate ROI with minimal integration effort.