The Problem
Many AI ops teams run custom dashboards and telemetry pipelines that are hard to swap between cloud providers. They often rely on vendor‑specific tooling, which forces them to rebuild the control plane each time a new provider is added.
What This Does
ai-engine is a single Go binary that exposes a templ + htmx dashboard and a REST API for GPU fleet telemetry, node remediation, and multi‑cloud node‑pool provisioning. The core logic lives in cmd/api/main.go, which spins up an HTTP server (internal/server/server.go) that registers routes from internal/server/routes.go. Telemetry is collected by internal/telemetry/scraper.go and broadcasted over SSE via internal/telemetry/broadcaster.go. Node state is kept in memory by internal/fleet/manager.go. Provisioning and remediation use the internal/providers interface; a mock implementation exists in internal/providers/mock.go while CAPI stubs are in internal/providers/capi.go and its templates.
How It Is Wired
Entry Point – cmd/api/main.go calls server.NewServer and starts listening on PORT. Routing – internal/server/routes.go maps URLs (e.g. /api/nodes/{id}/drain) to handlers that call into internal/remediation/controller.go or internal/fleet/manager.go. Telemetry Loop – internal/telemetry/scraper.go runs every 2 s, queries GPU metrics, and sends them through internal/telemetry/broadcaster.go to all connected SSE clients. Provider Registry – internal/providers/provider.go registers adapters (mock.go, capi.go); the /api/clusters POST handler invokes the selected adapter to provision a node pool. Wiring Cost – The server is a single hub; most functionality is reachable via the server routes, so a change to internal/server/routes.go can ripple across the API surface.
How To Use It
Setup – Install Go (≥1.22) and run:
make build # generates templ code, builds Tailwind CSS, compiles binary
Configuration – The binary reads environment variables (PORT, BRAND_NAME, etc.) described in the README. No external config files are required. Running –
PORT=8090 ./main
or for live reload:
make watch # requires air
Container – Build a Docker image:
docker build -t ghcr.io/nutcas3/ai-engine:latest .
The Dockerfile and docker-compose.yml provide a ready‑to‑run stack.
Real-World Use
An ops team can point the dashboard at a collection of GPU nodes across AWS, CoreWeave, and on‑prem Metal3. Each node streams metrics to /api/telemetry/stream; a degraded node can be drained through /api/nodes/{id}/drain, which swaps the card to a “Draining (CAPI)” state. The same API can provision a new node pool on a CAPI‑enabled cluster via /api/clusters, using the provider adapters defined in internal/providers.
Code Health & Issues
- License – No
LICENSEfile; redistribution rights are unclear. - Test Coverage – 3 test files (
internal/errs/errs_test.go,internal/server/routes_test.go,internal/providers/capi_test.go) cover a fraction of logic. - CI – GitHub Actions (
.github/workflows/ci.yml) runmake verifyon each push; releases are automated via Release Please. - Documentation –
docs/architecture.mdanddocs/openapi.yamlprovide API contracts, but inline comments are sparse. - Dependency Management – Go modules (
go.mod,go.sum) are present; Docker builds are defined.
The Bottom Line
ai-engine delivers a lightweight, cloud‑agnostic control plane with a live dashboard and extensible provider adapters. Its Go‑centric, templ‑driven design keeps the runtime small, but the lack of a license and limited tests reduce confidence for production use. Teams that need rapid telemetry and remediation across heterogeneous GPU fleets will benefit, provided they can add proper testing and licensing.