The Problem

Enterprise data often lives in multiple silos with inconsistent names, units, and lineage. Agents that query this data can produce fast answers, but the answers are unreliable if the underlying schema is ambiguous. The pain point is the lack of a shared, machine‑readable ontology that tells an agent where a metric comes from, how it is computed, and which tables it depends on.

What This Does

auto_ontology builds a semantic layer on top of relational metadata.

  • Catalog & Normalization – auto_ontology/catalog/extract.py pulls schema information from a source database, normalize.py reconciles naming conflicts, and write.py persists the ontology in Postgres.
  • Query Generation – auto_ontology/retrieval/text_to_sql/main.py translates natural‑language questions into deterministic SQL using NVIDIA NIM, then runs the query via connectors/postgres.py.
  • Ingestion – auto_ontology/ingestion_service/ingest.py reads tabular datasets, derives embeddings, and writes both rows and links into the catalog.
  • API & UI – auto_ontology/server/__init__.py exposes FastAPI routes that expose catalog data, ingestion triggers, and NL‑to‑SQL endpoints. The frontend in frontend/ consumes these routes.

How It Is Wired

Entry points

  • auto_ontology/server/__main__.py – launches the FastAPI app (uvicorn auto_ontology.server.app:app).
  • auto_ontology/ingestion_service/__main__.py – starts the ingestion worker (uvicorn auto_ontology.ingestion_service.app:app).
  • auto_ontology/retrieval/text_to_sql/main.py – CLI for a one‑off text‑to‑SQL run.

Internal call graph

server/__main__.py
 └─ imports server.app
     ├─ includes router from ingestion_service/router
     │   └─ calls ingestion_service.ingest.ingest_dataset
     │       ├─ uses catalog.extract / catalog.normalize
     │       └─ writes via connectors/postgres
     ├─ includes router from retrieval/text_to_sql
     │   └─ calls retrieval.text_to_sql.main.run_query
     │       ├─ calls connectors/postgres.execute_sql
     │       └─ returns result
     └─ includes router from semantic/router
         └─ serves ontology endpoints

External touches

  • Postgres database (postgresql://…) accessed through connectors/postgres.py.
  • NVIDIA NIM inference endpoints defined in .env.example; used by retrieval/text_to_sql.
  • Optional Vault credentials via connectors/vault.py.

Responsibility per file

  • catalog/extract.py – schema discovery.
  • catalog/normalize.py – name reconciliation.
  • catalog/write.py – persistence.
  • ingestion_service/ingest.py – ingestion orchestration.
  • retrieval/text_to_sql/main.py – NL‑to‑SQL pipeline.
  • server/__init__.py – FastAPI app and router assembly.
  • connectors/*.py – database adapters.

The module graph contains a single cycle: server ↔ ingestion_service via shared connectors. Changing the connector registry requires touching both connectors/registry.py and the routers that import it, which is the only high‑blast‑radius area.

How To Use It

# 1) Clone
git clone https://github.com/moses-y/auto-ontology
cd auto-ontology

# 2) Install
# backend
uv run install -e .
# frontend
cd frontend && pnpm install

# 3) Configure
# copy example env
cp .env.example .env
# edit .env to set DB, NIM endpoints, Vault URL if used

# 4) Run stack
# Docker Compose builds all services
docker compose up -d
  • The backend listens on 3001 (FastAPI).
  • The ingestion worker listens on 3002.
  • The frontend UI is served on 3000.
  • Postgres runs on 5432 inside the compose network.

Alternatively, launch the backend directly:

uvicorn auto_ontology.server.__main__:app --host 0.0.0.0 --port 3001

Real‑World Use

A data‑science workflow can query a metric via the web UI:

import requests
resp = requests.post(
    "http://localhost:3001/api/text_to_sql",
    json={"question": "What is the average revenue per user in 2024?"}
)
print(resp.json()["sql"])

The response contains the deterministic SQL, the provenance path, and the result set. The same endpoint is exposed to agents that need to retrieve context before calling a LLM.

Code Health & Issues

  • Tests – 183 test files cover catalog, connectors, and ingestion.
  • CI – GitHub Actions (.github/workflows/ci-python.yml, ci-mcp.yml) run tests, lint, and build Docker images.
  • License – Apache‑2.0 (LICENSE).
  • Missing – No static analysis report is present; code quality metrics such as coverage or lint scores are not auto‑reported.
  • Observations – The repo is well‑structured with clear separation of concerns. The only potential maintenance cost is the shared connector registry, which is already documented.

The Bottom Line

Auto Ontology provides a pragmatic, well‑tested stack for turning raw relational metadata into a graph‑aware, queryable ontology. Its clear separation between ingestion, catalog, and retrieval makes it suitable for production teams that need reliable provenance and deterministic query generation. The main trade‑off is the need to run a Docker stack and maintain the PostgreSQL schema, but the provided docker‑compose.yml and Makefile targets minimise operational friction.