The Problem

Organizations need a single platform that can create embeddings, run vector search and orchestrate language‑model workflows without piecing together disparate libraries. Manual glue code inflates maintenance costs and slows delivery.

What This Does

txtai bundles semantic search, LLM orchestration and multimodal pipelines into one Python package.

  • Embeddings database (src/python/txtai/embeddings/engine.py) stores sparse and dense vectors, graph indices and relational tables.
  • Pipelines (src/python/txtai/pipelines/*.py) expose reusable LLM tasks such as summarisation, QA, transcription and image captioning.
  • Workflows (src/python/txtai/workflows/*.py) allow chaining pipelines and adding business logic.
  • Agents (src/python/txtai/agents/*.py) coordinate the components autonomously.

The framework ships with a FastAPI web service (src/python/txtai/web.py) and CLI entry point (src/python/txtai/console/__main__.py) that can be run as txtai.

How It Is Wired

Execution starts at txtai entry point: src/python/txtai/console/__main__.py.

  1. main() parses CLI arguments and loads the configuration (src/python/txtai/config.py).
  2. The Engine is instantiated (src/python/txtai/embeddings/engine.py) which registers vector stores (src/python/txtai/embeddings/store.py) and graph modules.
  3. For a search request, Engine.search() calls Embeddings.query() which delegates to the underlying vector index (src/python/txtai/embeddings/index.py).
  4. Pipeline execution (src/python/txtai/pipelines/llm.py) is triggered via the Pipeline.run() method; it loads a Hugging Face model, runs the prompt, and returns the result.
  5. Workflows (src/python/txtai/workflows/task.py) wrap pipelines and expose a REST endpoint through FastAPI (src/python/txtai/web.py).

The Engine is the widest blast radius: it touches the filesystem for model weights, the network for remote LLM calls, and the database for vector storage. The Pipeline layer is the next most influential, as it drives all model inference.

How To Use It

Setup

git clone https://github.com/moses-y/txtai
cd txtai
uv pip install .          # or pip install .

Configuration Create config.yml in the project root or set environment variables used by src/python/txtai/config.py. Typical keys: model, vector_store, db_url.

Running it

# Start the REST API
txtai serve --config config.yml

Or run a single pipeline:

txtai run pipeline/llm/summarise -i input.txt -o summary.txt

Docker support is provided via docker/api/Dockerfile; build and run with:

docker build -t txtai/api -f docker/api/Dockerfile .
docker run -p 8000:8000 txtai/api

Real‑World Use

A document‑centric search service can be built by indexing a corporate knowledge base:

from txtai.embeddings import Embeddings

e = Embeddings({"method": "sentence-transformers/all-MiniLM-L6-v2"})
e.index("docs/*.pdf")          # create embeddings
results = e.search("annual report 2023", limit=5)
print(results)

The returned hits can be fed into a RAG pipeline (src/python/txtai/pipelines/llm/rag.py) that answers questions using the retrieved context.

Code Health & Issues

  • Low – Dependency declaration without lockfile (pyproject.toml), reproducibility risk.
  • Low – No requirements.txt; builds rely on pyproject.toml.
  • Medium – 119 test files exist, but CI (.github/workflows/build.yml) only runs linting and tests on push, no coverage thresholds.

The repository contains a LICENSE (Apache 2.0) and a comprehensive docs/ tree, indicating good documentation practice.

The Bottom Line

txtai delivers a cohesive stack for embedding, vector search and LLM pipelines in Python. It is ready for rapid prototyping and can scale to production via Docker or Kubernetes. The main trade‑off is the lack of a deterministic lockfile, which may affect reproducibility in tightly regulated environments. Engineers familiar with Hugging Face and FastAPI will find the codebase approachable and extensible.