The Problem
Teams building AI‑augmented applications often stitch together disparate LLM calls, image generators, and search APIs with ad‑hoc scripts. The resulting pipelines are fragile, undocumented, and hard to audit because each step is hard‑coded and model‑specific.
What This Does
Pipelex introduces a declarative “method” language (.mthds files) that describes typed AI steps, routing, and output contracts. The core runtime lives under the .pipelex/ folder – e.g. pipelex/inference/backends/.toml define 60+ model back‑ends, while pipelex/inference/deck/.toml group them into functional decks (LLM, image, extraction, search). The CLI (exposed via the pipelex console script) parses a method, selects a backend based on the type field, and executes it with strict input/output validation. Documentation is extensive: docs/features/ explains the language, docs/get-started/ shows first‑run steps, and docs/cookbook/ provides ready‑made pipelines.
How To Use It
Setup
Install the published package (recommended) pip install pipelex
Or, develop locally
git clone https://github.com/Pipelex/pipelex.git cd pipelex make install # target defined in the repository Makefile
The repository ships a Makefile with typical install, test, and lint targets; running make install creates an editable install and pulls dev dependencies.
Configuration
Copy the example environment file: cp .env.example .env Populate required keys for the back‑ends you intend to use (e.g., OPENAIAPIKEY, ANTHROPICAPIKEY). The files in pipelex/inference/backends/.toml read these variables at runtime. Optionally adjust routing or deck selection in pipelex/inference/routingprofiles.toml or pipelex/inference/deck/.toml.
Running a Method
Create a method file, e.g. summarize.mthds (see README example). Then execute:
pipelex run summarize.mthds
The CLI validates the method against the schema in pipelex/pipelex.toml, selects an appropriate backend (as defined in pipelex/inference/backends/openai.toml for OpenAI models), and returns structured JSON output.
For interactive development, the docs include a VS Code extension (docs/features/cli.md) and a Claude Code plugin (docs/features/claude-code-skills-plugin.md) that invoke the same pipelex run command under the hood.
Real‑World Use
A procurement system needs to extract line‑item tables from PDF invoices and then summarize total spend. Using Pipelex:
invoiceextractor.mthds [pipe.extracttable] type = "PipeExtract" inputs = { document = "PDF" } output = "Table" prompt = "Extract a structured table of line items from $document."
[pipe.summarizespend] type = "PipeLLM" inputs = { table = "Table" } output = "Text" prompt = "Summarize total spend and top three categories from the extracted table."
Running pipelex run invoice_extractor.mthds yields a typed table, which feeds directly into the second pipe without additional parsing code.
Code Health & Issues
Low – Missing explicit Python build config – No pyproject.toml or setup.cfg visible; packaging relies on CI (.github/workflows/publish-pypi.yml). May hinder custom builds. Low – Limited test coverage – Only 7 test files across 200+ source files; core inference routing paths are likely untested. Medium – Backend secrets handling – API keys are expected in environment variables; no secret‑management wrapper is provided, raising risk if keys are committed accidentally. Low – Documentation depth – Docs are comprehensive, but the quick‑start in README.md omits the exact CLI command (pipelex run). None – License – MIT license present (LICENSE). None – CI – GitHub Actions enforce linting, dependency review, and test badge generation; CI appears functional.
The Bottom Line
Pipelex delivers a focused, declarative framework for building reproducible AI pipelines, with solid documentation and CI scaffolding. It is best suited for teams that need typed, model‑agnostic method definitions and are comfortable configuring back‑ends via environment variables. The main drawbacks are modest test coverage and the lack of an in‑repo Python packaging manifest, which may add friction for custom builds.