kreuzberg - Moses Yebei

kreuzberg

Abstract

A polyglot document intelligence framework with a Rust core. Extract text, metadata, images, and structured information from PDFs, Office documents, images, and 97+ formats. Available for Rust, Python, Ruby, Java, Go, PHP, Elixir, C#, R, C, TypeScript (Node/Bun/Wasm/Deno)- or use via CLI, REST API, or MCP server.

The Problem

Teams extracting text from documents face a fragmented toolchain: separate libraries for PDFs, Office files, and images, each with its own API and output format. Kreuzberg consolidates this into a single extraction pipeline with one interface across 97+ formats, plus code intelligence for 305 programming languages.

What This Does

The Rust core (in crates/kreuzberg/) handles extraction, OCR, layout analysis, and chunking. It exposes bindings for 14 languages, a CLI (crates/kreuzberg-cli/src/main.rs), a REST API, and an MCP server. The packages/ directory contains the language-specific wrappers, e2e/ holds cross-language integration tests, and test_documents/ provides 2,405 sample files.

A plugin system (crates/kreuzberg/src/plugins/registry/processor.rs) supports custom OCR backends and post-processors. The .ai-rulez/ directory documents architecture decisions and domain-specific rules for extraction, OCR, and plugin development.

How It Is Wired

Execution starts at main in crates/kreuzberg-cli/src/main.rs, which reaches 262 functions. The shortest path to external effect is main -> parse with a filesystem read via reader.read_bytes. For server mode, serve_command in crates/kreuzberg-cli/src/commands/server.rs starts a REST API that reaches 120 functions.

The call graph shows 4,799 resolved edges. extract_file_sync (called 124 times) and extract_bytes (119 times) are the primary public APIs. is_empty is called from 207 places—the hottest function in the codebase. Key routing hubs:

crates/kreuzberg/src/core/extractor/bytes.rs — single function extract_bytes, called from 28 files, calls into 19 others
crates/kreuzberg/src/core/extractor/sync.rs — sync wrappers, called from 22 files
crates/kreuzberg/src/core/config/formats.rs — format dispatch, called from 84 files
crates/kreuzberg/src/utils/string_utils.rs — string helpers, called from 42 files

The counter.rs in keywords/yake/ is the most connected module (called from 109 files) despite being small—changing it has wide blast radius. The pool.rs utility manages resource pools with hit-rate tracking.

How To Use It

Setup: Install the Rust core and bindings via your package manager. The repo uses pnpm for Node packages. For Python:

pip install kreuzberg

Configuration: No mandatory environment variables. Settings live in .ai-rulez/config.yaml for development. OCR backends (PaddleOCR, PDFium) are optional plugins.

Running it: The CLI is the simplest entry point:

cargo run --bin kreuzberg -- --input document.pdf --output extracted.txt

Or use the REST API via serve_command, or the Python API:

from kreuzberg import extract_file
result = extract_file("document.pdf")

Real-World Use

A document processing pipeline for a legal firm: ingest PDFs, scanned images, and DOCX files, extract text and metadata, run OCR on scanned pages, then chunk results for embedding into a vector database. The unified API means one code path handles all formats, with language bindings for the team's stack.

Code Health & Issues

Static analysis found 45 findings (1 high, 17 medium, 27 low). The high-severity issue is duplicated code blocks across 171 files (44 repeated 6-line blocks), particularly in scripts/ and docs/snippets/. Medium issues include deep nesting in stopword language files and high branching density in check-release-assets.sh and hwp/mod.rs.

Security audit flagged 3 high-severity issues in CI/CD:

High — GitHub Actions unpinned (@v1, @v4 tags) in .github/workflows/; tags can be moved
High — publish.yaml discards exit codes at lines 1949-1950, masking failures
High — automated push to default branch in publish.yaml, bypassing PR review

Medium issues: mutable Docker base images (alpine:3.21), no dependency vulnerability scan in CI, 13 blobs over 5MB in test_documents/ (largest 59.2MB), persist-credentials not disabled on checkout, and no non-root user in the Dockerfile.

The Bottom Line

Kreuzberg is a serious, well-engineered document extraction framework with exceptional language coverage and a clean plugin architecture. The CI/CD security issues are fixable but should be addressed before production use. This is a strong choice for teams needing one extraction API across many formats and languages, though the 7,803-file repository is substantial to navigate.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

7,803 files

Markdown1759
JSON1547
Rust835
Java365
C#354
Elixir334

ReactFlaskExpressRailsSpringLaravelDockerKubernetesTransformersYOLOLangChain

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high3

medium5

Pin third-party GitHub Actions to a commit SHAhigh33 occurrences

.github/workflows

kreuzberg-dev/actions/free-disk-space-linux@v1, docker/setup-buildx-action@v4, docker/build-push-action@v7, kreuzberg-dev/actions/setup-python-env@v1

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Stop discarding the exit code of steps whose failure mattershigh2 occurrences

.github/workflows/publish.yaml

lines 1949, 1950 discard failure

The job reports green while the artifact it was supposed to produce was never produced, so a crash becomes a successful run built on stale data.

Fix: Let the step fail, or test the specific expected condition instead of discarding all exit codes.

Open a pull request instead of pushing to the default branchhigh

.github/workflows/publish.yaml

git push)

Automated commits land on the branch that deploys, with no test having run against the result.

Fix: Push to a bot branch and open a pull request, or restrict the push to a tag ref.

Pin the container base image by digestmedium2 occurrences

docker/Dockerfile.cli

alpine:3.21, alpine:3.21

An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.

Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Move large binaries to Git LFS or out of the repositorymedium13 occurrences

test_documents/pdf/proof_of_concept_or_gtfo_v13_october_18th_2016.pdf

13 blobs over 5MB: proof_of_concept_or_gtfo_v13_october_18th_2016.pdf 59.2MB, fundamentals_of_deep_learning_2014.pdf 54.1MB, intel_64_and_ia_32_architectures_software_developer_s_manual_combined_volumes_1_4_june_2021_325462_sdm_vol_1_2abcd_3abcd.pdf 50.5MB

One repository here carries twenty blobs over 5MB including a 10.9MB spreadsheet, so every clone and every CI checkout pays for data nobody diffs.

Fix: Track those extensions with LFS, or move datasets to object storage and fetch them in a setup step.

Set persist-credentials: false on checkoutmedium

.github/workflows/ci-docs.yaml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Add a non-root USER to the imagemedium

docker/Dockerfile.cli

CMD or ENTRYPOINT with no USER directive

A process running as root in the container is root against every mounted volume, and it turns any container escape or writable-mount mistake from a contained problem into a host one.

Fix: Create an unprivileged user, chown what it needs, and end the Dockerfile with USER.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.