The Problem

Extracting structured content (layout, formulas, tables, OCR text) from heterogeneous PDFs is still a manual‑heavy process. Engineers need a single toolbox that wraps state‑of‑the‑art models and lets them be swapped via configuration, without writing glue code for each document type.

What This Does

PDF-Extract-Kit bundles ready‑to‑run models for layout detection, formula detection/recognition, OCR and table parsing. The core code lives in pdf_extract_kit/ (52 files) and is driven by the YAML files in configs/. For example, configs/layout_detection.yaml selects a YOLO‑based detector, while configs/ocr.yaml points to PaddleOCR. The registry/registry.py module (16 dependents) registers each task class, allowing the CLI in scripts/pdf2markdown to instantiate the requested pipeline with a single call.

How It Is Wired

Execution starts at the script project/pdf2markdown/scripts/pdf2markdown (the only entry point with a main function). It parses command‑line arguments, loads the global config via pdf_extract_kit/utils/config_loader.py, and looks up the requested task in pdf_extract_kit/registry/registry.py.

  • registry.py → imports the concrete task modules (e.g., pdf_extract_kit/tasks/layout_detection/task.py, pdf_extract_kit/tasks/ocr/task.py).
  • Each task inherits from pdf_extract_kit/tasks/base_task.py, which defines run() and delegates to model‑specific code under tasks/<task>/models/.
  • Layout detection ultimately calls layoutlmv3_util/layoutlmft/models/layoutlmv3/modeling_layoutlmv3.py (the largest file, 1012 lines) to produce bounding boxes.
  • OCR invokes pdf_extract_kit/tasks/ocr/models/paddle_ocr.py, which wraps PaddleOCR inference.
  • Results are streamed back to the CLI script, which writes Markdown files under assets/readme/ or user‑specified locations.

The registry module is the central hub (16 inbound imports, 0 outbound), so any change there ripples through the entire pipeline. No circular imports were detected, keeping the dependency graph acyclic.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/PDF-Extract-Kit
cd PDF-Extract-Kit

# Install Python dependencies
pip install -r requirements.txt      # top‑level manifest
pip install -r docs/requirements.txt    # docs‑only deps (optional)

# Choose a config (e.g., layout detection with YOLO)
export CONFIG_PATH=configs/layout_detection.yaml

# Run the demo conversion script
python project/pdf2markdown/scripts/pdf2markdown \
    --input assets/demo/PDFs/DDPM.pdf \
    --output output.md

The script expects the CONFIG_PATH environment variable or --config flag to locate the YAML file. No Dockerfile or server entry point is provided; the toolkit is intended for direct CLI or library use.

Real‑World Use

A content‑management system can call the library as a Python module:

from pdf_extract_kit.registry import Registry
from pdf_extract_kit.utils.config_loader import load_config

cfg = load_config('configs/ocr.yaml')
ocr_task = Registry.get('ocr')(cfg)
text = ocr_task.run('docs/example.pdf')
store_in_cms(text)

This pattern lets the CMS swap OCR back‑ends simply by editing the YAML, without touching the integration code.

Code Health & Issues

  • High – No test suite – 58 source files, zero test files.
  • High – Missing lockfile – pyproject.toml and requirements.txt declare ranges, no poetry.lock/requirements.lock.
  • High – No CI workflow – repository lacks .github/workflows/.
  • Medium – Deep nesting – 15 occurrences, e.g., pdf_extract_kit/tasks/base_task.py (max indent depth 7).
  • Medium – Files opened without context manager – 5 spots, e.g., pdf_extract_kit/dataset/dataset.py.
  • Medium – Duplicated config helpers – identical blocks across docs/en/conf*.py and docs/zh_cn/conf.py.
  • Medium – Broad exception handling – generic except: in pdf_extract_kit/tasks/ocr/models/paddle_ocr.py.
  • Medium – Oversized modules – modeling_layoutlmv3.py (1012 lines) and visualizer.py (large monolith).
  • Medium – High branching density – 4 files with >30 branches per 100 lines.
  • Medium – TODO/FIXME markers – three occurrences in beit.py and visualizer.py.
  • Medium – Large binary assets – PDFs >5 MiB in assets/demo/PDFs/ (e.g., DDPM.pdf 9.8 MiB).

The Bottom Line

PDF-Extract-Kit delivers a functional, modular pipeline for PDF content extraction, with clear configuration entry points and a broad model portfolio. However, the codebase lacks automated testing, reproducible dependency locking, and CI, and contains several maintainability hotspots (deep nesting, oversized files). It is suitable for teams willing to add test coverage and CI scaffolding before integrating it into production workloads.