The Problem

Organizations struggle to transform voluminous unstructured text—research papers, earnings reports, legal contracts—into usable, structured knowledge. Manual curation is slow, inconsistent, and scales poorly. There is a need for a system that can intelligently parse documents and produce predictable, strongly-typed knowledge representations without custom code for each domain.

What This Does

Hyper-Extract is a Python framework that transforms unstructured text into structured knowledge using LLMs. The core library lives in hyperextract/ (94 files), with 57 example scripts demonstrating extraction across domains. It supports 8 knowledge structures ranging from simple Lists/Sets to complex Knowledge Graphs, Hypergraphs, and Spatio-Temporal Graphs. Ten-plus extraction engines (GraphRAG, LightRAG, Hyper-RAG, KG-Gen) are available out of the box, alongside 80+ YAML templates for finance, legal, medical, and general domains. The repository includes 131 internal Python modules with 188 import edges and 8 modules in circular dependencies, indicating a moderately interconnected codebase where changes to central modules ripple widely.

How It Is Wired

Execution begins at hyperextract/cli/cli.py:46 via the main entry point, which reaches 94 functions and is called from a single place. The call graph shows main invokes create, feed_text, and show frequently. Downstream, feed (cli.py:686) reaches 92 functions and writes to the filesystem via os.makedirs, while talk (cli.py:605) reaches 79 functions. Key hub modules include hyperextract/types/__init__.py (30 importers, instability 0.23) and hyperextract/__init__.py (21 importers, instability 0.16), both of which carry high blast radius—modifying these requires caution due to wide-ranging dependencies. Eight modules participate in import cycles (hyperextract/utils/template_engine/__init__.py, hyperextract/cli/cli.py, hyperextract/utils/template_engine/gallery.py), creating fragile inter-module dependencies. The tracing shows main -> configure_logging touches the filesystem via log_path.parent.mkdir, and feed -> dump via os.makedirs. Five files exceed 800 lines (hyperextract/types/graph.py, hyperextract/cli/cli.py, hyperextract/types/hypergraph.py), making control flow hard to follow.

How To Use It

Setup: Install with uv tool install hyperextract. No Dockerfile or Makefile is present; the project uses standard Python tooling.

Configuration: Create an API key config with he config init -k YOUR_OPENAI_API_KEY. The .env.example file provides template variables; no lockfile is committed, so dependency versions may shift between installs.

Running it: To extract from a document:

he parse paper.pdf -t general/academic_graph -o ./paper_kb/
he show ./paper_kb/

The parse entry point (cli.py:217) reaches 99 functions; build_index (cli.py:758) reaches 69 functions and is called from 10 places.

Real-World Use

A research team feeds a set of academic papers into Hyper-Extract to build a knowledge graph of key concepts, authors, and citations. Using the general/academic_graph template, they run he parse on each PDF, outputting structured graphs to a local knowledge base. New papers can be incrementally fed to expand the graph without re-processing prior documents. The team uses he search to query risk factors or methodology constraints across the accumulated corpus. For local deployment, they configure vLLM endpoints via create_client to keep data on-premise.

Code Health & Issues

  • [HIGH/cognitive_load] Deep nesting observed in hyperextract/types/graph.py, hyperextract/types/base.py, and hyperextract/types/hypergraph.py (max indentation depth 8), making control flow hard to follow.
  • [HIGH/clarity] Hub modules: hyperextract/types/__init__.py (30 dependents, instability 0.23), hyperextract/__init__.py (21 dependents, instability 0.16), and hyperextract/utils/logging.py (16 dependents, instability 0) have high blast radius; changes here affect many downstream modules.
  • [HIGH/soundness] Import cycle member: hyperextract/utils/template_engine/__init__.py, hyperextract/cli/cli.py, and hyperextract/utils/template_engine/gallery.py participate in circular dependencies, creating fragile import paths.
  • [MEDIUM/resilience] Broad exception handling in hyperextract/types/base.py swallows errors indiscriminately via bare except clauses.
  • [MEDIUM/cognitive_load] Oversized files: hyperextract/types/graph.py, hyperextract/cli/cli.py, and hyperextract/types/hypergraph.py each exceed 800 lines, hard to hold in one mental model.
  • [HIGH/clarity] Duplicated code blocks: 721 repeated 6-line segments across 83 example files (e.g., examples/en/autotypes/graph_demo.py, examples/zh/autotypes/graph_demo.py).
  • [HIGH] Third-party GitHub Actions pinned to version tags (astral-sh/setup-uv@v7, pypa/gh-action-pypi-publish@release/v1, codecov/codecov-action@v6) without commit SHAs, risking non-reproducible builds when owners move tags.
  • [MEDIUM] No dependency vulnerability scanning in CI; no gate exists to catch known-vulnerable packages before build.
  • [MEDIUM] Checkout retains the token across all later steps in docs.yml workflows; persist-credentials: false not set.
  • [LOW] Four workflow jobs in docs.yml declare no timeout-minutes, risking overlap on the platform's six-hour default.

The Bottom Line

Hyper-Extract delivers a capable, LLM-powered framework for structuring unstructured text with minimal code—particularly strong for graph-based knowledge extraction and incremental updates. However, the codebase shows signs of growing organically: deep nesting, oversized files, import cycles, and duplicated example logic increase the cost of future changes. The absence of a lockfile and dependency scanning further risks build reproducibility and security. Teams that need to extract and evolve knowledge graphs from documents will find immediate value, but engineers maintaining the framework should budget time to flatten nesting, break circular imports, and extract duplicated helpers before scaling contributions.