The Problem

Reading long PDFs (textbooks, manuals, research monographs) with an LLM fails because context windows truncate content and single-pass summarization loses detail. Extracting structured knowledge from a 300-page book requires either chunking the document yourself or accepting shallow output. This tool automates the page-by-page extraction and interval summarization workflow.

What This Does

read_books.py is a single-file Python script that processes a PDF page-by-page, sending each page to an AI model for knowledge-point extraction, then generating progressive summaries every N pages. It writes structured JSON knowledge bases and Markdown summaries to book_analysis/, and supports resuming from a partial knowledge base.

The repo is minimal: one script, a requirements.txt, two sample PDFs (meditations.pdf, infinite_math.pdf), and a README. No tests, no CI, no packaging.

How It Is Wired

Execution starts at read_books.py's top-level constants (PDF_NAME, ANALYSIS_INTERVAL, TEST_PAGES, MODEL, ANALYSIS_MODEL), then flows through the main routine. The script reads the PDF with a PDF library, iterates pages, calls the AI model API per page to extract knowledge, accumulates results, and writes outputs to book_analysis/knowledge_bases/ and book_analysis/summaries/.

Key functions include page processing, knowledge extraction, interval summary generation, and resume logic that checks for an existing knowledge base JSON before starting. The widest blast radius is the model call function—every page and every interval summary routes through it, so an API failure or rate limit halts the entire run. The requirements.txt declares dependencies (likely pypdf or similar, plus an AI SDK) but has no lockfile, so builds are not reproducible.

File-by-file map:

  • read_books.py — all logic: PDF parsing, model calls, knowledge extraction, summary generation, file I/O, resume logic. Owns every external effect (API calls, filesystem writes).
  • requirements.txt — dependency list, no versions pinned.
  • README.md — usage and configuration documentation.
  • LICENCE — license file.
  • meditations.pdf, infinite_math.pdf — sample input PDFs.

External effects: network calls to an AI model API (per page and per interval), filesystem writes to book_analysis/ (knowledge base JSON, summaries, PDF copy). Three hops from entry point to external effect: script start → page loop → model API call.

How To Use It

Setup:

git clone https://github.com/moses-y/AI-reads-books-page-by-page
cd AI-reads-books-page-by-page
pip install -r requirements.txt

Configuration: Edit read_books.py directly. Set PDF_NAME to your PDF's filename, place the PDF in the project root. Optionally adjust ANALYSIS_INTERVAL, TEST_PAGES, MODEL, and ANALYSIS_MODEL.

Running it:

python read_books.py

Outputs land in book_analysis/knowledge_bases/ (JSON) and book_analysis/summaries/ (Markdown). Set TEST_PAGES = None to process the whole book; ANALYSIS_INTERVAL = None to skip interval summaries.

Real-World Use

A technical writer processing a 400-page API reference manual: run the script with ANALYSIS_INTERVAL = 50, let it extract per-page knowledge into JSON, and use the 50-page interval summaries as chapter outlines. The resume capability means an interrupted run (API timeout, laptop sleep) picks up where it left off instead of re-processing pages.

Code Health & Issues

Static analysis has not been run on this repo, so no measured metrics are available. Structural observations:

  • Med - No tests - single script with no test files; page-processing and resume logic are untested.
  • Med - No CI/CD - no .github/ or CI config; nothing gates changes.
  • Low - No lockfile - requirements.txt without pinned versions means non-reproducible installs.
  • Low - Sample PDFs committed - meditations.pdf and infinite_math.pdf add binary weight to the repo.

The Bottom Line

A functional single-purpose tool that solves a real problem: structured knowledge extraction from long PDFs with progressive summaries and resume support. It is not production-grade—no tests, no CI, no lockfile—but for a personal or small-team workflow where you control the environment, it will do the job. Forked from a 2,679-star project, so the core approach has community validation.