Benchmarks
PDF library benchmarks, measured on real files.
pdfboss against PyMuPDF, pypdfium2, pdfplumber and pdf_oxide on text extraction, opening and parsing, rendering and scanned documents, plus the 200-document opendataloader-bench for Markdown quality. Every number below comes from the harness in the repository, and every table is reproducible.
Text extraction: ~24× PyMuPDF, ~36× pdf_oxide.
Open + parse: ~2.3× pdf_oxide.
Rendering the mixed corpus: ~35% ahead of pdfium itself.
On a 544-page JBIG2 scanned book, fastest of the four measured.
Text extraction
40 real-world PDFs · pages per second, higher is better
~24× PyMuPDF, ~36× pdf_oxide. Open + parse tells the same story: 397,000 pages/s to pdf_oxide’s 171,000.
Rendering
38 certified files · 888 pages · pages per second
Fastest on the mixed corpus, ~35% ahead of pdfium itself, with no C in it. pdfplumber renders via pdfium.
Scanned documents
544-page JBIG2 book · pages per second
~23% ahead of the C-backed renderers, and the only one of the four with no C in it.
Every file in the rendering benchmark is certified before timing: pages reporting dropped or approximated content exclude their file, and an ink-coverage gate catches silently skipped work.
| Engine | Reading-order NID , higher is better | Total time |
|---|---|---|
| pdf-inspector | 0.915 | 0.44 s |
| liteparse | 0.913 | 0.75 s |
| opendataloader | 0.902 | 2.57 s |
| pymupdf4llm | 0.886 | 17.12 s |
| pdfboss md | 0.883 | 0.15 s |
| pdfboss text | 0.873 | 0.15 s |
| markitdown | 0.844 | 16.17 s |
pdfboss md scores mid-field on reading order and reads the whole corpus in about a seventh of a second, ~3× faster than the fastest competing Markdown engine.
Method
How the numbers were taken.
Text and parsing. Best-of-3 per file over 40 real-world PDFs, aggregated over the files every library handled, all tables from one session on one machine. The pure-Python readers are roughly 90× to 430× slower on extraction. doc.extract_text() spreads pages across cores, which widened the gap over the sequential libraries from the 7× measured before that landed; since every span also carries its style (font, weight, decorations, color), the extraction rows pay a few percent for it. Opening is close to free because the page tree loads lazily, so the open + parse ratio says more about what the other libraries do eagerly than about pdfboss.
Rendering. A renderer that skips work looks fast, so every file is certified before the stopwatch starts. pdfboss rasterizes each page through render_reporting with font substitution on (what the other engines do by default); a file where any page reports dropped or approximated content is excluded, with the reason printed and counted. A second gate renders each file’s first page in every library and excludes files whose ink coverage disagrees, because a blank page renders instantly and means nothing. 38 of the 40 files certify (888 pages); the two that fail each draw a code their font has no glyph for.
Scanned documents. A 544-page JBIG2 book (1994 × 2832 samples per page) rasterized to PNG at 1:1, all four libraries timed in one pass. The ink coverage on page 1 agrees across libraries (4.71% to 4.87%), which is what makes the timings comparable: a library that cannot decode a scan’s codec usually hands back a blank page instead of raising. Four fifths of pdfboss’s time goes to the JBIG2 arithmetic decoder, a serial dependency chain that neither vectorizes nor parallelizes.
Markdown quality. Quality rows come from opendataloader-bench’s own evaluator over all 200 documents. The two pdfboss timings are the median of five single-process runs after a warm-up; the other engines’ timings are the ones published with the corpus from a different machine, so read them as order-of-magnitude context rather than a same-machine race. On the benchmark’s combined metric the Markdown adapter scores 0.810 (reading order 0.883, headings and lists 0.709, table structure 0.494). Per document, the plain-text output beats the leading engine’s reading-order score on 92 of the 200 files, ties on 15 and loses on 93; the losses concentrate in table regions.
Text quality. On py-pdf/benchmarks, the pypdf project’s 14-document text-extraction suite scored by Levenshtein similarity against hand-edited ground truth, pdfboss averages 96.66%, the highest score in the field the suite publishes (pypdfium2 96.65%, pypdf 96.43%), measured with the suite’s own harness and postprocessing.
Numbers are machine-dependent. Compare rows against each other, and reproduce them with benchmarks/bench.py and bench_render.py in the repository.
What is not measured
Where the tables stop.
- Rendering is timed only on files pdfboss provably rasterizes completely. Timing it against full renderers on the rest would credit it for work it skips.
- The Markdown table measures reading order and structure; visual fidelity is out of scope. pdfboss sits mid-field on quality and first on time.
- No OCR, no form filling, no editing of existing PDFs are compared, because pdfboss does not do them.