Document parsing and extraction / Benchmark leaderboard
olmOCR-bench
PDF-to-Markdown fidelity; current card describes 1,403 PDFs and 7,010 unit tests.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Document property/unit-test pass rate.
Execution requirements
Document parser producing normalized Markdown.
Scope and limitations
The unit tests measure selected properties, not all possible extraction fields; use alongside a schema-extraction test.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Established newer suite
- Hugging Face · allenai/olmOCR-bench ↗Public · Selected document parsing suite.
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
ABBYY: Direct document API after output mapping
Extend: Direct document API after output mapping
Instabase: Direct document API after output mapping
LlamaIndex: Direct document API after output mapping
Mistral: Capability-aligned; adapter/access to qualify
Reducto: Direct document API after output mapping
Rossum: Direct document API after output mapping
Unstructured: Direct document API after output mapping