Modern Open Source OCR With an API in 2026: Tika, Docling, DeepSeek-OCR and the Rest

OCR used to be boring. You threw a scanned page at Tesseract, got back a wall of text with a few mangled characters, and moved on with your life. That was the whole genre for about fifteen years.

Then vision language models showed up and ate the entire category. In the last eighteen months, document parsing went from “extract the characters” to “understand the page” — reading order, table structure, LaTeX formulas, handwriting, multi-column academic layouts, the works. And because everyone building RAG pipelines suddenly needed clean Markdown out of ugly PDFs, the tooling exploded.

So here’s the state of open source OCR in August 2026, focused on the thing that actually matters when you’re building something: can you run it behind an API on your own hardware? I’m looking at feature sets, server requirements, how painful they are to actually live with, and — importantly — who’s still shipping releases versus who quietly stopped.

What got me started?

Started a deep dive for a current business project: automatically extracting business card data into a form for an archive, with minimal manual work. I focused mainly on Tika, paired with a local LLM for post-processing and JSON conversion. Docling worked well too, but felt too slow for my taste — though that’s obviously hardware-dependent ;) Lets dive deeper …

TL;DR: The Short Version

• Apache Tika 3.3.2 — still the boring, bulletproof answer for “extract text from 1,400 file formats.” CPU only, Java, REST server in one container. Not smart, just relentlessly reliable.
• Docling 2.119.0 + docling-serve 1.30.0 — the best all-rounder. MIT licensed, Linux Foundation governed, stable v1 REST API, runs on CPU or GPU. This is my default recommendation.
• MinerU 3.4.4 — the accuracy monster with a proper async API and load balancer. Slightly odd license, but permissive enough for basically everyone.
• DeepSeek-OCR-2 — the interesting one. Compresses pages into visual tokens instead of text tokens. Genuinely novel, needs real GPU.
• PaddleOCR-VL 1.6 — 0.9B parameters, 109 languages, tops OmniDocBench. Absurd accuracy-per-gigabyte.
• Watch the licenses. Several of the “open source” models have weights under restricted licenses, and one of them isn’t licensed in the EU at all.

Why Does OCR Need an API?

Quick Answer: An API turns OCR into a service your whole stack can share, instead of a heavy dependency every app has to install, load into memory, and keep updated.

Here’s the deal. Modern OCR models are big. A 7B vision language model takes twenty-something gigabytes of VRAM and thirty seconds to warm up. You do not want that loading inside your web app, your cron job, and your import script separately.

Running OCR behind an HTTP endpoint fixes a pile of problems at once:

• One model in memory, many clients. Your GPU loads the weights once instead of once per worker.
• Language independence. Your PHP app, your Node service, and your Python pipeline all just POST a file.
• Isolation. OCR is a notorious source of segfaults, OOM kills, and hung processes on malformed PDFs. Keep that in its own container where it can crash in peace.
• Scaling that makes sense. Queue the work, batch it, and scale the OCR tier independently from your app tier.
• Your data stays home. The whole reason to self-host is not shipping customer documents to a third party.

The Two Camps: Pipelines vs Vision Models

Every tool in this article falls into one of two architectural camps, and understanding the split saves you a lot of confusion.

Pipeline OCR chains together specialised models. One detects the layout, one finds text regions, one recognises characters, one reconstructs tables. Each stage is small and fast, most run happily on CPU, and you can debug them individually. The downside is error propagation — if layout detection splits a table wrong, everything downstream inherits the mistake.

VLM OCR throws the whole page image at a single vision language model and asks for structured Markdown back. It sees the page the way you do, so it handles weird layouts, handwriting, and mixed content far better. The catch: it needs a GPU, it’s slower per page, and when it fails it fails creatively — hallucinating plausible-looking text that was never on the page.

The 2026 answer, increasingly, is both. MinerU’s hybrid backend and Docling’s optional VLM path both let you run cheap pipeline extraction by default and escalate to a vision model only for the pages that need it. That’s the pattern worth copying.

The Classic Workhorses

Apache Tika — The One That Refuses to Die

Apache Tika has been around since 2007 and it’s still getting commits this month. Version 3.3.2 landed 16 July 2026, with 4.0.0-beta-1 out since 3 July 2026 — the 4.x line ports the SAX-based OOXML parsers back into 3.x and makes SAX the default, which is a meaningful memory win on large Office files.

Tika isn’t really an OCR engine. It’s a content detection and extraction framework that speaks about 1,400 file formats, and it shells out to Tesseract when it hits an image. That sounds unglamorous until you realise it’s exactly what most people actually need: point it at a directory of mixed garbage — DOCX, XLSX, EML, MSG, PDFs, JPEGs, ZIP archives — and get text and metadata back from all of it.

The killer feature is tika-server. One JAR, one port, a REST endpoint, no Python dependency hell. It runs on a Raspberry Pi. It runs in a 512MB container. It has never once made me install CUDA.

Heads up: Tika 3 enables Tesseract OCR by default, which can massively increase processing time on image-heavy PDFs. If you’re seeing 30-second parses on documents that used to take 200ms, that’s why. Set X-Tika-PDFOcrStrategy: no_ocr or disable the parser in tika-config.xml.

Perfect for: Search indexing, mail archives, mixed-format ingestion, anywhere you need breadth over depth, and any environment where “you need a GPU” is a non-starter.

Tesseract — Still the Engine Under Everything

Google’s Tesseract is the OCR equivalent of libc. Nobody gets excited about it, everything depends on it. Apache 2.0, and — genuinely surprising for a project this old — commits as recent as this week.

On clean, printed, single-column text it’s fast, accurate, and free. On a skewed phone photo of a receipt it’s a disaster. It has no concept of layout, reading order, or tables. It’s a character recogniser, and it’s a good one, but that’s the whole job description.

OCRmyPDF — The Right Tool for Scanned PDFs

OCRmyPDF (MPL-2.0) solves a specific problem beautifully: you have scanned PDFs and you want them to stay PDFs, but searchable. It adds an invisible text layer under the original image, so the page looks identical and Ctrl+F works.

This is the thing people keep reaching for a VLM to do, and they shouldn’t. If your goal is a searchable archive rather than structured data extraction, OCRmyPDF is faster, cheaper, and preserves the original document exactly. It’s what powers the OCR in Paperless-ngx, and there’s a reason for that.

docTR and EasyOCR — The Python Natives

docTR from Mindee (Apache-2.0) is a clean, well-maintained PyTorch/TensorFlow OCR library with proper text detection plus recognition. Actively developed, last push July 2026.

EasyOCR is the one I’d now flag with a caution. 80+ languages, dead simple API — but the last commit was December 2025. Eight months of silence in this particular corner of the ecosystem is a long time. It still works, it’s just no longer keeping pace.

The Document AI Layer

This is where the action is. These tools don’t just read characters — they reconstruct the document.

Docling — My Default Pick

Docling started at IBM Research Zurich and was donated to the LF AI & Data Foundation, where it was formally inducted on 29 April 2025. That governance detail matters more than it sounds — it means the project isn’t one product manager’s reorg away from disappearing.

It’s grown into a monster: MIT licensed, and version 2.119.0 shipped 10 August 2026. The input format list is genuinely ridiculous — PDF, DOCX, PPTX, XLSX, HTML, EPUB, images, LaTeX, ODF, XBRL financial filings, EML and MSG email, plus audio (WAV, MP3) and video (MP4, MKV, WebM) with transcription. Output goes to Markdown, HTML, lossless JSON, WebVTT and DocTags.

For OCR it doesn’t reinvent anything — it wraps Tesseract, RapidOCR and Surya, so you pick your engine. And if you want full VLM parsing, it ships Granite-Docling-258M — released January 2026 under Apache 2.0, replacing the old SmolDocling-256M preview. At 258 million parameters with a Granite 3 backbone and SigLIP2 vision encoder, it’s small enough to run on a potato and still holds its own against models many times the size.

The API story got properly good this year. docling-serve hit a stable v1 REST API in v1.18.0 (7 May 2026) and is now at 1.30.0 (7 August 2026). Endpoints live under /v1, there’s a Swagger UI at /docs, and an actual interactive playground at /ui if you flip an env var. Prebuilt CPU, GPU and CUDA 13 container images are published on Quay.

Perfect for: Basically everyone. It’s the tool with the fewest sharp edges, the most permissive license, and the widest format coverage.

MinerU — Accuracy and a Real Async API

MinerU from OpenDataLab is one of the most widely deployed parsers in this roundup, and version 3.4.4 shipped 10 July 2026. It converts PDFs, images, DOCX, PPTX and XLSX into Markdown and JSON, strips headers and footers and page numbers, converts formulas to LaTeX, tables to HTML, and pulls images out with their captions.

What makes it stand out is the honesty of its backend matrix. The project publishes its own OmniDocBench numbers per backend, which is refreshingly transparent:

Backend	OmniDocBench	CPU support	Min VRAM
pipeline	86.47	Yes	4 GB
hybrid (high)	95.39	No	8 GB
hybrid (medium)	95.26	No	8 GB
vlm	95.30	No	8 GB

Note that hybrid at effort=medium gives you 35–220% more speed for a 0.13 point accuracy drop. That’s the setting you actually want in production, and I appreciate a project that tells you so.

The v3.4 release swapped in PP-OCRv6 for the pipeline backend, which the team measures at roughly 11% better OCR accuracy on OmniDocBench v1.6 and about twice the speed. The API server offers POST /file_parse for synchronous work and POST /tasks for proper async submission with status polling — and there’s a mineru-router component that load-balances across multiple workers.

The catch: the license isn’t plain Apache 2.0. See the licensing section below — though for the overwhelming majority of readers it’s a non-issue.

Unstructured — The RAG Plumbing

Unstructured (Apache-2.0, actively maintained) sits slightly apart from the rest. Its job is getting documents into a chunked, embedded, vector-database-shaped form, with connectors for a long list of sources and destinations. The OCR is decent rather than best-in-class, but if your endpoint is a vector store rather than clean Markdown, it removes a lot of glue code.

Marker — Fast, With License Strings Attached

Marker from Datalab hit 2.0.0 on 20 July 2026 and is seriously quick. The project reports 7.4 pages/second in fast mode on GPU, 2.9 pages/second balanced, and a wild 23.7 pages/second on CPU when OCR is skipped entirely on text-native PDFs. It scores 76.0% on olmOCR-bench.

It ships a FastAPI server via marker_server, though the project’s own docs describe it as “not very robust” for production scale — worth taking at face value and putting a queue in front of.

The catch: the code is Apache 2.0 but the model weights are under a modified AI Pubs OpenRAIL-M license — free for research, personal use, and companies under $5M in funding or revenue. Above that, you need a commercial license.

The Vision Language Models

DeepSeek-OCR and DeepSeek-OCR-2 — The Genuinely Novel One

DeepSeek-OCR landed 20 October 2025 with an idea that made a lot of people sit up: what if you stopped treating OCR as a text problem?

The premise is “contexts optical compression.” A page of text costs a lot of text tokens, but a picture of that same page costs far fewer visual tokens. DeepSeek showed roughly 97% decoding accuracy at 10x compression, still holding around 60% at 20x. The implication is bigger than OCR — it’s a possible route to stuffing far more context into an LLM by rendering it as images.

DeepSeek-OCR-2 followed on 27 January 2026 under the subtitle “Visual Causal Flow,” introducing DeepEncoder V2. Where the original encoder scanned image patches in fixed raster order, V2 dynamically reorders segments based on semantic content — closer to how your eyes actually jump around a page than to a typewriter. That translates into better reading order on multi-column layouts, forms, and mixed content. Reported OmniDocBench performance is around 91.09, roughly 3.7 points up on the first version.

Both run on vLLM or plain Transformers, wanting CUDA 11.8+ and PyTorch 2.6.0. On an A100-40G the original hits about 2,500 tokens/second on concurrent PDF processing. Note the license changed between versions — OCR-1 is MIT, OCR-2 is Apache 2.0.

Heads up: the maintenance picture tells a story. OCR-2’s last repository push was February 2026, and it never picked up the community momentum the original did. The research is fascinating, but OCR-2 has not seen the same sustained engineering attention as Docling or MinerU. Treat it as a strong model rather than a maintained product.

PaddleOCR-VL — Absurd Accuracy Per Gigabyte

PaddleOCR is the elephant in the room, Apache 2.0, and its VL line is the current benchmark leader. PaddleOCR-VL-1.6 arrived 28 May 2026 reporting 96.33% on OmniDocBench v1.6, following 1.5 in January at 94.5% on v1.5.

The remarkable part is the size. It’s a 0.9B parameter model — a NaViT-style dynamic resolution vision encoder bolted to the lightweight ERNIE-4.5-0.3B language model — covering 109 languages including Chinese, Japanese, Arabic, Hindi and Thai. It serves through vLLM, SGLang or FastDeploy, and fits comfortably in 8GB of VRAM.

If you want the best accuracy-to-hardware ratio available under a real open source license right now, this is it.

Chandra — The Handwriting Specialist

Chandra 2 from Datalab arrived in March 2026 and is the one to beat on messy human documents. It scores 85.8% on olmOCR-bench, with 92.1% on tables and 89.1% on math, plus 77.8% across a 43-language multilingual benchmark. Throughput is about 1.44 pages/second on a single H100 80GB.

If your corpus is handwritten forms, historical scans, or genuinely gnarly tables, Chandra is the strongest open-weights option. Serving is a one-liner via chandra_vllm.

The catch: same as Marker, but tighter. Code is Apache 2.0, weights are modified OpenRAIL-M — free for research, personal use and startups under $2M in funding or revenue, and explicitly not usable to compete with Datalab’s own API.

olmOCR — The Fully Open One

olmOCR from Ai2 deserves credit for being the most genuinely open entry here. Apache 2.0 across the board, and they released not just weights but the training data — olmOCR-mix-1025, 270,000 PDF pages — plus the olmOCR-Bench that half this article’s numbers are quoted against.

olmOCR 2 is built on Qwen2.5-VL-7B and scores 82.4 on its own benchmark, up from 78.5. The clever bit is the training method: reinforcement learning against verifiable unit tests that assert things like “table structure preserved” and “reading order consistent,” rather than fuzzy similarity scores.

Heads up: the last repository push was March 2026, and at 7B parameters it’s the heaviest model here for benchmark scores that newer 1B models now beat. Brilliant research, increasingly outpaced in practice.

dots.ocr — Multilingual on a Budget

dots.ocr from RedNote packs layout detection and content recognition into a single 1.7B model under a clean MIT license. Its real strength is low-resource languages, where it beats considerably larger models. The 1.5 release was rebranded to dots.mocr in March 2026.

Like several entries here, momentum has slowed — last push was March 2026. Still a solid, permissively licensed choice if multilingual coverage is your priority and you don’t need bleeding edge.

HunyuanOCR — Great Model, Read the License First

Tencent’s HunyuanOCR (now at 1.5) is a 1B model posting excellent OmniDocBench numbers, and it’s actively developed — last push July 2026.

And if you’re reading this from Germany, Austria, Switzerland or anywhere else in Europe, you probably can’t use it. The license opens with this, in capitals:

“THIS LICENSE AGREEMENT DOES NOT APPLY IN THE EUROPEAN UNION, UNITED KINGDOM AND SOUTH KOREA AND IS EXPRESSLY LIMITED TO THE TERRITORY, AS DEFINED BELOW.”

“Territory” is defined as worldwide excluding the EU, UK and South Korea, and the agreement states plainly that use outside the Territory “is unlicensed and unauthorized.” Disputes go to a court in Hong Kong. This is a standard clause across Tencent’s Hunyuan model family, almost certainly there to sidestep EU AI Act obligations — but the practical effect is the same: for European deployments, this model is off the table. Ignore anyone who calls it open source without that asterisk.

Feature Comparison

Tool	Type	Tables	Formulas	Handwriting	Built-in API	Formats in
Apache Tika	Pipeline	Basic	No	Weak	tika-server REST	~1,400
Docling	Both	Strong	Yes	Via VLM	docling-serve v1	20+
MinerU	Both	Strong	LaTeX	Via VLM	Sync + async	5
Marker	Both	Strong	Yes	Moderate	FastAPI (basic)	7
DeepSeek-OCR-2	VLM	Strong	Yes	Good	Via vLLM	Images/PDF
PaddleOCR-VL 1.6	VLM	Strong	Yes	Good	Via vLLM/SGLang	Images/PDF
Chandra 2	VLM	Best	Yes	Best	chandra_vllm	Images/PDF
olmOCR 2	VLM	Good	Yes	Moderate	Via vLLM	PDF
dots.ocr	VLM	Good	Yes	Moderate	Via vLLM	Images/PDF
Tesseract	Engine	No	No	Weak	None	Images
OCRmyPDF	Pipeline	No	No	Weak	None	PDF

Server Requirements: What Will This Actually Cost You?

Quick Answer: Pipeline tools run fine on a 2-core VPS with no GPU. VLM OCR realistically needs 8GB VRAM minimum, and 24GB if you want the 7B models.

This is where a lot of homelab plans quietly die. Here’s the honest hardware picture. MinerU’s figures come from the project’s own documentation; the VLM estimates assume bf16 weights without quantisation.

Tool / Model	GPU needed?	Typical VRAM	System RAM	Notes
Apache Tika	No	—	512MB–2GB	JVM, Java 11+
Tesseract	No	—	256MB	Scales with CPU cores
OCRmyPDF	No	—	1–2GB	Parallelises per page
Docling (pipeline)	Optional	—	4–8GB	GPU speeds it up, isn’t required
Docling + Granite-Docling	Optional	~2GB	8GB	258M params, CPU-tolerable
MinerU (pipeline)	No	4GB	16GB min	Project-stated minimum
MinerU (hybrid/vlm)	Yes	8GB	32GB rec.	Volta or newer, or Apple Silicon
PaddleOCR-VL 1.6	Yes	~8GB	16GB	0.9B params
dots.ocr	Yes	8–16GB	16GB	1.7B params
DeepSeek-OCR-2	Yes	16–24GB	32GB	~3B, CUDA 11.8+, torch 2.6.0
Chandra 2	Yes	24GB+	32GB	Benchmarked on H100 80GB
olmOCR 2	Yes	24GB+	32GB	Qwen2.5-VL-7B base

The sweet spot for a homelab in 2026 is a used RTX 3090 or 4090 with 24GB, which runs anything on this list. If you’ve got 8–12GB, PaddleOCR-VL and MinerU’s hybrid backend are your friends. If you’ve got no GPU at all, Tika and Docling’s pipeline mode will still serve you well — and honestly, for a lot of workloads that’s completely fine.

Licensing: Read the Fine Print

Quick Answer: Several popular OCR models ship Apache-2.0 code with restricted model weights. “Open source” on the README does not always mean open source in the LICENSE.

This is the section I’d most want you to actually read. The pattern across 2025–2026 has been permissive code paired with restricted weights, and it catches people out at exactly the wrong moment — after they’ve built the pipeline.

Project	Code license	Weights license	Real-world restriction
Apache Tika	Apache-2.0	—	None
Docling	MIT	Apache-2.0	None. Cleanest in the roundup
Tesseract	Apache-2.0	Apache-2.0	None
OCRmyPDF	MPL-2.0	—	Weak copyleft on the files themselves
PaddleOCR-VL	Apache-2.0	Apache-2.0	None
olmOCR	Apache-2.0	Apache-2.0	None. Training data published too
dots.ocr	MIT	MIT	None
DeepSeek-OCR-2	Apache-2.0	Apache-2.0	None (OCR-1 was MIT)
MinerU	Apache-2.0 +	Same	Commercial license above 100M MAU or $20M/month revenue; must credit MinerU in online services
Marker	Apache-2.0	OpenRAIL-M mod.	Free under $5M funding/revenue
Chandra 2	Apache-2.0	OpenRAIL-M mod.	Free under $2M funding/revenue; no competing with Datalab’s API
HunyuanOCR	Tencent Hunyuan Community		Not licensed in the EU, UK or South Korea

MinerU’s extra terms are worth a second look because they read scarier than they are. The thresholds are 100 million monthly active users or $20 million in monthly revenue. If you cross either of those, congratulations, you can afford a lawyer. The attribution requirement for public online services is the clause that actually applies to normal people — put “Powered by MinerU” somewhere visible and you’re fine.

Who’s Actually Shipping? Release Activity

Benchmarks age badly. Maintenance doesn’t. Here’s where each project stood in August 2026, pulled straight from the repositories.

Project	Latest version	Released	Last repo activity	Momentum
Docling	2.119.0	10 Aug 2026	12 Aug 2026	Very strong
docling-serve	1.30.0	7 Aug 2026	10 Aug 2026	Very strong
MinerU	3.4.4	10 Jul 2026	11 Aug 2026	Very strong
Apache Tika	3.3.2	16 Jul 2026	11 Aug 2026	Strong, steady
Tesseract	5.x	—	12 Aug 2026	Strong for its age
Unstructured	0.25.2	3 Aug 2026	11 Aug 2026	Strong
Marker	2.0.0	20 Jul 2026	7 Aug 2026	Strong
OCRmyPDF	—	—	6 Aug 2026	Strong
Surya	0.22.1	20 Jul 2026	23 Jul 2026	Healthy
PaddleOCR	VL 1.6	28 May 2026	22 Jul 2026	Healthy
HunyuanOCR	1.5	—	29 Jul 2026	Healthy (EU-blocked)
Chandra	2	Mar 2026	26 Jun 2026	Moderate
olmOCR	0.4.27	12 Mar 2026	25 Mar 2026	Slowing
dots.ocr	1.5 / mocr	Mar 2026	24 Mar 2026	Slowing
DeepSeek-OCR-2	—	27 Jan 2026	3 Feb 2026	Research drop
EasyOCR	—	—	5 Dec 2025	Stalled

The pattern is clear enough. The infrastructure projects — Docling, MinerU, Tika, Unstructured — ship constantly. The research model drops land with a splash and then go quiet. That’s not a criticism of the research, it’s just a reason to build your pipeline around the infrastructure and treat the models as swappable parts.

Standing These Up: Actual Code

Apache Tika Server in Docker

The fastest useful OCR endpoint you can deploy. One container, done.

Use the -full image tag — the slim one doesn’t bundle Tesseract and you’ll wonder why OCR silently does nothing.

Docling Serve

Prebuilt images on Quay, CPU and GPU flavours, with a browser playground included.

Here my stack with a bit of optimization for lower hardware:

Hit http://localhost:5001/ui for the playground and /docs for full Swagger docs. Being able to hand a colleague a URL where they can drag a PDF in and see what comes out is worth more than it sounds.

MinerU With Sync and Async Endpoints

The async endpoint is the one you want in production. Synchronous HTTP calls and 400-page scanned PDFs are a bad marriage — you’ll hit proxy timeouts long before the parse finishes.

Serving a VLM With vLLM

Most of the vision models expose themselves through vLLM’s OpenAI-compatible server, which means one pattern covers PaddleOCR-VL, DeepSeek-OCR-2, olmOCR and dots.ocr.

From there it’s a standard chat completion with an image attached:

A Tiered Stack With Docker Compose

Here’s the pattern I’d actually run: cheap extraction by default, expensive extraction on demand. Tika handles the long tail of formats on CPU, Docling handles structured document parsing, and you only pay for the GPU tier when a document earns it.

Route on document type: text-native PDFs and Office files go to Tika, structured documents go to Docling, and anything scanned or handwritten escalates to MinerU on the GPU. Most corpora are 80% easy documents, so this keeps your GPU mostly idle and your throughput high.

Which One Should You Actually Pick?

• You just need text out of many file formats — Apache Tika. No GPU, no Python, one container, done.
• You’re building a RAG pipeline — Docling. Best format coverage, cleanest license, stable API, and it scales down to CPU when you need it to.
• You need the best accuracy you can self-host — MinerU on the hybrid backend at effort=medium, or PaddleOCR-VL 1.6 if you want a pure VLM.
• You have 8GB of VRAM or less — PaddleOCR-VL 1.6 or MinerU’s pipeline backend.
• You have no GPU at all — Tika, or Docling in pipeline mode. Both are genuinely fine.
• Your documents are handwritten or historical — Chandra 2, assuming you’re under the revenue threshold.
• You need searchable PDFs, not structured data — OCRmyPDF. Stop overthinking it.
• You need lots of languages — PaddleOCR-VL (109) or dots.ocr for low-resource scripts.
• You need an unambiguously open license for a commercial product — Docling, PaddleOCR-VL, olmOCR or dots.ocr. Avoid the OpenRAIL-M weights.
• You’re in the EU — skip HunyuanOCR entirely, however good the benchmarks look.

What I’d Deploy Tomorrow

If someone handed me a fresh server today and said “make our documents searchable,” I’d run Tika and Docling side by side on CPU and not add a GPU until something proved it needed one. That covers a genuinely surprising percentage of real workloads, costs nothing to run, and neither project is going anywhere.

When the GPU tier becomes necessary — scanned archives, handwriting, dense tables — MinerU’s hybrid backend is the best balance of accuracy, hardware appetite and API maturity available right now. PaddleOCR-VL is the one I’d pick if VRAM were tight.

And the thing I’d resist? Building the pipeline around whichever model topped a benchmark this month. The benchmark leader has changed four times since January. Build around the serving layer, keep the model behind an interface you can swap, and let the leaderboard fight it out without you.

Glossary

OCR (Optical Character Recognition)

Turning pictures of text into actual text. The classic definition stops there, which is why the newer tools describe themselves as document parsers instead.

VLM (Vision Language Model)

A model that takes images and text as input and produces text. For OCR it means the model sees the whole page at once, so it understands layout rather than just characters.

Pipeline OCR

The traditional approach: separate models for layout detection, text detection, character recognition and table reconstruction, chained together. Fast and CPU-friendly, but errors compound down the chain.

OmniDocBench

A document parsing benchmark from OpenDataLab covering layout, tables, formulas and reading order across document types. Currently the number most projects quote.

olmOCR-Bench

Ai2’s benchmark, built as deterministic unit tests that assert specific properties like “table structure preserved” rather than fuzzy text similarity. Harder to game.

DocTags

IBM’s markup format for describing every element on a page — tables, charts, formulas, captions — along with position and relationships. The native output of Granite-Docling.

Contexts Optical Compression

DeepSeek’s idea that rendering text as an image and encoding it as visual tokens is cheaper than text tokens. Roughly 10x fewer tokens at about 97% accuracy.

vLLM

The de facto open source inference server for large models. Gives you an OpenAI-compatible HTTP API in front of local weights, which is how most OCR VLMs get served.

VRAM

Memory on your graphics card. The hard limit on which models you can run. 8GB gets you the small VLMs, 24GB gets you everything in this article.

OpenRAIL-M

A model license that grants broad rights but attaches use restrictions. Common on OCR weights in 2026, usually with revenue thresholds attached. Not an OSI-approved open source license.

Reading order

The sequence text should be read in. Trivial on a single column, genuinely hard on a magazine spread with sidebars and pull quotes. The main thing VLMs improved over pipelines.

RAG (Retrieval Augmented Generation)

Feeding an LLM relevant chunks of your own documents at query time. The reason document parsing got hot — garbage extraction means garbage answers.

FAQ