Design a Document Processing Pipeline

You're asked to design a system that turns a large volume of unstructured documents into structured data. Customers upload PDFs, scans, and photographs of invoices, contracts, receipts, and medical forms, and the system extracts specific fields from each one, validates them, and hands the result to a downstream system. It should handle a single document uploaded through the UI and a batch of two hundred thousand dropped in overnight, with the same code path.

Each document passes through stages with different failure modes. A scan may be unreadable, a model may be uncertain, or a page may be rotated. Let's work through how the system tracks and retries hundreds of thousands of independent workflows without repeating expensive work.

Clarify the requirements

• What are we extracting? Named fields from known document types require a different system from open-ended understanding. Clarify this first.
• How accurate does it need to be? If errors are costly, include human review as a primary component rather than an afterthought.
• What's the latency expectation? Seconds for an interactive upload, hours for an overnight batch. Supporting both means priority rather than two systems.
• How varied are the inputs? Clean digital PDFs and photographs of crumpled receipts are different problems, and mixed input is the realistic assumption.
• Is the extraction model ours? Usually you can treat it as an external service with a latency and a cost per call, which keeps the design focused on the pipeline.

Assume: known document types with named fields, accuracy that justifies human review, mixed interactive and batch traffic, and a model accessed as a service.

Back-of-envelope numbers

• Volume: $1\text{M documents/day} \approx 12/\text{sec}$ average, with overnight batches producing peaks near $500/\text{sec}$
• Pages: $1\text{M docs} \times 5 \text{ pages} = 5\text{M pages/day}$, which is the real unit of work
• Storage: $1\text{M docs} \times 2\text{ MB} = 2\text{ TB/day}$ of originals, retained for the contractual period
• Model cost at $\$0.01\text{/page}$: $5\text{M pages} \times \$0.01 = \$50\text{k/day}$, which dominates every other cost in the system

The model call dominates cost. Use caching, cheap prefilters, and durable checkpoints to avoid unnecessary calls and paid work during retries.

High-level architecture

Components

1. Intake API. Accepts one document or a batch manifest, returns immediately with a job ID.
2. Object storage. The original file and every intermediate artifact, since documents are far too large to pass through queues.
3. Job store. The source of truth for which stage each document has reached and what it produced.
4. Stage queues. One per stage, so each stage scales independently.
5. Stage workers. Classify, split, preprocess, extract, validate.
6. Extraction model. External, slow, and the dominant cost.
7. Review queue. Documents whose extraction wasn't confident enough to accept.
8. Structured output. Validated records handed to the downstream system.

Deep dive 1: the pipeline as independent stages

Do not put every stage in one worker. The stages have different costs and failure modes, and coupling them forces cheap work to repeat when an expensive stage fails.

Split them, with each stage reading from its own queue and writing its result to object storage:

• Classify. What kind of document is this? Cheap, fast, and it determines everything downstream.
• Split. Separate a multi-document scan into individual documents and pages. Purely mechanical.
• Preprocess. Deskew, denoise, correct rotation, normalize resolution. Meaningfully improves extraction accuracy for a small amount of CPU.
• Extract. The model call. Slow, expensive, and the stage that needs its own scaling.
• Validate. Check the extracted fields against rules: does the invoice total equal the sum of line items, and is the date plausible? Cheap and catches a surprising fraction of errors.

What this buys you is independent scaling and independent failure. Extraction needs far more capacity than classification, and a model provider outage backs up one queue while the others keep draining. A retry costs one stage rather than the whole document, which matters enormously when one stage costs a hundred times more than the others.

State lives in the job store, not the queue. The queue says "this document is ready for extraction"; the job store says "this document has been classified as an invoice, split into three pages, and preprocessed, and here are the artifact locations." That separation is what lets you answer "where is document X?" and resume a partially processed document without redoing paid work.

Deep dive 2: making retries cheap

Every stage will fail sometimes, and the naive retry re-runs the expensive part.

Key artifacts by content, not by attempt. Each stage writes to a deterministic location derived from the document ID and the stage name, so a re-run overwrites rather than appends and a completed stage is detectable by the existence of its output. A worker that picks up a retried job checks for existing artifacts before doing work, which turns "retry extraction" into "skip extraction, it's already done."

Hash the document to deduplicate. The same invoice submitted twice, a common occurrence in real batches, hashes identically, and the extraction result can be reused outright. On real document workloads this eliminates a meaningful fraction of model calls at essentially no cost.

Classify failures before retrying. A model provider timeout is retryable with backoff. A corrupt or password-protected PDF is not, and retrying it three times wastes time and produces the same failure. Route permanent failures to a dead-letter path with the reason attached, so a human can see a list of genuinely broken documents rather than an undifferentiated pile.

Leases with heartbeats keep a document from being stranded when a worker dies mid-extraction, using the same mechanism described in async jobs and workers.

Deep dive 3: confidence and human review

An extraction system without a confidence signal is a system that fails silently, and silent failure is the expensive kind when the output is an invoice total that goes into someone's accounting.

Every extracted field should carry a confidence score, and the pipeline routes on it rather than accepting everything:

• High confidence: accepted automatically and passed downstream.
• Medium: queued for human review, with the extracted value pre-filled and the source region of the document highlighted so the reviewer confirms rather than retypes.
• Low, or validation failed: flagged for full manual entry.

Two design consequences follow. The review queue is a real product surface, not a dumping ground: it needs prioritization by document age and business value, and a reviewer interface that makes confirmation fast, because review throughput is a hard capacity limit on the whole system.

Treat corrections as training data. Each human fix is a labeled example of a model error, so capture corrections systematically and feed them into evaluation and future training.

Deep dive 4: mixing interactive and batch traffic

A single document uploaded through the UI needs a result in seconds. Two hundred thousand documents dropped overnight need to finish by morning. Running two separate systems duplicates every piece of logic, and running one FIFO queue means the interactive upload waits behind the batch.

Priority queues per stage solve it with one code path. Interactive documents enter at high priority and are picked up first; batch documents fill the remaining capacity. Since the batch has hours of slack, being preempted costs it nothing measurable.

Add two refinements:

• Reserve a slice of capacity for interactive work rather than relying on priority alone, so a batch large enough to saturate the fleet can't starve the interactive path entirely.
• Rate-limit against the model provider deliberately, since it has its own quota. Spending the whole quota on an overnight batch means the next morning's interactive uploads get throttled, which is the failure the customer actually notices.

Common pitfalls

• One worker doing every stage. Stages have different costs and failure modes, and coupling them means a retry re-runs the expensive part.
• Non-deterministic artifact paths. Retries then duplicate work and produce inconsistent output.
• No confidence scores. The system fails silently, which is the most expensive failure mode here.
• Treating human review as an afterthought. Review throughput is a hard limit on the system's overall capacity.
• One queue for interactive and batch. The overnight batch delays the upload someone is watching.

Leveling signals

Related lessons

Design Facebook Data ExportMedium

Gather a user's data from many services into one downloadable archive, surviving partial failure.

Design an LLM Query Batching SystemHard

Batch inference requests to maximize GPU utilization while users wait synchronously.