One of the more interesting AI projects I built was a data parsing pipeline: take a pile of documents (PDFs, primarily), extract structured data from them, and store that data in a database for downstream use. The documents were messy - different formats, inconsistent layouts, varying levels of completeness. Traditional parsing (regex, fixed-position extraction) had already been tried and failed.
Here's the architecture and what I learned from building it.
The Problem With Traditional Parsing
Traditional document parsing approaches - regular expressions, positional extraction, template matching - work when documents are consistent. They fail when documents vary.
Real-world documents vary. Vendors use different formats. Old documents have different layouts than new ones. Scanned documents have OCR artifacts. Some fields are present, some are absent, some are in unexpected locations.
An AI model can handle this variability. It reads the document the way a human would, understands context, and can extract the right value even when the field label is worded differently or the field appears in an unexpected location. This is exactly what language models are good at.
The tradeoff is cost and latency. AI extraction is slower and more expensive than regex. The architecture needs to account for this.
Pipeline Architecture
The pipeline I built has four stages:
Stage 1: Document ingestion. PDFs arrive via a monitored folder or API endpoint. Each document is stored in S3 with a unique key, and a record is created in the database with the document's location, ingestion timestamp, and status (pending).
Stage 2: Text extraction. PDFBox extracts raw text from the PDF. For most PDFs, this produces clean text. For scanned PDFs (image-only), it produces nothing - those require OCR, which is a separate path. Text is stored alongside the document record.
Stage 3: AI extraction. The extracted text is submitted to the OpenAI Assistants API with a prompt that specifies exactly what fields to extract and in what format. The response is a structured JSON object. The JSON is parsed and validated.
Stage 4: Storage and review. Extracted fields are written to the database. Records that the model flagged as uncertain (low-confidence fields, missing required fields) are queued for human review.
The Prompt Is the Core Logic
The extraction prompt is where the real work is. A well-designed prompt produces clean, structured output. A vague prompt produces inconsistent output that requires downstream cleanup.
A few patterns that worked:
Specify the output schema explicitly. Include a JSON template in the prompt with every field the model should extract. Include field names, data types, format expectations, and examples. Don't leave the model to guess what format you want.
Include validation rules in the prompt. If a date field should always be in ISO 8601 format, say so in the prompt. If a numeric field should never be negative, say so. The model will apply these constraints and flag violations.
Handle missing fields explicitly. Tell the model what to do when a field isn't present in the document: use null, use a specific sentinel value, or flag the record for review. If you don't specify this, you'll get inconsistent behavior across documents.
Ask for confidence. Include a confidence field in the output schema and ask the model to rate its confidence in the extraction (high/medium/low). Use low-confidence extractions as the queue for human review rather than trying to automatically filter them.
Here's a simplified version of the structure:
Extract the following fields from the document text below.
Return your response as a JSON object with exactly these fields.
If a field is not present in the document, use null.
Rate your overall confidence in the extraction as "high", "medium", or "low".
Fields to extract:
- invoice_number (string): The invoice or reference number
- invoice_date (string, ISO 8601): The date the invoice was issued
- vendor_name (string): The name of the vendor
- total_amount (number): The total amount due
- line_items (array of objects): Each line item with description, quantity, and unit_price
Confidence: high | medium | low
Reason for low confidence (if applicable): [explain any uncertain or missing fields]
Document text:
[text goes here]
OpenAI Assistants API
The OpenAI Assistants API is suited for this use case because it supports a conversation model: you can send multiple messages with follow-up questions if the first extraction is incomplete. You can also use file attachments directly rather than embedding the document text in the prompt.
A few things to know about the Assistants API:
Runs are asynchronous. You create a run, poll for completion, then retrieve the result. Unlike the standard completions API (which returns synchronously), the Assistants API is async by design. Build your pipeline to poll and handle delays.
Thread management. Each conversation is a thread. For a batch processing pipeline, you typically create one thread per document, run the extraction, store the results, and then discard the thread. Don't reuse threads across unrelated documents.
Token limits apply. The Assistants API has the same context window limits as the underlying model. Large documents may need to be chunked. For most business documents (invoices, contracts, forms), chunking isn't necessary and the full text fits comfortably. For long documents (reports, books), you need a chunking strategy.
Handling Scanned PDFs
PDFBox can extract text from text-based PDFs. It returns nothing useful for image-based PDFs (scans).
For scanned documents, you need OCR. Options:
- Azure AI Document Intelligence (formerly Form Recognizer) handles both text and image PDFs, extracts text, and can also identify document structure (tables, form fields, key-value pairs).
- Tesseract is an open-source OCR engine that can be called from Java via Tess4J. It works well for clean scans, less well for low-quality images.
- Convert to image first (using PDFBox's rendering capabilities), then pass the image to an OCR service.
The practical approach is to detect whether PDFBox extraction returns meaningful text (more than some minimum threshold of characters). If yes, use it directly. If no, route to OCR.
The Human Review Queue
Not every extraction will be complete and confident. A well-designed pipeline doesn't try to hide this, but rather surfaces it.
Every record gets a status field: auto_accepted, pending_review, or rejected. Extractions with all required fields at high confidence go to auto_accepted directly. Extractions with missing required fields or low confidence go to pending_review.
The review interface shows the original document alongside the extracted fields, with flagged fields highlighted. A reviewer can correct values, mark the extraction as complete, or reject the document if it's unprocessable.
Track the review rate over time. If it's high, your prompt or your input document quality has a problem. If it drops over time as you tune the prompt, that's a success metric.
Cost Management
AI extraction has a per-token cost that doesn't exist with regex. For a high-volume pipeline, this adds up quickly.
A few levers:
Extract text before sending to the model. Don't send the raw PDF to the model - extract the text first and send only the text. PDFs contain a lot of non-text content that contributes to token count without helping the extraction.
Truncate aggressively. Most of the information you need is in the first few pages of most documents. If your extraction fields are all in the header and first page, truncate to 2-3 pages before sending. This can cut token costs by 70-80% for long documents.
Cache document text. If a document might need to be re-extracted (due to prompt changes or errors), store the extracted text so you don't need to re-extract from the PDF. PDFBox extraction is cheap; re-extracting text from the same PDF repeatedly is waste.
Batch during off-peak hours. If the pipeline doesn't need real-time results, schedule batch processing during off-peak hours when API rate limits are less likely to be a constraint.