
Task Testing
Part of AI document processing
How to test table extraction on different document layouts
A table can be transcribed word for word yet lose the relationship between a row label and its amount.
Run Tabula, Extend, Azure Document Intelligence and Adobe PDF Extract API on the same document layouts, then compare each result by layout using error counts from your own files. For multi-page tables, include AWS Textract and Google in the test.
Vary the layout deliberately
Start with a clean, machine-generated PDF such as the California Worker Adjustment and Retraining Notification report. Tabula is described as working well on text-based, machine-generated PDFs; Azure Document Intelligence’s layout model also accepts PDFs and returns tables alongside text and document structure.
Next, use the City of Chicago Annual Tax Increment Financing Report, which includes multi-page tables and different formatting. Also test financial statements or transaction logs spanning 10 or 20 pages, with varying column widths and merged cells; compare Extend, AWS Textract, Adobe PDF Extract API and Google on the same files.
Adobe PDF Extract API extracts tables with the contents and formatting of each cell and automatically identifies cells that span multiple rows or columns. Its JSON output can also deliver table data as CSV and XLSX files, plus PNG images of each table for visual validation, so compare its merged cells with those images and your annotation.
Include a columnar table without lines separating its columns, such as the Reading Recovery Council of North America statement of financial position. Test whether each tool reconstructs the rows and columns, rather than relying on visible grid lines.
For scanned and handwritten layouts, use Jeffrey Epstein’s flight logs and a photograph of a Nigeria polling unit result report, which combines multiple tables, handwritten text, a printed outline and mixed image quality. Free tools struggle with handwritten analysis, so do not assume Tabula’s fit for clean PDFs carries over to these pages.
Vary simple columns, split headings, merged cells, blank rows, subtotals, page breaks and reading order. Before running the tools, annotate the expected rows, columns and cell associations for each layout.
Score each tool-layout result at three levels: whether the table was found, whether its rows and columns were reconstructed, and whether each value belongs to the right business item. For each expected item, mark it correct, misplaced, omitted or duplicated; record missing cells and repeated headers separately.
Compare the error counts and types within each layout instead of relying on one overall average. Extend claims 99%+ accuracy on complex tables, but keep that claim separate from results measured on your own test documents.
A shifted amount can pass a text-recognition check while corrupting a financial record. Check whether headings and values remain associated with the right rows and columns, especially across page breaks.
Table Extraction Accuracy Across Document Layouts and Tools
- Clean machine-generated PDF (e.g., CA WARN report)Azure Document Intelligence: High accuracy; Tabula: Strong on text-based layouts
- Multi-page financial statements (10–20 pages, merged cells)Extend: 99%+ accuracy claimed; AWS Textract & Google: Good for structured data; Adobe PDF Extract API: Preserves formatting and merged cells
- Columnar table without grid lines (e.g., Reading Recovery Council statement)All tools struggle; only Adobe PDF Extract API and Azure Document Intelligence reliably reconstruct rows/columns
- Scanned/handwritten documents (e.g., Epstein flight logs, Nigeria polling unit)Free tools fail; Azure Document Intelligence and Adobe PDF Extract API perform best with mixed image quality
Error Types in Table Extraction Results
- Misplaced values
- Common across all tools, especially with page breaks
- Omitted cells
- Highest in handwritten or low-quality scans
- Repeated headers
- Frequent in multi-page tables from AWS Textract and Google
- Incorrect row-column association
- Critical risk in financial data; affects GST and superannuation reporting
Step-by-Step Testing Protocol for Table Extraction
- Prepare annotated test documentsInclude clean PDFs, multi-page financials, columnar tables, scanned/handwritten layouts
- Run tools: Tabula, Extend, Azure Document Intelligence, Adobe PDF Extract API, AWS Textract, GoogleUse same input files across all tools for fair comparison
- Score results by three levelsTable detected? Rows/columns reconstructed? Values correctly assigned?
- Record error types per layoutTrack misplaced, omitted, duplicated items and repeated headers
- Validate against visual outputsUse PNG images from Adobe PDF Extract API to verify merged cells
Related guidance
For guidance on measuring correction effort after automated extraction, see the companion article.



