Comparing table extraction from different document layouts

Run Tabula, Extend and Azure Document Intelligence on the same document layouts, then compare each result by layout. For multi-page tables, include AWS Textract, Adobe and Google in the comparison.

Vary the layout deliberately

Start with a clean, machine-generated PDF such as the California Worker Adjustment and Retraining Notification report. Tabula is described as working well on text-based, machine-generated PDFs; Azure Document Intelligence’s layout model also accepts PDFs and returns tables alongside text and document structure.

Next, use the City of Chicago Annual Tax Increment Financing Report, which includes multi-page tables and different formatting. Also test financial statements or transaction logs spanning 10 or 20 pages, with varying column widths and merged cells; compare Extend, AWS Textract, Adobe and Google on the same files.

Include a columnar table without lines separating its columns, such as the Reading Recovery Council of North America statement of financial position. Test whether each tool reconstructs the rows and columns, rather than relying on visible grid lines.

For scanned and handwritten layouts, use Jeffrey Epstein’s flight logs and a photograph of a Nigeria polling unit result report, which combines multiple tables, handwritten text, a printed outline and mixed image quality. Free tools struggle with handwritten analysis, so do not assume Tabula’s fit for clean PDFs carries over to these pages.

Vary simple columns, split headings, merged cells, blank rows, subtotals, page breaks and reading order. Before running the tools, annotate the expected rows, columns and cell associations for each layout.

Score each tool-layout result at three levels: whether the table was found, whether its rows and columns were reconstructed, and whether each value belongs to the right business item. For each expected item, mark it correct, misplaced, omitted or duplicated; record missing cells and repeated headers separately.

Compare the error counts and types within each layout instead of relying on one overall average. Extend claims 99%+ accuracy on complex tables, but keep that claim separate from results measured on your own test documents.

A shifted amount can pass a text-recognition check while corrupting a financial record. Check whether headings and values remain associated with the right rows and columns, especially across page breaks.

Tool Suitability by Document Type

Related guidance

For guidance on measuring correction effort after automated extraction, see the companion article.