Task Testing

Measuring correction effort after automated extraction

The cost of document extraction includes work after the model responds.

Correction effort is the active staff time and edits needed to turn extracted output into a record that meets the organisation’s acceptance rules. Measure that touch time separately from end-to-end elapsed time, then calculate labour and total workflow cost per accepted record so savings claims can be checked.

Time the complete route

Start the end-to-end clock when a document arrives and stop when its record is approved in the destination system. Log preparation, upload, processing wait, review, correction, re-entry and exception handling separately; include rejected documents and those staff choose not to process automatically. Keep a one-field edit distinct from a table rebuild.

Define correction touch time as active staff time spent finding, checking and repairing extracted output, including reopening the source and reconciling the record. Start the timer when staff begin that work and stop when correction is complete; pause it for unattended waits. Use system event timestamps where available, or a timing sheet with the document ID, staff member, start and stop times, pauses and final disposition.

Report correction touch time per document as total active correction minutes divided by all documents processed. Also report minutes per corrected record by dividing the same total by records that needed at least one edit; report correction rate as records needing an edit divided by records reviewed. Count corrected fields and structural table repairs separately, using the same counting rule throughout.

Before timing, set the acceptance rule for the destination record: specify required fields and line-item or table requirements, permitted blanks and validation conditions. Apply the same rule to manual and automated records, and log pass or fail and the reason with the rule version. Count a record as accepted only when it passes and is approved in the destination system.

On a controlled sample with known answers, count incorrect values accepted as well as errors caught.

Microsoft Document Intelligence’s invoice model returns structured invoice fields and line items; its layout model returns text, tables, selection marks and document structure.

Calculate per accepted record

Correction labour cost is active correction hours multiplied by the organisation’s applicable staff cost per hour. To calculate total cost per accepted record, add service charges, integration work allocated to the period, and staff time for review, correction, re-entry and exceptions, then divide by records that meet the acceptance rule. Include costs for rejected and non-automated cases.

Take a representative sample from the workload, and report its period and document count. Break results out by supplier, layout or document type so hard cases remain visible; compare with manual processing of a similar mix rather than a convenient average from a different workload.

Use the result to set boundaries. Straightforward digital invoices might flow automatically while scans with complex tables take a different route. Recheck after changes to templates or model versions; the aim is lower total work at an acceptable error rate, not the highest number of pages sent through an API.

More from Task Testing

Task Testing

Comparing table extraction from different document layouts

A table can be transcribed word for word yet lose the relationship between a row label and its amount.

Task Testing

AI image and design tools

Choose an AI image or design workflow by commercial use, text accuracy, editability and the file your team needs to deliver.

Task Testing

AI output quality testing

Set acceptance rules, check AI outputs against approved sources and compare the work needed to reach a usable result.