Human review

If extractions contain errors, for example as the result of hard-to-read handwriting, you can flag extractions for manual correction by a human reviewer. Flag extractions automatically at scale in production by configuring rules based on validations and extraction coverage. Use the Sensible app's Human review tab to manually correct the extracted fields, then approve or reject the extraction. Once you approve or reject, you can't change the status again.

In a document type, in the Human Review tab, set the criteria for Sensible to flag an extraction for manual review:

Enable human review for all documents in a type, or set criteria to trigger review. Criteria include:

• Extraction coverage

• Validation errors and warnings, if configured. Select all errors, all warnings, or individual validations.

After you run extractions in the document type, the Human review tab displays the document types that contain extractions flagged for review:

Click a document type to review the flagged extractions:

Click a field's value to:

• view the field's full value for tables, sections, and other complex fields
• edit the field's value
• view the field's source location highlighted in the document

Click the checkmark icon next to each field to mark it approved. When you're done editing field values and approving individual fields, click Approve Extraction or Reject Extraction to remove it from the extractions flagged for review.

Review status

You can check review status for past extractions in one of the following ways:

• Sensible API/SDK: If you enable human review for a document type, then set a webhook for each extraction request in the document type. Sensible pushes the extraction, including any manual corrections the reviewer made and the review status, to the specified webhook when a reviewer approves or rejects an extraction. You can also filter by extraction status by specifying the review_statuses parameter on the List extractions endpoint.

• Sensible app: Filter past extractions by review status. In the Extraction history tab, use the Review Status criterion.

Review lifecycle

The following diagram shows how to integrate human-in-the-loop review into your application:

1. Configure review triggers: Configure extraction quality validation for a document type, for example, tax documents or pay stubs. Any extraction that doesn’t meet your quality validations triggers a human review.
2. Specify a webhook for each document extraction: When extracting data from a document using Sensible’s API or SDK, specify a webhook destination URL that receives updates to the extraction’s review status.
3. Notify a reviewer: When the webhook indicates that a completed extraction needs review and correction, notify a reviewer and send them a link to the review interface.
4. Ingest corrected extractions: When the webhook indicates that a reviewer approved an extraction, ingest the document data into your system.