[Scenario 6: Structured Data Extraction] You are building a structured data extraction system: pulling information from unstructured documents, validating output against JSON schemas, handling edge cases gracefully, and integrating with downstream systems.

Since adopting tool use, extractions are always syntactically valid — yet invoices arrive where line items don't sum to the stated total, and values occasionally land in the wrong fields. What should you understand and do?

Question

[Scenario 6: Structured Data Extraction] You are building a structured data extraction system: pulling information from unstructured documents, validating output against JSON schemas, handling edge cases gracefully, and integrating with downstream systems.

Since adopting tool use, extractions are always syntactically valid — yet invoices arrive where line items don't sum to the stated total, and values occasionally land in the wrong fields. What should you understand and do?

Accepted Answer

Strict schemas eliminate syntax errors but not semantic ones — add semantic validation, e.g., extracting calculated_total alongside stated_total and flagging discrepancies with a conflict_detected boolean

Answer

Schema compliance was falsely advertised; file a bug

Answer

Make every field an unconstrained string so nothing can be wrong

Answer

Re-run each extraction until the numbers happen to sum

Paper 3 · Question 54

Related Prompt Engineering & Structured Output questions