CCARF Practice Question

Paper 3 · Question 56

D4 Prompt Engineering & Structured Output · Paper 3

[Scenario 6: Structured Data Extraction] You are building a structured data extraction system: pulling information from unstructured documents, validating output against JSON schemas, handling edge cases gracefully, and integrating with downstream systems. An overnight batch of 10,000 documents completes with 200 failures: most are oversized documents that blew past context limits, plus some transient errors. What is the correct failure-handling workflow?

• A.Identify the failed requests by custom_id and resubmit only those, with modifications where needed — e.g., chunking the oversized documents before resubmission
• B.Resubmit the entire 10,000-document batch and keep whichever results arrive first
• C.Discard the 200 failures as an acceptable loss rate
• D.Switch the 200 failures to the synchronous API unchanged, where context limits don't apply

Show answer & explanations

Correct answer: A

A: custom_id exists for exactly this: correlate failures to their source documents, fix what caused each failure (chunk the oversized ones), and resubmit only the fixed subset — paying again for 200 documents, not 10,000.

B: reprocesses 9,800 successes to retry 200 failures

C: silently drops 2% of the corpus

D: misunderstands context limits, which belong to the model, not the API path — unchunked oversized documents fail synchronously too

Related Prompt Engineering & Structured Output questions

More: Paper 3 · All practice questions