CCARF Practice Question

Paper 1 · Question 60

D4 Prompt Engineering & Structured Output · Paper 1

Your extraction pipeline has a validation step that checks extracted financial data. When validation fails, the system retries by sending just the original document with the same prompt. Retry success rates are below 10%. The most common failure is the model placing the 'net amount' value in the 'gross amount' field and vice versa. How should the retry be restructured?

• A.Increase retries from 1 to 5, since more attempts at the same prompt will eventually produce the correct field mapping by chance
• B.Send the original document, the failed extraction, and the specific validation error naming the swapped fields.
• C.Add a post-processing rule that automatically swaps net and gross amounts when validation detects the pattern
• D.Switch to a different model for the retry, since the original model has a persistent field- mapping bias

Show answer & explanations

Correct answer: B

A: If the model consistently maps fields incorrectly with the same prompt, more retries without feedback will produce the same incorrect mapping. The model needs to see its specific error.

B: Retry-with-error-feedback is dramatically more effective than naive retries. The model seeing its specific mistake (net and gross amounts swapped) and the validation error allows targeted self-correction rather than repeating the same misinterpretation.

C: Hard-coded swap rules are brittle and mask the underlying extraction error. They also fail if the issue manifests differently in other documents. Error feedback teaches the model to extract correctly.

D: The issue is the absence of error context in the retry, not a model-specific bias. Any model retrying without seeing the specific validation error will have a low success rate.

Related Prompt Engineering & Structured Output questions

More: Paper 1 · All practice questions