CCARF Practice Question

Paper 1 · Question 34

D4 Prompt Engineering & Structured Output · Paper 1

Your extraction pipeline retries failed documents, but the retry simply resends the original document with the same prompt. Success rates on retries are only marginally better than the first attempt. Developers have identified that most failures are format mismatches where the model places values in the wrong fields. How should you improve the retry mechanism?

• A.Increase the number of retries from 1 to 3, since extraction is non-deterministic and more attempts improve odds
• B.Switch to a different model for retries so a fresh perspective catches the errors
• C.Skip retries and route all failures directly to human review to avoid wasting API costs
• D.Send the original document, the failed extraction, and the validation error naming the misplaced fields.

Show answer & explanations

Correct answer: A

A: Naive retries without error feedback produce the same mistakes. More attempts at the same flawed approach yield diminishing returns. The model needs to see what went wrong.

B: The issue is not the model but the absence of error context. Any model retrying without seeing the specific validation error will likely repeat the same mistakes.

C: Format mismatches are fixable errors. The model can self-correct when given proper error feedback. Skipping retries wastes the model's self-correction capability for errors that are resolvable.

D: Retry-with-error-feedback is dramatically more effective than naive retries. Sending the original document, the failed extraction, and the specific validation error allows the model to self-correct. Format mismatches are fixable errors that respond well to this approach.

Related Prompt Engineering & Structured Output questions

More: Paper 1 · All practice questions