CCARF Practice Question

Paper 1 · Question 25

D4 Prompt Engineering & Structured Output · Paper 1

The pipeline validates each tool_use extraction against a Pydantic model with model_validate(). Validation succeeds and every field has the correct type, yet extracted totals still disagree with the documents' stated totals. What explains this, and what is the fix?

• A.model_validate() is failing silently; pin an older Pydantic version until the regression is fixed
• B.Type checks passed but no rule relates the totals; add a model validator for the sum rule and route its errors into retries
• C.The schema needs numeric minimum and maximum constraints on the stated_total field so that mismatched totals are rejected at parse time
• D.Semantic disagreement indicates hallucination; lower the temperature so extraction becomes deterministic

Show answer & explanations

Correct answer: B

A: model_validate() did its job: the extraction conforms to the model's types. Nothing failed silently; the missing check was never defined.

B: Schema conformance and semantic correctness are different layers; the sum relationship must be written as a validator, whose errors drive the retry loop.

C: A per-field range cannot express a relationship between two fields; the mismatch is only detectable by comparing them in validation code.

D: Sampling settings do not create correctness, and determinism would only repeat the same wrong extraction; this failure class belongs to semantic validation.

Related Prompt Engineering & Structured Output questions

More: Paper 1 · All practice questions