CCARF Practice Question

Paper 1 · Question 26

D4 Prompt Engineering & Structured Output · Paper 1

An extraction pipeline uses `tool_use` with a 15-required-field schema. For some document types, only 8 of those fields ever appear in the source material. The model consistently fabricates plausible values for the missing 7, and the instruction 'do not fabricate data' has not helped. Should the team (A) keep all fields required and add post-extraction validation, or (B) redesign the schema? Which is correct and why?

• A.Keep all fields required and add post-extraction validation to detect and remove fabricated values
• B.Redesign the schema to make the 7 sometimes-absent fields optional/nullable, allowing the model to return null instead of fabricating
• C.Create separate schemas for each document type, each containing only the fields present in that document type
• D.Switch to prompt-based JSON extraction where you can instruct the model to omit absent fields entirely

Show answer & explanations

Correct answer: B

A: Post-extraction validation cannot reliably distinguish fabricated values from correct ones — fabricated values are designed to look plausible. The schema itself forces fabrication by making all fields required.

B: Making fields optional/nullable when source data may be absent removes the schema-level pressure to fabricate. The model can legitimately return null for missing information, which is a correct representation of reality. This addresses the root cause at the schema level.

C: Multiple schemas add maintenance complexity and require document type classification before extraction. Making fields nullable in a single schema is simpler and equally effective.

D: Prompt-based JSON sacrifices schema compliance guarantees. tool_use with optional fields gives both flexibility for absent data and schema validation for present data.

Related Prompt Engineering & Structured Output questions

More: Paper 1 · All practice questions