CCARF Practice Question

Paper 4 · Question 7

D4 Prompt Engineering & Structured Output · Paper 4

A moderation team introduces a severity scale ('low', 'medium', 'high', 'critical'). The same post (a user threatening to 'destroy' a competitor's product in a review) is classified as 'high' on some runs and 'low' on others. The prompt defines severities with phrases like 'high means the content is clearly harmful' and 'low means the content is mildly inappropriate'. What should they change?

• A.Remove the severity scale and use binary classification (violation / not violation) to reduce inconsistency
• B.Replace the prose severity descriptions with a concrete example per level as a calibration anchor.
• C.Run the classification three times over and take the majority vote to settle the severity for each item.
• D.Set the temperature to 0 to eliminate classification variance from one run of the pipeline to the next.

Show answer & explanations

Correct answer: B

A: Binary classification loses valuable granularity needed for routing decisions (auto-remove vs escalate vs warn). The fix is better severity criteria, not removing severity entirely.

B: Concrete examples anchor each severity level to specific, observable patterns. When the model sees 'destroy a competitor's product' in a review context, it can match against the 'low severity — figurative language' example rather than interpreting 'clearly harmful' differently each run.

C: Majority voting averages out noise but triples cost and latency without fixing the root cause. With vague criteria, the model may consistently misclassify in the same direction, making voting ineffective.

D: Lower temperature reduces randomness but does not fix ambiguous criteria. The model is inconsistent because the severity definitions are vague, not because of sampling variance.

Related Prompt Engineering & Structured Output questions

More: Paper 4 · All practice questions