CCARF Practice Question

Paper 4 · Question 31

D4 Prompt Engineering & Structured Output · Paper 4

The moderation system's prompt instructs Claude to 'be conservative when moderating and err on the side of caution.' Reviewers find that innocuous posts about cooking with knives, news articles about violence, and fictional war stories are all being flagged as policy violations. What is the root cause and fix?

• A.Replace 'be conservative' with explicit categorical criteria defining each violation category with concrete examples.
• B.Add an allowlist of safe topics such as cooking, news and fiction that should never be flagged at all by the classifier.
• C.The model is too sensitive, so lower the temperature until it stops over-flagging borderline posts in the moderation queue.
• D.Add a second moderation pass that re-reads each flagged post under the same 'be conservative' guidance and strips out the false positives before any action.

Show answer & explanations

Correct answer: A

A: Vague directives like 'be conservative' give the model no basis for distinguishing between genuine violations and legitimate content that mentions sensitive topics. Explicit criteria with examples of both violations and non-violations (news, fiction, educational content) enable consistent, calibrated decisions.

B: Allowlists are brittle and impossible to maintain comprehensively. New safe topics will constantly appear. The fix is better criteria that teach the model to distinguish context, not a growing list of exceptions.

C: Temperature affects randomness, not the interpretation of moderation criteria. The over-flagging is caused by vague instructions, not sampling behaviour.

D: A second pass using the same vague 'be conservative' criteria will reproduce the same false positives. Fix the criteria first; add verification layers second.

Related Prompt Engineering & Structured Output questions

More: Paper 4 · All practice questions