CCARF Practice Question

Paper 3 · Question 45

D4 Prompt Engineering & Structured Output · Paper 3

[Scenario 5: Claude Code for Continuous Integration] You are integrating Claude Code into a CI/CD pipeline for automated code review, test generation, and pull-request feedback — designing prompts that produce actionable findings while minimizing false positives. Your review prompt says “be conservative and only report high-confidence findings,” yet false positives remain high. What does effective precision engineering look like instead?

• A.Add “be very, very conservative” for stronger emphasis
• B.Report all findings but sort them by the model's stated confidence
• C.Reduce the number of files reviewed per run
• D.Replace confidence language with explicit categorical criteria: define which issue types to report (bugs, security) and which to skip (minor style, local patterns), with concrete boundaries

Show answer & explanations

Correct answer: D

A: intensifies an instruction that has no operational content

B: reorders noise instead of reducing it — self-stated confidence is poorly calibrated

C: relabels the same false positives

D: Vague conservatism doesn't transfer — specific criteria do. Defining reportable versus skippable categories with concrete boundaries gives the model an operable decision rule, which is what actually moves precision.

Related Prompt Engineering & Structured Output questions

More: Paper 3 · All practice questions