CCARF Practice Question

Paper 3 · Question 47

D4 Prompt Engineering & Structured Output · Paper 3

[Scenario 5: Claude Code for Continuous Integration] You are integrating Claude Code into a CI/CD pipeline for automated code review, test generation, and pull-request feedback — designing prompts that produce actionable findings while minimizing false positives. The reviewer labels near-identical issues “critical” one day and “minor” the next. Severity-based merge gates are therefore unreliable. How do you get consistent severity classification?

• A.Define explicit severity criteria with concrete code examples for each level, so every severity has an operational definition the model can match against
• B.Remove severity levels and treat all findings equally
• C.Let the model choose severity freely but average it over five runs
• D.Map severity to line count of the affected code

Show answer & explanations

Correct answer: A

A: Consistency comes from operational definitions: severity levels anchored by concrete examples give the classifier something to match, turning a vibe into a rubric. That is what makes severity gates dependable.

B: discards the signal the merge gate needs

C: statistically launders an unanchored judgment

D: measures size, not impact — a one-line auth bypass outranks a fifty-line comment tweak

Related Prompt Engineering & Structured Output questions

More: Paper 3 · All practice questions