CCARF Practice Question

Paper 4 · Question 27

D4 Prompt Engineering & Structured Output · Paper 4

Your CI/CD pipeline uses a system prompt with a tool named 'check_security' and also includes instructions that say 'check the security of each function'. Developers report that the model sometimes produces a text-based security analysis instead of calling the check_security tool. What is the most likely cause?

• A.The 'check the security' instruction keyword-overlaps the 'check_security' tool name, so the model follows the text instead of calling the tool.
• B.The system prompt has grown too long, so the model skips over the tool definitions near the end and never registers that check_security is available to call.
• C.The model's temperature is set too high, which is causing it to select tools at random rather than by their fit to the request in hand, so the wrong tool wins.
• D.The model cannot call tools at all from inside a CI/CD pipeline context, so the calls never reach it and nothing is extracted at all.

Show answer & explanations

Correct answer: A

A: When system prompt instructions use phrasing that closely mirrors tool names, the model can interpret 'check the security' as a general instruction rather than a trigger for the check_security tool. Distinct naming reduces this ambiguity.

B: System prompt length does not cause tool definitions to be skipped. The model processes tools and instructions together. The issue is keyword overlap creating ambiguity.

C: Temperature affects randomness in generation but is not the primary cause of tool call confusion. The issue is prompt design, not sampling parameters.

D: Tool calling works in any context. The issue is prompt design, not an environmental limitation.

Related Prompt Engineering & Structured Output questions

More: Paper 4 · All practice questions