CCARF Practice Question

Paper 4 · Question 14

D2 Tool Design & MCP Integration · Paper 4

Production logs show an agent frequently calls get_customer when users ask about orders (e.g. 'check my order #12345'), instead of calling lookup_order. Both tools have minimal descriptions ('Retrieves customer information' / 'Retrieves order details') and accept similar identifier formats. What is the most effective first step to improve tool selection reliability?

• A.Expand each tool's description to include input formats, example queries, edge cases, and boundaries explaining when to use it versus similar tools.
• B.Consolidate both tools into a single lookup_entity tool that accepts any identifier and internally determines which backend to query.
• C.Add 5–8 few-shot examples to the system prompt demonstrating correct tool selection patterns for order-related queries.
• D.Implement a routing layer that parses user input before each turn and pre-selects the appropriate tool based on detected keywords.

Show answer & explanations

Correct answer: A

A: Tool descriptions are the primary mechanism LLMs use for tool selection. Expanding them is the lowest-effort, highest-leverage fix that directly addresses the root cause.

B: Consolidation is a valid architectural choice but requires significantly more effort than expanding descriptions. The exam favours proportionate first steps.

C: Few-shot examples add token overhead without fixing the underlying issue. The root cause is that descriptions do not differentiate the tools.

D: A routing layer is over-engineered as a first step. It bypasses the LLM's natural language understanding and adds infrastructure complexity.

Related Tool Design & MCP Integration questions

More: Paper 4 · All practice questions