When The Bot Has Three Tools Available And Confidently Picks The Wrong One

When a conversational system has multiple external tools available, a new failure category emerges beyond individual tool functionality. The model must decide which tool to use for a given request, but this routing decision often proves fragile in real-world scenarios. Tool calling, or function calling, is the modern term for this process, where the model selects the appropriate tool based on the user's message before proceeding.

Most demonstrations use clear cases, like booking an appointment or checking a balance, but real conversations are rarely that simple. The true difficulty arises when two tools seem plausible for a request, as often occurs between a general knowledge base search tool and a more specific structured lookup tool. A user asking about order status should go to the order status tool, but a user asking about return policy for damaged items could logically be handled by either tool.

Without explicit routing guidance, the model's choice fluctuates based on phrasing, leading to inconsistent and subtly degraded responses that may not generate the same error signals as complete failures. The issue becomes more severe when tools have real side effects, such as booking actions or payment triggers, as these require confirmation before execution.

The most common approach to fixing routing ambiguity is to provide more detailed descriptions for each tool, but this only goes so far in resolving requests that sit in the overlap zone between two tools. A more effective solution involves building an explicit disambiguation layer into the system prompt, instructing the model to recognize when a request could plausibly map to multiple tools and either ask for clarification or apply a tiebreaking rule.

For instance, if a request could be handled by either the general knowledge tool or the order-specific tool, prefer the order-specific tool when an order reference is present in the conversation. This approach resolves ambiguity deterministically, rather than relying on the model's implicit judgment, which causes inconsistency. For higher stakes cases involving write-capable actions, the focus should be on requiring confirmation before invoking any tool capable of real side effects, rather than attempting to perfect routing accuracy.

This aligns with the broader principle of tiered permission handling, where read-only tools can be invoked on reasonable inference, while tools with real consequences require explicit confirmation.