Nobody Hacked It. The Lookup Tool Just Took a Phone Number.

When clients ask if a voice agent can be hacked, the initial response often focuses on guardrails, refusals, and the difficulty of jailbreaking a well-written system prompt. However, this perspective overlooks the true risks posed by AI voice agents. These agents are not dangerous because they might say odd things, but because they can perform actions.

They can read from a CRM, write to a calendar, send texts, transfer calls, and trigger automations that interact with systems the caller will never see. The critical point is that these actions are directly accessible by anyone who knows the phone number, without any login, signup, email verification, or bot checks. The endpoint is simply a phone number that answers on the first ring, making it an attractive target for attackers.

To safeguard against potential threats, the key is to understand what the agent can do, rather than just focusing on the intended functions or conversation design. For a typical receptionist agent, the actions include looking up a contact, reading back an appointment, creating or cancelling/rescheduling an appointment, sending an SMS, transferring the call to a human, and sending a payload to the CRM.

When considering these actions from an attacker's perspective, each verb transforms into a question, strategically asking who the contact is, which appointment to cancel, which number to send an SMS to, and to whom the call should be transferred. This approach highlights the inherent risks in the agent's capabilities.

One significant issue is that the lookup tool uses the phone number provided by the caller as a parameter, unlike other functions that require explicit input. The model relies on the information supplied by the caller, leading to a potential security breach. Caller ID serves as a mere hint rather than a reliable identity verification method.

It can be easily spoofed, and callers often share the same number, such as family members, reception lines, or representatives acting on behalf of others. This reliance on caller ID creates a flawed authorization mechanism. Consequently, the number alone cannot definitively prove the caller's identity.

To mitigate these risks, a crucial step is to separate consequences based on the potential impact of the agent's actions. Low-stakes tasks, such as warmly greeting a known caller by name and offering their usual appointment type, can operate using only the provided phone number. However, any action that exposes stored data or modifies records requires additional verification from the caller, typically through a date of birth or appointment date, which must be checked by the automation layer rather than solely relying on the model's judgment.

In medical settings, for instance, certain requests should not have a self-service path, regardless of how well the caller verifies their information.

Another fundamental aspect is the client-side validation mechanism embedded within the system prompt. This validation shapes normal behavior, enhances the common path, and remains ineffective against deliberate attempts to bypass it. Similar to client-side validation in traditional forms, it is essential to ensure that the model cannot perform actions that could be problematic.

This involves scoping tools effectively instead of simply instructing the agent. For instance, the lookup tool should not accept a phone number parameter explicitly but instead use the number already present in the call, injected by the pipeline. This prevents the model from requesting a different number, as no field exists to accommodate such a request.

As a result, the model only accesses the fields necessary for the conversation, preventing unauthorized access to sensitive data during high-pressure situations. Actions like cancelling or rescheduling appointments are handled through separate tools that operate only on records matched to the verified caller, thus minimizing potential security breaches.

Transfers are limited to a predefined list of destinations defined in the configuration, ensuring the agent cannot be exploited to place calls on behalf of others. Outbound SMS messages are sent to the calling number or a verified number linked to the record, rather than to any number mentioned during the call, further safeguarding against unauthorized actions.

These measures collectively address direct financial risks and prevent the agent from being used as a relay for malicious purposes. Additionally, implementing concurrency caps, a per-number rate limit, and spend alerts can effectively handle common threats, such as automated dialing attacks. However, the most insidious threat often stems from indirect injection, where agents inadvertently read system prompts, knowledge bases, and CRM fields, potentially exposing the model to sensitive information.