Build an AI Shipment Agent with SMS, Voice, and Telnyx Inference

Most package tracking systems force customers to take action to check shipment progress. You receive a tracking number and periodically refresh a webpage or receive generic text updates. When a real question arises, you must seek assistance elsewhere. The shipment-agent example aims to invert this dynamic by treating a shipment as an autonomous AI agent.

This Python and Flask application allows a package to proactively send SMS updates, understand customer replies using Telnyx AI inference, and handle inbound calls with relevant shipment information. Code for this example is available at https://github.com/team-telnyx/telnyx-code-examples/tree/main/shipment-agent.

The core of the application is the ShipmentAgent, which manages shipment status, carrier and tracking context, customer phone number, interaction history, and messaging and voice behavior. Contrary to typical stateless chatbots, this agent exists alongside the shipment's lifecycle. When a carrier updates status, a Flask webhook notifies the ShipmentAgent, which updates its internal state and sends a message to the customer via Telnyx Messaging.

If the customer responds, the app forwards the message along with shipment context to Telnyx AI Inference. This enables the agent to provide accurate, context-aware responses rather than generic support. If the customer calls the provided Telnyx number, the Call Control flow can deliver the same contextual information, integrating SMS and voice into a unified communication channel.

Key components utilized in the example include Telnyx Messaging for proactive SMS updates and customer replies, Telnyx Call Control for handling inbound voice interactions, Telnyx AI Inference for natural language processing, and Telnyx webhook signing to verify the authenticity of incoming events. Verification is crucial, as the agent relies on these events to update shipment status in real-time.

To run the example, clone the Telnyx code examples repository, create a virtual environment, install dependencies, copy the .env.example file, fill in your Telnyx API details, and run the app.py script. This launches a Flask server locally, ready to integrate with your Telnyx messaging and voice configurations.

For production deployment, the example suggests expanding the app with a persistent database to store event history, implementing idempotency to handle repeated carrier events, and enhancing the handling of delivery instructions. Customer authentication, escalation to human support for complex issues, and retry mechanisms for SMS and webhook processing should also be considered.

Additionally, observability measures should be put in place to track any missed or delayed events. While the current example focuses on a minimal architecture, these additions would provide a robust solution for turning a shipment into a versatile communications agent capable of managing conversations through both SMS and voice channels. Resources to explore further include the Telnyx AI toolkit, documentation, and the Telnyx Portal.