{
  "id": 972122,
  "title": "Building a Voice Agent in 10 Days — My VoiceForBharat Journey",
  "url": "https://urgent.news/2026/08/15/building-a-voice-agent-in-10-days-my-voiceforbharat-journey",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-15T06:03:19.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/nittala_koushik_3b5521b57/building-a-voice-agent-in-10-days-my-voiceforbharat-journey-2abh"
  },
  "original_language": "en",
  "account": "Creating a Voice Agent: A 10-Day Journey for a Local Indian Store\n\nConstructing a voice agent may seem straightforward at first glance. Listen to the user, transmit the text to a language model, generate a response, and speak it back. However, as you introduce real-world requirements such as memory, safety protocols, multilingual conversations, phone calls, human intervention, analytics, and specialist transfers, it transforms into an entirely different engineering challenge.\n\nOver the past ten days, I embarked on a similar journey as part of the 10 Days of Voice Agents - VoiceForBharat Edition project. My objective was to construct a voice assistant tailored for a local Indian store, capable of handling simple customer inquiries like product availability, pricing, store timings, promotions, alternative product recommendations, and handling damaged or incorrect products. The assistant should seamlessly integrate into customers' lives, eliminating the need to open an app, type inquiries, or navigate through menus.\n\nThe Problem at Hand: Addressing Simple Customer Queries\nCustomers of local grocery or general stores often have straightforward questions, such as checking product availability, understanding prices, checking store timings, identifying ongoing offers, requesting alternative products, and addressing issues related to damaged or incorrect products. These queries don't necessarily require human intervention, and a voice-first assistant could efficiently handle these conversations.\n\nDesigning the Voice Agent: A Local Indian Store Assistant\nMy project aimed to build a voice assistant for a local Indian grocery/general store, equipped to:\n\n- Assist customers with shopping-related questions\n- Handle English and Hindi conversations\n- Understand Hinglish-style conversations\n- Remember returning customers\n- Adhere to safety and accuracy guardrails\n- Transfer return and refund requests to a specialist agent\n- Initiate outbound phone calls\n- Generate call outcome statistics\n\nThe core objective was to create an AI that not only converses but also behaves like a practical customer-service system.\n\nSystem Architecture: From Speech to Action\nThe core voice processing pipeline consisted of the following components:\n\nUser Speaks → Deepgram Speech-to-Text → Google Gemini LLM → Tools / Memory / Agent Logic → Murf Falcon TTS → LiveKit Real-Time Audio → User hears the response\n\nDeepgram Nova-3 handled speech recognition, Google Gemini managed language understanding and response generation, Murf Falcon – using an Indian English voice – took care of text-to-speech, and LiveKit facilitated real-time audio communication between the user and the agent.\n\nIndian Voice: Tailoring Murf Falcon for Indian English\nA crucial aspect of this project was selecting an appropriate voice. I opted for Murf Falcon, leveraging its Indian English voice and configuring the agent to switch between en-IN and hi-IN based on the conversation. Murf Falcon, known for its speed, was chosen as the text-to-speech layer of the assistant. The project employed multilingual turn detection, voice activity detection, preemptive generation, and noise cancellation to ensure a more natural conversation flow.\n\nMemory for Returning Customers\nTo enhance user experience, I incorporated caller memory using SQLite. The agent utilizes two tools: lookup_caller() and save_caller_memory(). When a caller is recognized, the agent retrieves previously saved information. If the customer shares any preferences, the agent requests permission before storing them. This design decision ensures that the agent does not silently collect information without explicit consent. The system prompt strictly prohibits the agent from fabricating memories or disclosing internal database information.\n\nImplementing Guardrails and Agent Personality\nDefining the boundaries of the agent's capabilities was crucial. The assistant had clear objectives and guardrails, ensuring it:\n\n- Never makes up product prices\n- Does not claim items are in stock without verification\n- Does not promise unconfirmed discounts\n- Does not confirm orders that have not actually been confirmed\n- Refrains from answering unrelated requests\n- Avoids providing harmful or illegal assistance\n- Acknowledges when it doesn't have an answer instead of guessing\n\nThese boundary conditions were essential to ensure the agent's responses were accurate, reliable, and beneficial to customers.\n\nSpecialist Agent Handoff: Seamless Transfers for Complex Issues\nAnother noteworthy feature I implemented was the specialist handoff mechanism. The main store assistant dealt with routine shopping inquiries, while the dedicated Returns and Refunds Specialist handled issues like returns, refunds, damaged or defective products, incorrect or missing items, and return eligibility. The flow was designed to preserve the existing conversation context, eliminating the need for customers to repeat their problems. The handoff process was as follows:\n\nCustomer → Main Store Assistant → Return / Refund request detected → Returns & Refunds Specialist → Continue existing conversation\n\nThis seamless transition made the project feel more akin to a genuine customer support system rather than a single chatbot.\n\nOutbound Phone Calls: Enabling Direct Customer Interaction\nTo further enhance the customer experience, I incorporated outbound calling using LiveKit's SIP capabilities. The outbound call process created a unique LiveKit room, dispatched the agent into that room, and established an outbound SIP participant connecting to the phone network. This enabled the agent to initiate direct phone conversations with customers, handling calls with precision and delivering a personalized touch. The challenge lay in coordinating the LiveKit agent, the room, the SIP configuration, and the phone connection.\n\nCall Analytics: Measuring Performance and Improving the Assistant\nWith the assistant now capable of making outbound calls, monitoring its performance became a priority. I introduced call tracking and analytics to measure the assistant's effectiveness. Each call recorded the following metrics:\n\n- Call ID\n- Call type\n- Duration\n- Intent\n- Outcome\n- Escalation status\n\nThese analytics would provide valuable insights into the assistant's performance, helping to identify areas for improvement and ensuring the system continues to evolve and meet customers' needs effectively.",
  "summary": "Building a Voice Agent for a Local Indian Store: 10 Days of Voice Agents — VoiceForBharat Edition Building a voice agent sounds simple at first. Listen to the user, send the text to an LLM, generate a response, and speak it back. But once you start adding real-world requirements — memory, safety guardrails, multilingual conversations, phone calls, human escalation, analytics, and specialist…",
  "key_points": [
    "Voice agent built in 10 days for local Indian store",
    "Handles English and Hindi, remembers returning customers",
    "Specialist handoff for returns, refunds, and complex issues"
  ],
  "editors_take": "Building a voice agent for a local Indian store with capabilities like memory, safety protocols, and specialist transfers transforms customer service, enabling efficient handling of simple inquiries and seamless support.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}