{
  "id": 1063978,
  "title": "Building an AI Voice Agent for Bharat: My 10-Day Journey",
  "url": "https://urgent.news/2026/08/15/building-an-ai-voice-agent-for-bharat-my-10-day-journey",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-15T15:22:31.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ayushshinde13/building-an-ai-voice-agent-for-bharat-my-10-day-journey-f6d"
  },
  "original_language": "en",
  "account": "In the recent 10 Days of AI Voice Agents - #VoiceForBharat Edition challenge, I embarked on a journey to create an AI voice agent called Sadie. My aim was to build a voice agent capable of understanding users, remembering conversations, utilizing tools, making phone calls, connecting users to humans, and smoothly transferring conversations to specialized agents. This expedition taught me that crafting a voice agent is far more complex than merely linking a large language model (LLM) with a text-to-speech (TTS) API.\n\nThe challenge stemmed from a common problem faced by many: ease of speaking over typing. This benefit is particularly valuable for individuals who prefer conversational interaction, require quick information, prefer speaking in both Hindi and English, or seek assistance without navigating complex interfaces. My objective was to construct an AI voice assistant that could render learning and obtaining information more organic. Instead of typing a query, users could simply communicate with Sadie.\n\nAs a result of this challenge, I developed Sadie, an AI voice agent equipped to:\n- Engage in real-time voice conversations\n- Comprehend Hindi-English code-mixed dialogues\n- Adhere to personality and safety guidelines\n- Remember information with user consent\n- Leverage external tools\n- Execute outbound phone calls\n- Delegate conversations to humans when necessary\n- Record call details\n- Direct conversations to expert agents\n\nThe architectural framework of my project entailed:\nUser | Voice ↓ LiveKit ↓ Deepgram STT | Deepgram TTS \\ Memory Tools Specialists \\ \\ ↓ Murf Falcon | User Voice\n\nThe key components included Deepgram for Speech-to-Text, Google Gemini serving as the agent's brain to interpret user requests, Murf Falcon for converting AI responses into natural speech, LiveKit for managing real-time audio communication, and other crucial elements like memory, external tools, telephony, human escalation, analytics, and specialist agents.\n\nFrom Day 1 to Day 3, I concentrated on imbuing the agent with a natural voice and defining its personality. I configured Murf Falcon Text-to-Speech with the Anisha voice, established a clear personality and instructions for the agent, and incorporated script rules to ensure the agent remained consistent and within its defined tasks. This step enhanced the agent's reliability and consistency.\n\nOn Day 4 and Day 5, I introduced persistent memory to Sadie by creating a SQLite database named \"sadie_memory.db\". This permitted the agent to retain useful information about returning users, provided memory storage was explicitly permitted. Additionally, I integrated the Free Dictionary API as an external tool, enabling Sadie to fetch pertinent dictionary information as required.\n\nOn Day 6, I tackled the intricate task of connecting Sadie to phone calls. I set up LiveKit SIP, a SIP trunk, and Linphone for outbound calling, intending to enable the voice agent to conduct automated tutoring calls. This phase was notably challenging due to the numerous configuration intricacies involved in telephony.\n\nThe subsequent day, I developed a human escalation mechanism for the AI agent. Before escalating to a human, the agent sought user permission. Upon approval, the system generated a reference ID and conveyed escalation details via a Discord webhook, establishing a basic link between the AI and a human. This situation emphasized that AI should not always strive to solve all issues autonomously; sometimes, human intervention is the most appropriate action.\n\nOn Day 8, I constructed a real-time analytics dashboard using Next.js, Chart.js, and SQLite. The dashboard featured a glassmorphism design and displayed call information. This move underscored the transition from merely building an agent to monitoring its behavior and call outcomes.\n\nThe final day was dedicated to introducing multi-agent functionality. The central agent could transfer the conversation to a specialist if the user's query necessitated a specific skill. Three specialists were created: a Grammar Specialist for handling English grammar inquiries, a Maths Specialist for addressing mathematical questions, and a Full Stack Development Specialist for assisting with web development and full-stack development inquiries. A critical aspect was preserving the conversation context during the handoff, ensuring the specialist understood the user's prior discussion without requiring repetition.\n\nThroughout the challenge, I encountered several hurdles. The most significant were the difficulties with outbound SIP calls and inaccurate call analytics in the dashboard. The SIP call problems were resolved by adjusting the Media Encryption Mandatory setting in Linphone from Mandatory to Disabled/Optional, which rectified the call connection issue. However, an issue surfaced in the analytics dashboard where unsuccessful browser disconnects were erroneously being classified as failed calls. This was rectified by segregating browser conversations from actual phone-call outcomes and modifying the dashboard to track genuine phone-call results.\n\nThis 10-day journey underscored the complexity of constructing an AI voice agent and highlighted the necessity of addressing both technical and functional elements to create a truly effective and user-centric voice assistant.",
  "summary": "Introduction For the past 10 days, I took part in the 10 Days of AI Voice Agents — #VoiceForBharat Edition challenge. During this challenge, I built an AI voice agent named Sadie. My goal was not just to make an AI that could talk. I wanted to build a voice agent that could understand users, remember conversations, use tools, make phone calls, connect users to humans, and hand conversations to…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}