Building Bharat Buddy: My 10-Day Voice AI Journey with Murf Falcon

In the VoiceForBharat Edition of the 10 Days of Voice Agents challenge, I embarked on a 10-day journey to build Bharat Buddy, a voice AI tutor designed to make learning more natural and conversational. The project aimed to create an AI learning assistant that could communicate with students in Hindi, English, or Hinglish, allowing them to ask questions naturally, practice problems, and receive assistance from specialists when needed.

Bharat Buddy's core functionality includes voice interaction, user memory, the ability to use tools, and handovers to specialist agents. The system was built using LiveKit for real-time voice communication, an LLM for reasoning, and Murf Falcon, a fast TTS API, for generating natural-sounding voice responses.

One of the key innovations in Bharat Buddy was its ability to remember users and context, making future interactions more meaningful. The system also incorporated tools to perform specific actions, such as making outbound calls for follow-ups, reminders, and notifications. However, the project also highlighted the importance of knowing when to hand off to a human specialist, particularly when the AI cannot safely handle a user's needs.

Another highlight was adding a specialized maths practice agent, which could handle arithmetic, percentages, fractions, ratios, and basic algebra and geometry, providing step-by-step explanations to students. The project also included a simple analytics dashboard to track call metrics, such as total calls, successful calls, and failed calls, using SQLite for data storage.

Throughout the project, I learned the importance of focusing on building a natural, conversational AI experience rather than simply adding features. By breaking down the agent into specialized components, I was able to create a more manageable and efficient system.