Building ASHA: A Real-Time Multi-Agent Voice AI Health Guide for Bharat

Building ASHA: A Real-Time Multi-Agent Voice AI Health Guide for Bharat Day 10 — 10 Days of AI Voice Agents #VoiceForBharat Challenge

As part of the 10 Days of AI Voice Agents challenge, I created ASHA, a real-time AI voice assistant aimed at simplifying access to basic health guidance. The foundation for this project was built using the Murf LiveKit Starter, which provided a pre-built real-time voice agent framework leveraging LiveKit, Deepgram, an LLM, and Murf Falcon TTS technology.

ASHA's primary goal was to transform the healthcare interaction process from a complex, text-based interface to a more natural, conversational voice experience. The idea behind ASHA was to act as an accessible community health guide, rather than a replacement for healthcare professionals. Users could simply describe their symptoms in everyday language, such as "I've been feeling dizzy since morning," and ASHA would offer appropriate guidance.

To achieve this, ASHA's architecture relied on several key components:

1. Real-time communication through LiveKit, enabling seamless audio streaming and multi-agent interactions.

2. Speech-to-Text using Deepgram Nova-3, converting spoken words into text for LLM processing.

3. LLM reasoning powered by Groq + LLaMA 3.3B, handling the core understanding and response generation tasks.

4. Text-to-Speech with Murf Falcon, transforming AI-generated text responses back into spoken audio for the user.

5. A Python backend, powered by FastAPI and SQLite, managing application logic, data persistence, and integration with external workflows.

One of the primary challenges in developing ASHA was navigating the additional engineering complexities introduced by the voice interface. Implementing real-time audio streaming, microphone permissions, speech recognition, LLM latency, TTS latency, connection state management, and interruptions required careful consideration and planning.

Despite these hurdles, the project offered a valuable learning experience, demonstrating the importance of understanding existing architectures before extending them for specific use cases.