Sehat Sathi: Building a Voice Agent for Health Access in Bharat

A 10-day endeavor yielded the creation of a multilingual health-access voice assistant known as Sehat Sathi, or Health Companion. Designed to cater to India's vast population that hasn't embraced smartphones or fast data plans, Sehat Sathi aims to assist those who need guidance on health matters, such as determining the severity of a common fever or locating the nearest government hospital.

Most voice AI solutions cater to users with smartphones, fast internet, and proficiency in English. Sehat Sathi bridges the gap by offering a voice-first health access assistant that can communicate in Hindi, English, or a mix of both languages. Its primary function is to provide safe and straightforward advice, steering users toward the appropriate course of action without offering a medical diagnosis.

Throughout a 10-day development period, Sehat Sathi evolved from a basic talking demo to a sophisticated system capable of various tasks. It can engage in natural, code-mixed Hindi-English conversations with Indian voices, remember returning callers, locate the nearest government hospital based on the district, identify serious symptoms requiring human intervention, place real outbound calls, track call success, and hand off appointment-booking inquiries to a dedicated agent.

The system is built on a core pipeline similar to many other voice agents in the challenge: Speech-to-Text (STT) converts the caller's voice into text, handling Hindi, English, and code-mixed speech. A Large Language Model (LLM) processes the text, deciding when to employ tools and generating replies in the caller's preferred language and script.

Text-to-Speech (TTS) then converts the replies into natural-sounding speech, with Murf Falcon's Indian English voice Anisha delivering responses at a latency of 110-150 milliseconds.

Key features of Sehat Sathi include an Indian voice that sounds natural, guardrails preventing it from diagnosing conditions or prescribing specific medications, memory capabilities with explicit consent, a real tool with an honest fallback for facility lookup, human escalation for serious symptoms, and outbound calls introduced with proper identification. Additionally, a live dashboard tracks call success, and a specialist agent named Meera handles booking-related queries.

The project faced several challenges, including ensuring the voice sounded natural despite not hardcoding the locale, accurately interpreting caller names to maintain memory functionality, resolving issues with deprecated API models, recording phone calls without screen recording limitations, and establishing a stable connection for TTS streaming during outbound calls.