Building KisanVani (किसान वाणी): An Ultra-Low-Latency Multilingual Voice Agent for Indian Agriculture with Murf Falcon & LiveKit

Building KisanVani: An Ultra-Low-Latency Multilingual Voice Agent for Indian Agriculture with Murf Falcon & LiveKit

The Problem & The Mission

Across rural India, millions of farmers depend on timely agricultural information to make decisions that impact their livelihoods. However, traditional apps struggle due to language barriers, low literacy, and complex UI. To address this, KisanVani was built as an empathetic, expert AI voice assistant for farmers, allowing them to interact in Hindi, Hinglish, or Indian English and receive real-time guidance on sowing, weather, pest alerts, and market rates.

System Architecture & Core Stack

KisanVani utilizes a tight integration of speech processing, language modeling, dynamic tool invocation, and streaming synthesis. The speech-to-text engine is Deepgram Nova-3, supporting multi-language recognition. The text-to-speech component is Murf Falcon 2, delivering ultra-fast (under 200ms) streaming audio with natural Indian English and Hindi pronunciation.

Google Gemini 2.5 Flash powers the large language model for fast reasoning, function calling, and multilingual fluency. LiveKit Agents SDK, including Silence VAD and multilingual turn detection, manage the pipeline. Persistent memory is stored in an SQLite database, ensuring privacy-compliant caller profiles and outbound telephony logs.

Key Features

KisanVani delivers ultra-low latency Indian voice synthesis using Murf Falcon 2 with sentence-level tokenization, achieving sub-250ms audio generation. The agent operates under strict safety guardrails, refusing non-agricultural requests, directing out-of-scope queries to the Kisan Toll-Free Helpline, and adhering to zero hallucination policies.

Multilingual support is provided for Hindi, Hinglish, and English, with a rule enforcing native script output for non-English languages. Caller memory is opt-in, ensuring privacy, and enabling personalized interactions. Real-time agricultural tools include weather forecasts, mandi price retrieval, and dynamic tool chaining for seamless information flow.

Proactive outbound alerts can be sent for emergency weather or pest warnings, following a strict human escalation protocol for complex issues.

Code Snippets & Walkthrough

The core pipeline and dynamic handoff are configured in Python, utilizing LiveKit plugins for voice AI processing, Gemini LLM, Deepgram Nova-3 STT, Murf Falcon for TTS, and Silero for turn detection. The agent's configuration includes setting up the speech-to-text, language model, and text-to-speech components, ensuring efficient and accurate interactions.