The Shopkeeper Who Never Gets a Break
Walk into any small shop in India — a kirana store, a general store, a corner medical shop — and you'll notice something. The owner is doing five things at once. Weighing rice for one customer, answering "are you open on Sunday?" from another on the phone, mentally tracking that Ramesh still owes ₹200 from last week, and somewhere in there, trying to remember if they need to reorder sugar. None…
In any small shop across India, the owner is invariably juggling multiple tasks simultaneously. They weigh rice for one customer, field questions about Sunday operations on the phone, mentally track outstanding payments, and attempt to recall whether sugar needs to be reordered. All these actions are mindfully handled without any external aid, as there's no structured documentation or formal system in place.
Interruptions, such as unexpected phone calls, further strain the owner's time, which is already limited. This inefficiency isn't due to a lack of technology; rather, it's the need for assistance that relieves the owner from numerous small burdens. The core issue isn't the necessity for an app, but rather the requirement for someone to alleviate fifteen minor responsibilities without demanding that the owner learn anything new.
The ideal interface, in this scenario, is one that aligns with their existing mode of communication: talking. The author initially envisioned creating a dashboard and form-based product, but soon realized that such a tool wouldn't be practical for the busy shop owner. They prefer interacting through voice, just as they would with an assistant standing beside them.
This led to the creation of Dukaan Mitra, a voice agent that operates more like a second person within the shop, remembering details, avoiding fatigue, and knowing when to seek assistance. The assistant is named Dukaan Mitra, which translates to 'shop friend'. The underlying technology for Dukaan Mitra mirrors that of any voice system: speech-to-text converts spoken words into text, a large language model (LLM) determines the appropriate response, text-to-speech converts the text response back into speech, and a real-time transport layer ensures seamless communication.
The pipeline starts with the caller's voice being captured, followed by Deepgram STT transcribing the speech into text. The Gemini LLM then analyzes the text to generate an appropriate response, which is subsequently processed by Murf Falcon TTS to produce coherent speech. This entire process is facilitated by LiveKit, ensuring real-time transmission of data.
However, building the technical framework was only the first step. The more critical decisions revolved around the agent's behavior: it must refrain from making decisions on its own, such as confirming discounts it wasn't involved in negotiating, providing profit figures to strangers, or predicting delivery times without explicit instructions.
Instead, the agent learns to ask for permission before acting independently. It inquires whether it can remember specific information for future interactions, and respects the owner's decision if they decline. The agent also knows when to escalate issues to a human, like resolving disputes or handling refunds. If a customer questions a charge or requests a refund, the agent explains its intention to hand over the case to a human, obtains permission, and creates a formal request with a reference number, mirroring the actions of a competent employee who would escalate matters beyond their authority.
The agent also proactively calls when stock levels for certain items are low, ensuring timely reorders without waiting for the owner to notice. The final product is not a chatbot with a shop theme, but rather a system designed to behave like a reliable employee. It is confident in its knowledge, transparent about its limitations, cautious with the information it shares, and ready to delegate tasks to a human when necessary.
The primary difference between an AI capable of conversing and one that can be trusted to handle shop-related inquiries lies in its restraint. This restraint, rather than advanced capabilities, enables the creation of an AI that can interact with shop owners in a trustworthy and effective manner. For those looking to develop a similar voice agent, the recommended structure includes speech-to-text, LLM, text-to-speech, and real-time transport as the core components.
A minimal database, such as SQLite, suffices for memory storage. Most importantly, the agent's limitations should be defined before its features. Clearly outlining what the agent should never claim, decide, or share is crucial for building trust. It's equally important to thoroughly test the failure scenarios as much as the success paths, as these are often where the most significant issues arise.
The source code for Dukaan Mitra is available on GitHub for those interested in exploring the implementation further.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
