AI Is Moving From Chatbots To Agents: Is India Ready For What Comes Next?

India's dialogue on artificial intelligence (AI) has primarily focused on computing prowess, data centers, language models, startups, and the aspiration to become an AI superpower. However, the conversation is shifting as questions of existential risk arise. Finance Minister Nirmala Sitharaman recently described AI as a double-edged sword, warning of potential misuse for cyberattacks, fraud, misinformation, and election interference.

This sentiment echoes within companies developing "frontier" AI systems, with leaders questioning the rapid pace and trajectory of the race. Anthropic CEO Dario Amodei has argued for slowing the pace of AI model improvements, while OpenAI CEO Sam Altman has acknowledged the need to manage risks associated with increasingly capable systems.

The global AI landscape is currently grappling with doomsday fears amidst a mounting US-China AI rivalry. India's position in this unfolding narrative is complex. While the country is eager to showcase its technological advancements, the topic of AI safety is often dismissed as an inconvenience. There is concern that excessive regulation could hinder innovation and prevent India from achieving its AI ambitions.

Nevertheless, AI safety is not a mere luxury but a fundamental aspect of ensuring the responsible deployment of AI systems. Ignoring safety issues may initially appear to facilitate innovation, but it can eventually become a hindrance to widespread adoption. As AI systems become more autonomous, the potential consequences of their failures become increasingly severe.

In India, the discussion of AI safety is often framed around consumer harm—scams, deepfakes, and job displacement. However, the global conversation has expanded to encompass the risk of rogue AI agents that can act independently, make decisions, and take actions with minimal human oversight. This distinction is crucial. While chatbots like ChatGPT primarily respond to prompts, AI agents can pursue goals, make choices, utilize tools, and execute actions with limited human intervention.

The potential cybersecurity risks associated with such autonomous agents are significant. India may not yet be developing frontier models, but it is not immune to the risks posed by them. Mumbai-based AI and privacy lawyer Dr. Prashant Mali emphasizes the importance of AI-safety sovereignty. He argues that India must move beyond merely regulating what AI systems say and instead focus on regulating what they can autonomously do.

India has adopted the India AI Governance Guidelines, but many of the proposed safety measures are currently voluntary and lack legal enforceability. These guidelines recognize the existence of autonomous AI agents and multi-agent collaboration, signaling the need for a reevaluation of governance approaches. India's current legal framework is reactive, addressing harm after it occurs.

However, there is a pressing need for clearer liability rules to determine responsibility among developers, deployers, platforms, and users. While the government has announced the establishment of an India AI Safety Institute to conduct safety research, standards, testing, and evaluation, the institute is not yet a statutory regulator with the authority to license frontier models, impose shutdowns, or compensate victims.

As of July 29, 2026, India still lacks the necessary legal infrastructure to effectively protect against AI-related risks. While India can learn from its Asian counterparts, such as Singapore, there are still challenges to overcome. Singapore's 2026 Model AI Governance Framework for Agentic AI offers a promising approach. This framework emphasizes the need to assess and bound the risks of AI agents before deployment, limit their autonomy and access to tools and data, and establish points at which human approval is required.

In one instance, agentic coding assistants were initially confined to internal employees, low-risk systems, and without external tools, while safeguards such as logging, monitoring, and other protections were tested. China appears to have realized the importance of such safeguards, treating AI agents as employees with restricted access rather than digital CEOs with unrestricted access.

India should consider implementing mandatory evaluation of frontier models, including agentic AI risk classification, human override and kill-switch requirements, compulsory incident reporting, developer-deployer liability rules, and mandatory AI insurance for high-risk autonomous systems. Failing to address these issues adequately could result in India being well-equipped to investigate accidents after the fact, rather than proactively preventing them.

The recent scare surrounding AI safety has brought the topic to the forefront. As AI continues to innovate, it is essential for India to determine the appropriate level of authority an AI system should have before human intervention becomes necessary. The question remains: how much control should be granted to AI systems before a human must step in?