VeloDB’s Nitin Waghulde on why AI agents break down after the demo

AI agents often appear impressive during demonstrations, but their performance in production environments is a different story. A key barrier to their success is the data layer, which can struggle to keep up with the demands of these agents. Gartner predicts that over 40% of agentic AI projects will fail due to escalating costs, unclear business value, or inadequate risk controls.

Nitin Waghulde, Regional Director, APAC at VeloDB, highlighted this issue in his keynote titled 'AI agents at scale: Surviving beyond the demo'. He explained how AI agents differ from humans in how they consume data. While people may run a single report to get a sales figure, an AI agent would pull order records from various systems, scan support tickets, and read sales team reports to answer the same question. This leads to hundreds of lookups for a single query, putting a heavy load on the data layer.

The challenge is exacerbated because business data is scattered across multiple sources, including transactional records, chat memories, documents, dashboards, pre-built reports, and vector stores. Each request to these sources adds cost and the potential for outdated information. Nitin compared this to driving with an inaccurate traffic map.

To address these issues, Nitin outlined four requirements for AI agents moving from pilot to production: access to all data types, the ability to handle large data volumes, immediate visibility into data changes, and a record of every interaction. VeloDB's solution is a single engine that handles these needs, supporting hybrid queries that combine SQL, vector search, and text search. This approach provides a unified source of truth, reducing the back-and-forth trips that slow down agents.

VeloDB, founded by the creators of Apache Doris, has built a system that can ingest data at 10 Gbps and query about 10 petabytes in under two seconds at a lower cost. This infrastructure supports petabyte-scale logging for companies like MiniMax, a Chinese AI company that serves 212 million users across 200 countries. By focusing on real-time, trustworthy data from a single source, Nitin argues that companies can move their AI agents faster, reduce costs, and make better decisions.