Why AI Agents Are Forcing Enterprises To Rebuild Their Data Foundations

Enterprises have been amassing vast amounts of data over the years. However, making this data usable for AI has proven to be a more challenging task. As organisations deploy AI systems to reason across their operations, they discover that simply having data somewhere is not sufficient. AI requires the ability to locate, interpret, and utilize the information within the appropriate context.

A recent Dun & Bradstreet survey reveals that 73% of Indian businesses have experienced measurable returns from AI investments, yet only 4% deem their enterprise data fully ready to support AI on a large scale. Bridging this readiness gap entails addressing the long-standing data challenges that have accumulated over the years. Customer identities may be fragmented across various systems, operational data may remain isolated, and information structured for reporting may not meet the needs of AI systems that require access and action.

Closing this gap necessitates the integration of disparate information. However, this alone is merely the initial step. Enterprises must then determine which data must be accessible in real time, how agents will comprehend the context, and the permissions governing access. These decisions significantly impact the infrastructure and models required to serve each use case, influencing the feasibility of deploying AI.

The discussion on how enterprises should navigate these decisions was the focus of a roundtable titled "Building The Data Foundation For AI At Scale," hosted by Inc42 in collaboration with Oracle at its CTO Summit 2026. The participants included Abhimanyu Saxena, co-founder of InterviewBit & Scaler; Dhruv Gupta, co-founder of Orchard Healthcare; Himanshu Garg, CTO of Kapture CX; Naveen Dachuri, CTO of Yulu; Palanivel Saravanan, head of infrastructure cloud engineering at Oracle; Priyanka Nahata, founder of CodeKarma Technologies; Syed Mohd Bilal, head of data science at Perfios; and Vinay Murthy, senior director of product at Zeta.

Led by Alexy Thomas, a partner at EY India, the roundtable deliberated on the importance of aligning business needs with data architecture, agent access, and deployment costs.

Gupta from Orchard Healthcare emphasized that the starting point should be the specific problem the business aims to solve, advocating for a backwards approach rather than starting with technology and searching for a use case. His team's six-month project to build a customer data platform exemplified this approach, which began only after defining how to enhance customer understanding, retention, and the broader business.

Bilal from Perfios echoed this outcome-focused strategy, stressing the importance of identifying the desired outcomes before selecting the appropriate tools. He noted the trade-off between building a system tailored to a specific use case versus investing in a platform that can support multiple applications. Bilal argued that while use-case-led development addresses immediate needs, a platform-first approach provides a broader foundation for future applications.

Palanivel from Oracle highlighted the necessity of understanding how data is structured and its meaning, especially when agents need to retrieve it efficiently. With data spread across multiple databases like Oracle, MySQL, PostgreSQL, and MongoDB, as well as unstructured knowledge, connecting agents directly to these systems presents challenges.

Agents may struggle to comprehend database schemas and generate efficient queries. To address this, Palanivel stressed the importance of a semantic layer that interprets data structure and meaning, making it more suitable for agent reasoning. He explained that traditional systems relied on humans to connect operational data with business decisions, but for autonomous agents, some reasoning must be encoded into the data layer itself, making a semantic layer increasingly critical.

Bilal from Perfios added that semantics can narrow the information an AI model searches through, enhancing retrieval efficiency. However, this contextualization requires input from individuals who possess deep knowledge of the data. Bilal argued that those closest to the data are best positioned to define its meaning consistently across various tools and platforms.

Nonetheless, ensuring that information is understandable does not mean every agent should have unrestricted access. Enterprises must establish boundaries around data access and usage permissions to mitigate legal and technical implications, particularly when employees use AI tools to work with business information.