Agentic AI and big data: what your platform needs before agents are useful

The promise of agentic AI on enterprise data holds great appeal: instead of a basic chatbot that answers one question at a time, it offers a system capable of planning, querying, self-checking, invoking additional tools, and delivering a complete analysis or even performing tasks. Integrating these two systems is challenging. Yesterday's pipelines often failed due to delays; agents aim to address this by opening tickets or drafting outreach lists.

However, the success of agents on real data depends more on the underlying infrastructure than the specific AI model used. An agent's performance differs significantly from a single-shot LLM feature. While the latter returns inaccurate information that a human easily recognizes and moves on, an agent can take a wrong step early, then build four more steps on top of it, generating an internally consistent but incorrect output that goes unnoticed without redoing the work.

Two main factors contribute to this difficulty: multi-step processes lead to compounded errors, and tool use carries real consequences. If each step in a chain has a 90% reliability, a five-step task is only 60% reliable end to end. When an agent can modify data, write tickets, or trigger jobs, hallucinations (incorrect information) are no longer just an annoyance but a serious incident.

The data platform serves as the primary bottleneck, not the model itself. Agents can retrieve data from documents or query tables. The ceiling for agents on table data is set by the semantic layer, not model quality. If a warehouse has inconsistent definitions of an 'active customer,' inconsistent grain between tables, or unnamed columns with unclear descriptions, an agent will behave similarly to a new analyst lacking proper context, selecting plausible information but ultimately being wrong.

Before pilots, focus on curated tables, not raw or production data. Point agents towards well-governed, cleaned gold tables with documented columns rather than raw or production data. Ensure each table and column has precise descriptions in a catalog. Agree on definitions for key metrics like revenue, churn, and active users. Establish known freshness by keeping agents informed about data lag.

Instead of granting broad SQL access, provide a small set of narrow, precisely described, deterministic tools for agents to use. This approach leads to fewer errors and easier debugging. When incorporating business logic, move it out of the agent's improvisation and into tested functions to avoid probabilistic evaluations. Define identity, permissions, and audit controls carefully.

Agents should query data under the requesting user's identity, log all tool calls with inputs, outputs, and requester, and limit write actions to service accounts with the narrowest possible permissions. Handle consequential actions, such as financial transactions or data modifications, through human approval or in a reversible sandbox.

Lastly, consider cost and latency at scale. Agentic processes amplify resource usage. A single user query may result in numerous model calls and queries, potentially overwhelming large data warehouses. To mitigate these issues, point agents at pre-aggregated tables, enforce query limits, row caps, and timeouts, and ensure agents operate within a sandbox for high-risk actions.