“We are not querying like humans anymore”: OpenSearch’s Bianca Lewis on why AI agent mistakes look like successes
This situation is more common than one might think: An AI agent calls the wrong API, gets a perfectly good The post “We are not querying like humans anymore”: OpenSearch’s Bianca Lewis on why AI agent mistakes look like successes appeared first on The New Stack .
AI agents are now making mistakes that look like successful responses, a shift that Bianca Lewis, executive director of the OpenSearch Software Foundation, calls the "revenge of the dinosaurs." These agents can call the wrong APIs and receive correct responses, leading to seemingly successful outcomes but actually taking the wrong path. This new way of querying differs significantly from how humans interact with applications, as developers are accustomed to spotting failed requests.
Lewis compares this situation to error 500 messages, which used to clearly indicate a failure. However, AI agents do not provide such errors anymore; instead, they might call the wrong API and still show a green dashboard. To address this issue, Lewis suggests tracing the agent's actions using OpenSearch and OpenTelemetry. By adding cost data to the trace, developers can identify where a run became expensive and where it deviated from the user's request.
The distinction between a single interaction and longer-lived information is crucial when evaluating the consequences of a wrong turn. Developers can use this evidence to revisit the context the application provided to the agent. Lewis emphasizes the value of putting search and observability on the same platform, especially for teams already using OpenSearch. This way, they can leverage their existing skills and consolidate business data in a single location, reducing the cost of changing platforms.
Written by urgent.news from The New Stack's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.