Urgent.News

What's breaking now, across thousands of outlets.

AI

Catching Silent AI Agent Failures in Production: Metrics, Drift and Hallucinated Actions

In Part 1 we instrumented an AI agent with OpenTelemetry and learned to read a trace: one run, broken into spans, showing exactly where a refund agent skipped a failed policy check. But nobody reads every trace. In production you need the system to tell you which traces to read . That's what this post is about: The metrics and alerts that actually matter for agents Agent drift : when behavior…

Catching Silent AI Agent Failures in Production: Metrics, Drift and Hallucinated Actions

In order to effectively monitor AI agents in production, it is crucial to establish appropriate metrics and alerts. This post discusses various aspects of observability for AI agents, including failure detection, drift, hallucination, and cost optimization.

Layered view of observability:

1. Track task success, escalation rate, and user feedback to measure direct outcomes

2. Assess quality by evaluating groundedness, correctness, and policy compliance

3. Monitor steps per task, loops, plan changes, and handoffs

4. Analyze tool error rates, argument validity, and unexpected tool usage

5. Identify cost and speed metrics, such as tokens, dollars per task, and latency

6. Monitor blocked inputs, injection signals, and PII in output to ensure safety

Key alerts to set up:

1. Success rate drop: Drop in success rate over a week

2. Looping agent: Maximum steps per run exceeding a threshold

3. Repeated tool calls: Repeated calls to a specific tool

4. Unexpected tool usage: Usage of a specific tool for a specific task type

5. Cost spike: A sudden increase in cost per task compared to the previous 24 hours

Observability, evaluations, and red teaming should be integrated into a single loop for comprehensive monitoring.

Selecting a tool and getting started:

1. Choose a suitable observability tool that aligns with your backend

2. Set up a few initial alerts based on the provided examples

3. Start with task type slicing to understand failures specific to each flow

4. Link every alert to a trace to pinpoint the exact run that triggered the alert

5. Review sessions to catch attacks and leaks that may span multiple turns

Agent drift:

Agent drift refers to changes in agent behavior without any code updates. This can occur due to model updates, prompt/policy edits, tool/data changes, or shifts in user input. To detect agent drift:

1. Pin and tag versions of models, prompts, and knowledge bases (KBs)

2. Compare behavior before and after every change, focusing on more than just success rates

3. Use tools like behavior_profile to analyze key metrics, such as success rate, average steps, cost, escalation rate, and policy score

By implementing these metrics, alerts, and drift detection methods, organizations can proactively identify and address issues with their AI agents in production, reducing the risk of costly failures and improving overall system reliability.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in AI

AI Is Thinning the Pipeline of Future Dealmakers

AI-driven headcount cuts save payroll today, but middle-market dealmakers warn they risk hollowing out the talent pipeline. The post AI Is Thinning the Pipeline of Future Dealmakers appeared first on Global Finance Magazine .

Bitcoin Is a Better Bet Than COIN Despite Anthropic Narrative, Analyst Says

Bitcoin (CRYPTO: BTC) still beats Coinbase (NASDAQ:COIN) stock as an investment, according to Milk Road’s in-house analyst Martin. Analyst Sees Attractive Risk-Reward In Bitcoin Bitcoin traded near $63,000 Wednesday, down roughly 50% from its all-time high, after closing six straight weekly candles in the red, something that has happened…

Google’s Gemini App Can Now Operate Apple Desktop Computers

Google’s Gemini app for Mac can now use that Apple computer to complete multi-step tasks across the user’s local files, desktop apps and the web, Google said in a Thursday (Nov. 19) blog post. “This saves you time on the repetitive work that normally takes dozens of clicks,” Michael Friedman, group product manager, Gemini App, […] The…

More from Thursday 19 November →