AWS CloudWatch Omni goes after the hardest question in agentic AI: Why did the agent do that?
For decades, the observability industry has answered one basic question: Is it running? Agentic artificial intelligence breaks that model. An agent can return a clean response, meet its latency target and throw no errors, yet still give a customer the wrong answer, call the wrong tool or pull from a stale knowledge base. By every […] The post AWS CloudWatch Omni goes after the hardest question in…
Amazon Web Services has launched Amazon CloudWatch Omni, a new observability solution targeting the challenge of monitoring agentic artificial intelligence systems. Unlike traditional monitoring, which focuses on whether AI agents are running smoothly, Omni shifts the focus to understanding why agents make incorrect decisions.
Key features of Omni include an evaluation engine that scores agents on coherence, helpfulness, faithfulness, and routing correctness, among other metrics. Teams can compare different prompts, build test datasets from production traffic, and automatically detect quality regressions. Continuous evaluation against live traffic ensures that any quality drift is immediately flagged, much like how a CPU spike would be noticed.
Sony is an early adopter of Omni, using it to oversee hundreds of agentic AI workloads, from proof-of-concept to production. With Omni, Sony can quickly move from a single trace to evaluation, AI analysis, comparison, or dataset creation. This is crucial for companies juggling dozens or hundreds of agents, each built by different teams with varying ideas of what constitutes "good" performance.
Developers and operators can use Omni through a native Visual Studio Code extension, Cursor and Kiro support, or a standalone web experience. The solution provides a single data layer for traces from developers debugging locally and operators investigating live systems. By enabling the AWS DevOps Agent by default in investigation sessions, Omni correlates signals and maintains a full investigation history, reducing the need for multiple tools and meetings to connect the dots.
Capital One, a design partner in Omni's development, highlighted the importance of data portability and ownership for regulated industries like finance. Omni captures investigation history, serving as audit evidence for handling AI incidents, a critical aspect in highly regulated sectors.
Omni is built on OpenTelemetry, supporting various agent frameworks such as LangChain, LangGraph, CrewAI, OpenAI Agents SDK, Strands, and third-party evaluators. It also integrates with Amazon Bedrock AgentCore agents and Azure ingestion, ensuring portability across multiple cloud environments. The IDE extension is free, while customers pay for the telemetry they send and store. Dashboards, alerts, and queries up to five times the monthly ingestion volume are included at no additional cost.
Written by urgent.news from SiliconANGLE's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
