Monitor on-premises and multi-cloud AI agents with AgentCore Observability
Set up Amazon Bedrock AgentCore Observability for AI agents running outside AWS: on-premises, on GCP, on Azure, or on developer machines. This walkthrough uses the AWS Distro for OpenTelemetry (ADOT) and IAM credentials to route session traces, span metrics, and token usage to the same AgentCore Observability dashboard.
When deploying AI agents built with frameworks like Strands Agents, LangGraph, and CrewAI, observability into their performance is essential, regardless of the environment they run in such as Amazon EKS, Amazon ECS, AWS Lambda, on-premises, or other cloud providers like Google Cloud Platform (GCP) or Microsoft Azure. To achieve this, Amazon Bedrock AgentCore offers a platform for building, connecting, and optimizing agents at scale, equipped with native tracing, monitoring, and analytics for agents deployed on AgentCore runtime within the AWS Cloud.
However, if agents run outside of AWS, additional configuration is required to send telemetry to the AgentCore Observability dashboard. This article demonstrates the setup of observability for agents running outside AWS, covering the configuration of AWS Distro for OpenTelemetry (ADOT) auto-instrumentation in non-AWS environments, routing telemetry to the AgentCore Observability dashboard, and validating the setup end-to-end.
The end-to-end observability pipeline involves the telemetry flowing from agents to the AgentCore Observability dashboard. Figure 1 illustrates this pipeline, showcasing how telemetry is routed from agents to the dashboard.
The solution outlined in this post utilizes the AWS Distro for OpenTelemetry (ADOT), which operates in-process with the agent application. ADOT auto-instruments the agent framework, capturing generative AI semantic convention spans and exporting telemetry directly to the Amazon CloudWatch OpenTelemetry Protocol (OTLP) endpoint using SigV4 authentication with AWS Identity and Access Management (IAM) credentials.
To route telemetry to the AgentCore Observability dashboard, three core components are required:
1. ADOT auto-instrumentation, which handles the intricacies of exporting telemetry from non-AWS environments.
2. IAM credentials, allowing ADOT to authenticate with CloudWatch and forward agent telemetry, including traces, metrics, and logs, to the AgentCore Observability dashboard.
3. Environment variables, containing specific OpenTelemetry settings related to routing and authentication.
As depicted in Figure 2, this cross-platform observability solution integrates multiple AWS services. CloudWatch serves as the foundation for telemetry ingestion and storage, while Amazon Bedrock AgentCore Observability adds specialized monitoring dashboards for AI agents. ADOT provides cross-platform instrumentation capabilities, and IAM ensures secure authentication between external environments and AWS.
To validate the setup, it is crucial to consider the prerequisites, including an AWS account with Amazon Bedrock model access (using Claude Haiku in this walkthrough), Amazon Bedrock AgentCore Observability and designated log groups, CloudWatch Transaction Search enabled in the account, Python 3.10 or later, IAM user credentials with necessary permissions for bedrock:InvokeModel, logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents, xray:PutTraceSegments, xray:PutTelemetryRecords, xray:GetSamplingRules, xray:GetSamplingTargets, cloudwatch:PutMetricData, and outbound HTTPS access to AWS endpoints.
Written by urgent.news from AWS Machine Learning's reporting — not their text. Machine-written — it may contain errors, so check the original before relying on it.
