Observability in Microsoft Foundry: Tracing Agent Runs, Continuous Evaluation, and the OpenTelemetry Data Plane
Day 10 of the Microsoft Foundry 100 Days / 100 Blogs series. You shipped an agent. It calls a model, invokes two tools, retrieves a few documents, and returns an answer. It works in your dev loop. Three weeks later, a support ticket lands on your desk: "the assistant gave a wrong price for SKU-4471." You have no idea which of the six internal steps produced that number, whether the tool returned…
This article explores the observability features of Microsoft Foundry, specifically focusing on how it captures and queries the execution details of agent runs using OpenTelemetry. Agents in Foundry can involve multiple operations such as model calls, tool invocations, retrieval queries, and other span-level tasks which can all have varying latency, token costs, failure modes, and error introduction points.
Without proper tracing, debugging such agents becomes a tedious process of searching through logs and making educated guesses.
Microsoft Foundry utilizes OpenTelemetry, a CNCF standard for distributed tracing, metrics, and logs, to implement its tracing model. In this model, a Trace represents the entire journey of a single request through the system, uniquely identified by a trace_id. A Span is a single unit of work within a trace, like an LLM call, tool invocation, or retrieval query.
Spans have attributes such as model name, token counts, tool name, and more, which are populated using the OpenTelemetry GenAI semantic conventions. This ensures that traces generated from different GenAI frameworks can be rendered uniformly across tooling.
Traces in Foundry are exported via a trace exporter to Azure Monitor Application Insights, which acts as the single source of truth for observability data. This means Foundry doesn't maintain a separate proprietary trace store; instead, it leverages the existing Application Insights infrastructure, benefiting from its mature tooling, familiar query language (KQL), and existing security and cost considerations. The Foundry portal provides a read-through view of this data, eliminating the need for data duplication.
The article delves into how tracing works at runtime, setting up traces both server-side and client-side, and reading traces through the waterfall view. It also discusses continuous evaluation, multi-agent tracing, and the importance of security, cost, and common pitfalls in implementing tracing. Overall, it highlights the architectural decisions behind tracing in Foundry and provides practical recommendations for effective observability implementation.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
