{
  "id": 20808887,
  "title": "Your AI Agent Returned 200 OK and Still Got It Wrong: A Developer's Guide to Tracing Agents",
  "url": "https://urgent.news/2026/11/19/your-ai-agent-returned-200-ok-and-still-got-it-wrong-a-developers",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-19T21:50:13.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/vivekdaily_1122/your-ai-agent-returned-200-ok-and-still-got-it-wrong-a-developers-guide-to-tracing-agents-3mkj"
  },
  "original_language": "en",
  "account": "Agent observability is the ability to reconstruct the entire journey of an AI agent after the fact, including what the agent was asked, what it saw, what decisions it made at each step, which tools it called with which arguments, and what came back. This is achieved through four signals: traces, metrics, logs, and evaluations. Evaluations are the AI-specific addition, as an agent can return a 200 status code and finish quickly, yet still be wrong.\n\nAgent failures often occur silently, with errors compounding across steps. Even if each individual step is successful 95% of the time, the end-to-end success rate can drop significantly (e.g., 95% each for 10 steps becomes only 60% overall). Agents can also fail non-deterministically, making them difficult to reproduce. Additionally, success codes like 200 OK do not guarantee the agent performed correctly, and costs can accumulate without notice.\n\nTo instrument an agent, use OpenTelemetry's GenAI semantic conventions, which include dedicated operations for agent work such as create_agent, invoke_agent, and execute_tool. This allows you to capture all the necessary information without reinventing a trace format. Use the opentelemetry-sdk, opentelemetry-exporter-otlp, and opentelemetry-trace libraries to set up telemetry and tracer providers, and start spans for each step of the agent's run. By following these steps, you can effectively debug any failed traces and gain a comprehensive understanding of your agent's behavior.",
  "summary": "Here's a bug report you'll eventually get if you ship an AI agent: \"The refund agent approved a refund for a 7-week-old order. Policy is 30 days. Customer was thrilled. Finance was not.\" You open your dashboards. Error rate: normal. p95 latency: normal. HTTP status: 200 . The final message to the customer: polite, on-brand, perfectly formatted. Everything is green, and the agent still did the…",
  "key_points": [
    "Agent observability reconstructs entire AI agent journey",
    "Evaluations reveal AI-specific correctness of agent actions",
    "OpenTelemetry s GenAI conventions enable effective agent tracing"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}