Urgent.News

What's breaking now, across hundreds of outlets.

AI

How to Monitor Agents Built with Pydantic AI or Google ADK

To monitor agents built with Pydantic AI or Google ADK, export the OpenTelemetry traces both frameworks already emit to a monitoring backend, and attach a user and session ID to each run. For production monitoring of whole conversations, Agnost AI is the pick: it receives those spans over OTLP, groups them by user and session, judges every conversation against plain-language rules, and alerts in…

To monitor agents built with Pydantic AI or Google ADK, export the OpenTelemetry traces emitted by both frameworks to a monitoring backend. Connect a user and session ID to each run. For comprehensive monitoring of entire conversations, consider using Agnost AI, which aggregates spans by user and session, validates conversations against plain-language rules, and sends alerts via Slack or email when issues arise.

Pydantic Logfire is the native tool to inspect Pydantic AI runs, while Google Cloud Trace serves as the default destination for ADK. Both frameworks adhere to the OpenTelemetry semantic conventions for generative AI, enabling the same backend to read traces from either framework. However, setup differs: Pydantic AI requires the Logfire SDK (or direct OpenTelemetry setup), while ADK needs its own OTLP configuration plus an optional plugin for user IDs and transcripts on the root span.

Agnost AI, an open-source tracing and evaluation workbench for ADK, is recommended for production monitoring with alerts and fix PRs. Pydantic Logfire is ideal for inspecting Pydantic AI runs during development. ADK on Google Cloud offers a built-in tracing solution, compatible with other GCP services. Phoenix or Langfuse, both listed in ADK's integrations catalog, can also be used for tracing and evaluation. The table below provides a quick comparison between the tools.

Rewritten by urgent.news; may contain errors — check the original.

Read the original at dev.to →

Report a problem with this story

More in AI

I built an open-source AI gateway for Kubernetes: budgets, kill switch and evals in CI

Every team wants AI. Then three questions arrive: What does it cost? Where are the keys? Can we stop it? I'm a platform engineer, and I built SanVi to answer all three on Kubernetes.

  • Sandhilk created open-source AI gateway called SanVi for Kubernetes
  • SanVi manages budgets, kill switch, and evaluations in CI
  • System logs all calls with team, model, tokens, and costs in Postgres

How to write prompts for AI video models: a six-part formula (Seedance, Kling, Veo)

Video models like Seedance 2.5, Kling 3.0 and Veo 3.1 understand plain text, but a short prompt such as "a girl dances on the street" almost always gives a random result: the camera jitters, the…

  • Subject: Specify single main subject with 2-3 details, e.g., "young woman in oversized white hoodie"
  • Action: Describe one action for the entire clip, e.g., "dances to upbeat beat"
  • Camera: Include one camera move per clip, e.g., "slow orbit around her"

More from Saturday 21 November →