From Flat Logs to Execution Trees: Debugging Modern AI Agents

An agent trace typically appears as a series of events due to the simplicity of creating an append-only data structure: span_started, span_started, span_ended, span_started, span_ended, span_ended. Developers prefer to debug the causal structure rather than the raw sequence. They construct an execution tree to visualize the relationships between operations.

However, building a reliable tree involves more than merely sorting events by timestamp. Events may arrive out of order, siblings may execute concurrently, spans may be incomplete, and retries may fail while the parent operation still succeeds.

To construct a reliable execution tree, each event must have stable trace and span identity. Start events assign a parent; end events designate outcome and duration. In JavaScript, the SpanKind type can be run, model, tool, retrieval, decision, or fallback. The TraceEvent type includes event (span_started or span_ended), traceId, spanId, parentSpanId, name, kind, and timestampMs.

Timestamps alone are insufficient to establish parentage. Two events occurring consecutively might be siblings, unrelated concurrent work, or operations from different traces. To address this, developers must explicitly assemble events into spans. The AssembledSpan type includes traceId, spanId, parentSpanId, name, kind, startedAtMs, endedAtMs, status, errorCategory, and metadata.

The AssemblyDiagnostic type signals assembly issues, such as duplicate_start, duplicate_end, end_without_start, or span_left_open. The SpanAssembler class manages the pool of spans, pending end events, and any encountered diagnostics.