Express Node.js Structured Logs — Poll Error Search for 30-Day Cost Attribution

The article discusses a structured logging approach for an Express Node.js service that utilizes AI-agent step completion. The service should send structured logs for each completed step, which includes relevant information such as tenant ID, workflow ID, run ID, step ID, model identifier, token counts, latency, and outcome. The logs should be concise, focusing on the essential fields necessary for diagnosis and attribution.

Key points include:

1. Sentient the Express service to emit logs after each step completes, ensuring duration, outcome, and usage metrics are known.

2. Include relevant fields such as tenant ID, workflow ID, run ID, step ID, model identifier, input and output token counts, elapsed milliseconds, and outcome.

3. Add an error class and sanitized error message for failure events.

4. Retain only the necessary fields, excluding bulky request and response bodies to minimize storage costs.

5. Consider three logical classes of events: attribution events (longer retention for reconciliation and trend comparisons), diagnostic excerpts (shorter retention and stricter access), and alert-delivery records (fingerprint, attempt state, and destination response category).

6. Store sanitized failure details for debugging specific runs, but avoid storing full conversational content.

7. Measure latency for the entire agent run and individual steps separately, without adding step durations to the wall-clock latency.

8. Use stable names and explicit units for fields, following Prometheus naming guidance for consistency in log-to-metric transformations.