A Screenshot Is Not an Agent Failure Artifact

A screenshot of an execution tree provides little information for a reviewer to assess. It does not include key details such as which trace generated the screenshot, whether the trace was altered after capture, the redaction policy used, or if any warnings were removed during cropping. Reviewers can inspect the raw JSONL output to answer some questions, but this may expose sensitive identifiers or previews that should remain private.

The ideal artifact to share is a reviewable package containing evidence of the failure. This package should include an HTML summary, JSON summaries, redacted trace data, check results, redaction reports, and evidence.json file. The source traces themselves remain private while allowing the reviewer to offline verify the evidence package.

The bundle is generated using the agent-inspect tool, specifying the run ID, directory, profile and output location. The package separates source hashes identifying the pre-redaction data from file hashes protecting the final packaged contents. Verification of the package checks the manifest, file integrity, unexpected files, assessments, and provenance.

Editing the summary or replacing the redacted trace would invalidate the verification. While the package provides integrity checks, it does not offer cryptographic signing or proof of durability or replayability. The default share profile is suitable for internal use, while strict profile is recommended for public sharing. The bundle verification ensures the final evidence package matches its manifest, but additional steps are needed to ensure the package's contents do not contain sensitive information.