Evaluating an Agent: Trajectory vs Outcome
Outcome grading is the obvious choice and mostly the right one: did the end state match what was asked? It is cheap, objective and hard to game. It is also blind to a set of failures that will hurt you in production, and every one of them is invisible in a green test run. Outcome grading, and what it cannot see Run the agent, inspect the final state — the file’s contents, the database row, the…
Outcome grading is the most common and practical method for assessing an agent's performance. It is simple, objective, and difficult to manipulate, but it does not account for certain types of failures that can have significant consequences in production. When an agent only evaluates the final state of an outcome, it may overlook important issues such as right answer with wrong reasoning, right answer after too many steps, collateral damage, attempts that were denied and then successful, and occasional success. These factors are not visible in a green test run and can lead to unforeseen problems.
Trajectory, on the other hand, provides a more comprehensive view of an agent's behavior by examining the sequence of steps taken during the task. This includes tool names, arguments, results, and the model's messages. While grading trajectory does not involve comparing it to a reference sequence, it is still important to analyze the properties of the path, such as tool recall, redundancy rate, cost to completion, illegal attempts, and pass^k.
These metrics offer valuable insights into the agent's effectiveness and reliability, especially in scenarios where one attempt may not always be sufficient to achieve the desired outcome.
In addition to trajectory analysis, it is crucial to incorporate process invariants as part of the evaluation process. These are properties that must always hold true for every run and case, similar to unit tests. Examples include ensuring that a file is not written to without first being read, no mutating tool appears before a read, and no tool call contains credential patterns.
These invariants act as strong safeguards against potential misbehavior and can be combined with trajectory assertions to create a comprehensive evaluation framework.
When building an evaluation set, it is essential to use real production traces, rather than fabricated scenarios. This ensures that the tasks are representative of the actual challenges the agent might face in a production environment. Additionally, including impossible and out-of-scope tasks, as well as a hostile case where the agent's instructions are ignored, can help measure the agent's resistance to false confidence and potential attacks.
Lastly, running each case multiple times (k) and analyzing the pass^k metric provides a more realistic estimate of the agent's overall reliability.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
