Trying Out AgentInspect
Imagine a travel agent answering: Tokyo is rainy today. Choose indoor sightseeing. Did it check the weather first? Or did it return that sentence without calling the weather tool? The final answer looks the same, but the execution paths are different. Checking that path matters. If a prompt should trigger a particular tool and the tool never runs, something has gone wrong before we even judge the…
AgentInspect is a TypeScript library that records an agent's execution locally and checks the process. It stores traces as JSONL files. The library allows defining rules to check if required tools were called, prohibited tools were avoided, or the run completed. For the experiment, two questions were separated: process checks and answer checks.
Process checks involved verifying if the agent called the weather tool according to the defined requirements, while answer checks focused on whether the returned text contained the expected information. The example was built using Vercel AI SDK and the official adapter. The code and output were checked with Node.js 22.22 or newer and pnpm 10.
The default demo did not require an API key and used AI SDK's MockLanguageModelV3 to supply predefined response sequences. The AI SDK executed the tool and passed its result into the next model call, with the adapter recording the execution. The example used metadata-only capture to keep the trace focused on execution structure, with tool and model calls recorded in the experimental_telemetry.integrations section.
The script then checks if the run met the defined contract, which specifies that the run must call the weather tool and that the allowed statuses are 'ok'.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
