Your agent waits a full second to send the number 3

Your agent waits a full second to send the number 3. The source material reveals that a timer placed around any agent loop consistently shows a pause of one to three seconds, followed by a single small action. In this case, the agent decides to click element 3 and keep the file, signaling that the build is complete. During this pause, the language model is analyzing the current page and identifying the most relevant navigation option to the user's goal, which in this instance is the third link in the results list.

The agent then parses this information and transmits a value between 1 and 30. The entire process takes approximately two of wall clock time and a few hundred tokens to complete. The key takeaway from this story is that language models are highly efficient at answering multiple-choice questions, but they are wasteful when generating prose to answer the same question.

A more efficient approach would be to provide the model with a set of options and have it select the most appropriate one. The story also introduces Jev, a system developed by TypeSafe, which takes a state plus typed questions and returns answers with confidences in a single forward pass. Unlike traditional language models, Jev does not generate token streams and therefore does not require waiting for them.

This system has several limitations, including being API-only and not providing a self-hosted path. However, it offers significant cost savings and speed advantages over traditional language models.