Our first step at defining a CPU benchmark for agentic AI
Over the past six months, CPUs have taken center stage as agentic AI workloads evolve. Here’s our first attempt at trying to measure CPU performance when running agentic workloads.
The emergence of agentic AI has caused a significant stir in the CPU industry over the past six months, yet there remains a lack of solid and repeatable benchmarks to evaluate the performance of the latest hardware in this area. In response, the writer and their team have been developing an internal set of tests to evaluate one aspect of agentic loops end-to-end, isolating CPU and memory performance from the accelerator.
This work-in-progress aims to provide insights into how these components contribute to the agentic loop. The team has prototyped the test and settled on a central design, but further work is required to ensure the accuracy of the produced numbers.
Agentic AI is described as a "loop," as tasks often require calling a tool multiple times or different tools. For instance, when working on a Python project, the LLM may be prompted to debug a script, calling various tools such as list files, read file, search code, edit file, and run command. This process forms an agentic loop, where the LLM prompts, evaluates, calls tools, receives information, re-evaluates, and may call more tools in a loop until it obtains the desired outcome.
The benchmark is designed to assess how agentic performance scales across the CPU, providing an ideal throughput for each chip while maintaining interactivity. Currently, the tests are meant to supplement, not replace, other testing of local AI. As the hardware characteristics become better defined, the team will be able to make more informed decisions about the future direction of this testing.
Written by urgent.news from Tom's Hardware's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
