AI Agents Can Now Optimize Your Slow Java Code: A Spring Boot Workflow That Used to Need a Specialist

Dan Luu, a renowned engineer known for his performance writing, recently shared an eye-catching tweet that sparked a viral discussion. Luu argued that rewriting software in hand-optimized assembly would solve concerns about LLM-generated bloat. The essay he penned, titled "There's no reason for software to be slow anymore," garnered significant attention on Hacker News, receiving 620 points in just a day.

The essay's core claim is straightforward and grounded in real experiments: performance work that once required a rare specialist can now be accomplished by anyone with basic typing skills. To illustrate this point, Luu used his own ripgrep usage as an example and demonstrated how an agent could optimize it, taking only about 2 minutes of his time.

Jamie Brandon, a performance engineer, tested Anthropic's public performance takehome exercise and let Claude handle the optimization after Brandon had worked on it. Claude outperformed Brandon's initial efforts, with some optimizations being so innovative that Brandon had not considered them before. For those familiar with Spring Boot services, the implications are clear.

While the examples provided may seem niche, the underlying principle - that measuring and attempting optimizations now takes mere minutes rather than days - applies to the slow endpoints that most real codebases inevitably accumulate. In this article, the author presents a practical workflow for using AI agents to optimize a slow Spring Boot hot path without producing subpar results.

It's important to note that the performance numbers cited from Luu's essay are based on his own experiments, not the author's. The workflow outlined below is adapted from the structure of Luu's agent loops and can be directly implemented. The reason this shift in optimization approaches matters more in Java than in any other environment is due to the historical Java tendency to prioritize performance optimization less.

The conventional wisdom has always been that the JVM's JIT compiler handles performance, so writing straightforward code and leaving optimization to HotSpot is the way to go. This was a reasonable approach when investigation costs for optimizations were high, as they would entail a specialist's time. However, when the investigation costs drop to just two minutes of typing, the rationale behind this advice becomes questionable.

Consider the optimizations that were deemed too costly on your previous projects. Replacing a stream chain in a hot loop with a plain indexed loop after profiling revealed boxing overhead. Caching a compiled Pattern instead of calling Pattern.compile per request. Switching a JSON serialization path to avoid reflective lookups on every call.

Tuning a bulk JDBC fetch size that defaulted to fetching rows one at a time at a time. These optimizations were often skipped because nobody had the time to prove their worth. Luu explains this calculus by comparing the estimation of optimization value to the time required to verify it. When the time required to verify an optimization drops by three orders of magnitude, the decision to pursue it becomes much easier.

This leads to an increase in the number of optimizations worth trying, including those that were previously uncertain. Michael Malis, quoted in Luu's essay, takes this idea even further, suggesting that with AI, we could tailor optimizations to a specific customer's workload as needed. Essentially, software could be customized to fit a particular workload rather than being generic.

To demonstrate how this concept could work in practice, the author provides a workflow for optimizing a slow Spring Boot endpoint using AI. The example chosen is a user-agent parsing endpoint, which is relatively slow but common in services that log traffic or perform analytics. The workflow consists of capturing real workload data, structuring the optimization loop, and employing a holdout gate to prevent overfitting.

First, real workload data must be captured from production, split into train and holdout sets, and frozen. This is crucial because AI agents may tend to overfit to synthetic workload generators. The author provides a Java class, Splitter.java, that can be run once to create two files: one for training and one for holdout testing.

By shuffling the user-agent strings and splitting them into two sets (40,000 for training and the rest as holdout), the workflow ensures that the agent optimizes against real-world data and tests its results against unseen data, preventing overfitting.