There's no reason for software to be slow anymore

In a viral tweet, someone claimed that concerns about LLMs leading to slow, bloated code would be forgotten once everything was rewritten in optimized assembly. However, the cost of specialized performance work has dropped significantly, making optimizations that were once too expensive for all but the largest projects accessible to anyone with a few sentences.

Marc Brooker agreed, noting that dynamic custom software, tailored to a specific workload rather than a class of workloads, seems likely. This reminds him of FFTW and old demoscene techniques that focused on speed and small size for particular problems and hardware. The meme that "code was never the hard part" is true in some domains, but in others, writing code was indeed challenging.

JIT compilers are an example, and with the help of LLMs, implementing a JIT compiler has become easier. Marc introduced pgrust, a project that aims to lower the barrier to entry for JIT compilers. Marc also mentioned databases, which were once the hardest software to build due to limitations. Now, with AI, building software is more ambitious.

He used FRE, a regex engine built by an agent over a month, which initially overfit to a specific benchmark but generalized well after warning the agent about a holdout benchmark. FRE's native AOT compiled version performed well in longer searches, and Marc experimented with running the native code compiler in another thread while ripgrep's normal matcher ran, achieving better performance for longer queries at the cost of shorter queries.

Marc also discussed the possibility of building a regex engine in a few minutes of human time, a task that would typically require significant time and expertise. He concluded that while creating an index for faster local searches might be more straightforward, the point is that this technical work, once time-consuming and demanding expertise, can now be done trivially.