Astra for Coding: Why Are We Doing This Again?

The latest iteration of AI, Astra, has been generating quite the buzz among AI engineers. Some are calling it "Neijuan," a term that describes a system of endless effort and competition without improving output. It's similar to the 996 work culture in China. When considering Astra's performance, it's an incredibly impressive model that can handle complex tasks with ease, such as image recognition and understanding complex topics.

However, for software engineering purposes, it's not yet clear how to utilize it effectively. In a weekend-long experiment, the reporter set up a software factory to let the model decide the workflow entirely, but the factory produced nothing of value after consuming 4 billion tokens. The model also exhibited some peculiar behaviors, such as using excessive Python code for tool calls and resorting to manual string manipulation, even in situations where editing tools were available.

This could be due to the model being rewarded for long-horizon tasks without penalizing for "shitty code." Astra's output is impressive but often unreadable and hard to understand, making it challenging to follow along with what it's doing.