Robot brain builders are pushing out of their GPT-2 era

Physical AI is currently one of the hottest sectors in venture investing, with companies raising billions to apply Large Language Model tools to robotics. Unitree, China's leading robot maker, saw its value surge to $66 billion following its IPO on China's equivalent of the NASDAQ, only to see its value drop nearly half in recent weeks.

Analysts attribute this decline to the fact that while robot physical capabilities are improving, they still lack the knowledge necessary to perform valuable work. At Actuate conference, developers building AI brains for robots expressed excitement, with the event having tripled in size since its inception in 2023 and attracting 1,500 attendees.

The issue at hand is the lack of high-quality training data for AI models, known as the "robotics data crisis." Attempts to build generalized robots that can perform any task are still in the distant future, and end-to-end learning for specific tasks hasn't yielded reliable, commercially successful products. Developers aim to mimic the advancements of leading AI labs by finding or creating more diverse datasets, experimenting with different training methods, and devising better reinforcement learning scenarios.

Harry Mellsop, a founder of Antioch, suggests that physical AI is currently in its "GPT 2 era," similar to how OpenAI's GPT-2 predated ChatGPT. To overcome this hurdle, more data and computational resources, particularly GPUs optimized for ray tracing, are needed to create high-fidelity simulations. Autonomous vehicles are the most advanced in this field due to the ability to collect relevant data from cars driven by humans and the task of avoiding contact rather than manipulating the physical environment.

Major car companies, including Tesla, Wayve, and Uber, are increasingly investing in ML tooling to compete with dedicated humanoid makers. Alex Kendall, CEO of Wayve, believes that starting with vehicles is the way to go, as manipulation robotics is akin to self-driving five years ago. Kendall argues that a truly general model should be more agnostic, with commonalities across different embodiments but some differences in the world model for each simulator.

Genesis AI, a vertically-integrated humanoid robotics company, is taking a different approach. CEO Théophile Gervet is not convinced that a "brain strategy" will work yet, but acknowledges the importance of co-designing hardware and AI. Gervet also notes that specific vertical focus is crucial for physical AI businesses, as general-purpose robots are not generating value due to low success rates.

Bedrock, for example, is deploying robots in industrial settings, such as autonomous excavators, while robotics companies targeting specific tasks are seeing progress in the field.

However, the temptation to invest in specific verticals is strong, as they provide not only revenue but also real-world deployment data. Bedrock CTO Kevin Peterson explains that their focus on excavation will help understand the challenges of manipulation in the wild, with plans to develop an intelligence layer that spans multiple construction machines.

Managing the data generated by these robots is a significant challenge due to the high density of visual and LiDAR data. Foxglove recently announced a new product built on top of Nvidia's Cosmos open weight world model, allowing engineers to search the data using natural language queries to accelerate triage and debugging.

For Kendall, a ChatGPT moment for physical AI would be eyes-off autonomy for less than $1,000 in a car, which would require significant investment in hardware. His company is already licensing models to car manufacturers to achieve this goal, which he believes is a multi-billion dollar opportunity. This development will pave the way for truly general physical AI models.