I Saw the Future of Robotics in a Virtual Duck That Chases My Cat

A simulated duck, named Microduck, was taught by OpenAI's GPT-6 Sol to navigate a room and detect bengal cats. Developed by Pollen Robotics and Hugging Face, the virtual duck waddled towards a target feline, as seen in the accompanying video clip. While awaiting the physical version of Microduck, I trained the simulated robot in just a few hours, a process that previously would have taken weeks.

The advanced capabilities of OpenAI's latest models, which excel at understanding and manipulating 3D representations of the physical world, facilitated this rapid training. University of California, Berkeley professor Ken Goldberg and his students used Astra, OpenAI's September 2022 release, to render 3D images. They discovered that this model could create detailed simulations with precise lighting, camera positioning, and materials, achieving this feat within an hour.

Using GPT-6 Sol, an agentic harness, and Nvidia's DGX Spark, I successfully trained Microduck to walk without falling over. Initially, I attempted to teach the robot to moonwalk using a GIF of Michael Jackson, but it only resulted in a backward jog. Researcher Jim Fan from Nvidia explained that AI's ability to control robots mirrors software agents accessing different functions on a computer.

The AI model Aspire, open-sourced by Fan, Goldberg, and others, enables robots to discover and employ new skills. At CoRL, Fan demonstrated how Astra controlled a robot arm to paint a scene, solving a Rubik's cube along the way. While the technology is still in its infancy and requires significant engineering expertise, it promises to revolutionize manufacturing and e-commerce.

As hobbyists and engineers grasp the simplicity of programming and training robots, the era of "vibe robotics" is likely to dawn.