Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
A robot policy needs to understand what a task is asking for in a scene, and how to move the robot to accomplish that task. To achieve these two skills, we introduce Zero-real Agent Policy (ZAP), a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness. We use a pretrained 27B parameter model to bootstrap world understanding and reasoning,…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
