NVIDIA Cosmos-H-Dreams: Bringing Real-Time Generative Simulation to Surgical Robotics

NVIDIA has unveiled Cosmos-H-Dreams, an advanced real-time generative simulator for surgical robotics. This technology builds upon the capabilities of its predecessor, Cosmos-H-Surgical-Simulator, to provide a more interactive and efficient environment for surgical robotics. By utilizing a causal, few-step student model and FlashDreams, an accelerated streaming-inference library, Cosmos-H-Dreams enables real-time action-conditioned generative simulation for surgical robotics.

The system is available for integration with the Versius surgeon controller, allowing for real-time operation on the Versius platform. Cosmos-H-Dreams addresses the challenge of preserving surgical dynamics while reducing generation cost by using a teacher-to-student training pipeline and self-forcing distillation techniques. These innovations enable the model to operate with streaming key/value cache and achieve interactive operation at approximately 160 frames per second on a single NVIDIA RTX PRO 6000 GPU.