New NVIDIA Cosmos 3 Open Models for Interactive World Simulation

NVIDIA Cosmos 3, a world model development platform, has introduced four new open-weight checkpoints to its Cosmos 3 Nano family. These checkpoints, each containing 16B trainable parameters, are trained using OpenMDW1.1, allowing both commercial and non-commercial use. The models are designed to generate interactive world simulations and multi-sensor multi-view driving scenes.

The four checkpoints are Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Bimanual, and Cosmos3-Nano-Sim-Transfer. Each serves a distinct purpose. Cosmos3-Nano-Transfer-Auto is a bidirectional model for generating offline synthetic driving data, while Cosmos3-Nano-Sim-Auto and Cosmos3-Nano-Sim-Bimanual are variants for streaming and bimanual robot simulations respectively.

These models offer various ways to create synchronized camera and LiDAR generation, allowing researchers to vary road layouts and signal states. They also provide per-camera World Scenario Map controls and text prompts to guide appearance. For bimanual robot simulations, the models can generate visual consequences from actions or turn a physics simulator's controls into generated observations.

Developers can adapt these models to their own sensor rigs and driving domains by post-training them on their data. Once adapted, the models can be used for offline data generation or undergo causal training and few-step distillation for fast streaming simulation. This open foundation aims to help researchers and developers create richer, more responsive simulated worlds.