NVIDIA has expanded its Cosmos world model development platform with four new open-weight checkpoints designed to generate interactive world simulations and multi-sensor driving scenes. The release, part of the Cosmos 3 model family, aims to provide researchers and developers with foundational tools for building more responsive simulated environments for autonomous systems.
What Happened
NVIDIA introduced four post-trained examples of the Cosmos 3 Nano model, each containing 16 billion trainable parameters. The new checkpoints are Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, and Cosmos3-Nano-Sim-Bimanual. These models are released under the OpenMDW1.1 license, which permits both commercial and non-commercial use according to its terms.
Cosmos3-Nano-Transfer-Auto is a bidirectional model for offline synthetic driving data generation. It produces synchronized video across up to 11 camera views, including fisheye lenses, and supports joint RGB and LiDAR generation. Users can control scene structure and traffic motion via Per-camera World Scenario Maps (WSM) and guide appearance with text prompts.
Cosmos3-Nano-Sim-Auto serves as a causally distilled streaming variant of Transfer-Auto. It generates successive chunks of synchronized camera video and LiDAR data in a few denoising steps, enabling fast streaming and long, interactive rollouts. This model uses generated history and updated driving controls to maintain continuity.
Cosmos3-Nano-Sim-Transfer converts simulator controls and text prompts into photorealistic video. It takes spatial controls such as depth, segmentation, edges, and blur from a physics simulator and renders color observations that a policy can consume, while keeping physics and ground-truth state within the simulator.
Cosmos3-Nano-Sim-Bimanual predicts video based on robot or camera actions. It supports interactive two-arm simulation for YAM and AgiBot embodiments, allowing users to start with an image and generate subsequent observations by moving the robot or camera. The model handles mixed-action control sequences, such as switching between viewpoint changes and manipulation.
Why It Matters
The release addresses critical needs in robotics and autonomous driving simulation. For robotics, the models offer complementary methods to build environments that respond to changing scenes and viewpoints. Sim-Bimanual predicts visual consequences from actions, while Sim-Transfer generates observations from physics simulator controls.
In autonomous driving, the requirement for appearance and geometry to agree is met by the synchronized camera and LiDAR generation capabilities. These tools allow researchers to vary road layouts and signal states to study how driving policies respond. The streaming generation feature connects video prediction to ongoing interaction, helping world-model researchers explore how actions and observations unfold together.
The models utilize a Mixture-of-Transformers backbone that connects separate text-processing and visual-generation networks through attention. The training process involves four stages: bidirectional adaptation, teacher forcing, consistency distillation, and self-forcing. The final self-forcing stage is designed to mitigate error compounding by training the model on its own generated history.
NVIDIA positions these open weights and post-training recipes as a shared starting point for adapting models, comparing methods, and contributing improvements. The company notes that challenges remain in measuring where these models faithfully support policy learning and evaluation, understanding their limits, and improving performance.
The Bottom Line
NVIDIA’s addition of four open-weight Cosmos 3 Nano checkpoints provides the SI community with specialized tools for interactive world simulation in autonomous driving and robotics. By offering synchronized multi-sensor data generation and streaming capabilities under a permissive license, NVIDIA aims to accelerate development in simulated environments for next-generation SI systems.