NVIDIA Cosmos Platform Accelerates World Model and Physical AI Development

NVIDIA Cosmos helps developers build world models for physical AI. The platform enables systems to understand and simulate the real world through open models, training tools, simulation checkpoints and Cosmos Data Factory blueprints.

Ranked among the top five on six open model leaderboards, the NVIDIA Cosmos platform gives model builders access to open models, data, training and optimization tools, and recipes for scaling training data, adapting models, testing behavior and optimizing performance before deployment.

Build Better Training Data for Physical AI

Building a world model starts with multimodal, multisensory data. World models and physical AI systems require diverse training examples that reflect the complexity of real-world environments.

NVIDIA Cosmos Data Factory brings together tools and skills for labeling, curating, augmenting and evaluating data at scale. Its GPU-accelerated Cosmos Curator solution supports multimodal, multisensory inputs and processing blocks for video training datasets.

Cosmos Data Factory currently offers five cloud-native blueprints through GitHub, Microsoft Azure, Nebius and CoreWeave:

• Generate self-driving car datasets: Expand training coverage by curating and labeling driving data, then add synthetic examples for difficult-to-capture scenarios.
• Extend image attributes: Change the appearance of objects to improve image search.
• Generate event video: Convert still images into composite videos depicting rare or dangerous events.
• Generate defect images: Create defect images of printed circuit boards, metal and glass for inspection models.
• Expand video data: Change the weather and lighting in existing footage while preserving the structure of the scene.

Cosmos Data Factory accelerates agent skills that curate and process data and generate synthetic data.

Enterprises are already using Cosmos Data Factory blueprints to accelerate physical AI development workflows.

Porsche is leading the development of a shared autonomous-vehicle data flywheel with Volkswagen Group brands, Voxel51 and Microsoft Azure. The effort is expected to reduce physical test-drive iterations by up to 90% and development cycle time by up to 85%.

Porsche uses Cosmos Data Factory to search and enrich self-driving car footage with Voxel51.

Anybotics generated more than 1,500 synthetic video clips using Cosmos Data Factory blueprints and validated the output with curated real-world data. The company uses the platform to train industrial inspection robots to recognize liquid spills and other changes during facility visits. Learn more about the project.

Anybotics generates abnormal clips using NVIDIA Cosmos Data Factory.

Ultralytics, a computer vision company, retrained its YOLO model with real images and targeted synthetic defects generated using Cosmos Data Factory to improve defect localization and recall.

Train, Adapt and Optimize World Models

Once training data is prepared, developers can train their own architecture or start with a pretrained model.

At GTC Berlin, NVIDIA expanded the Cosmos Framework. Together with Cosmos 3, it provides training, optimization and deployment tools and recipes as agent skills for more open models, giving model builders a faster path from experimentation to deployment.

Developers and agents can adapt GR00T, Alpamayo, Qwen, Pi0.5 and other compatible open models. They can build models through fine-tuning and reinforcement learning, optimize them through distillation and quantization, and deploy them using high-performance inference.

Micro AGI, a robotics research institute developing a world model specialized for egocentric operation, post-trained Cosmos 3 on videos of humans manipulating objects. This reduced the 32-billion-parameter inference capability to an 8-billion-parameter Nano model without sacrificing accuracy.

Developers can label data and post-train Cosmos 3, then deploy models for specific environments and tasks. Roboflow enables video inference in factories and warehouses using hosted or on-premises NVIDIA GPUs.

Accelerate Policy Testing and Model Execution

Alongside its development tools, NVIDIA continues to advance the underlying Cosmos models.

The new Cosmos 3 Nano model variations support simulations that let developers test autonomous policies in reproducible, hard-to-capture, multi-view and multisensor scenarios. The included code enables developers to adapt the model to train and evaluate policies for autonomous vehicles and two-handed robots in interactive simulations before real-world testing.

Developers can deploy core Cosmos models with the updated NVIDIA Cosmos 3 NIM. The microservices deliver up to two times faster inference without sacrificing quality in validated configurations and enable subsecond alerts with continuous video analytics.

For vision AI developers, the NVIDIA Metropolis Blueprint for Video Search and Summarization (VSS) 3.3 is built on Cosmos NIM microservices. The blueprint transforms a single prompt into an efficient video AI application and enables teams to monitor 40% more video streams on the same GPU.

Fogsphere’s MistIQ Agent Platform uses Cosmos and the NVIDIA Metropolis VSS Blueprint to detect safety hazards in real time across 65 ADNOC sites. For rare events, Fogsphere also uses Cosmos Data Factory to generate synthetic training data, improving detection without physically staging dangerous situations.

A Modular Platform for World Model Development

The Cosmos platform is open and modular. Developers can use Cosmos Data Factory to generate training datasets, the Cosmos Framework to train and adapt world models, and Cosmos models through NIM microservices for supported deployments.