Open source AI has demonstrated the rapid innovation possible when developers collaborate on models, data, and tools. Robotics, along with robotics and physics AI, also stand to benefit significantly. However, development efforts can be hampered by costly and disjointed resources, even with access to large datasets and advanced models.
In collaboration with Hugging Face, we are rolling out the NVIDIA Isaac GR00T 1.7, an advanced Open Inferential Vision Language Action (VLA) model for humanoid robots, along with the NVIDIA Isaac Teleop Framework to enhance LeRobot — Hugging Face’s open-source robotics library. Planned integrations with NVIDIA Cosmos 3, a leading model in physical AI, will provide developers with standardized pathways for end-to-end robot development, encouraging innovation and collaboration within the open robotics community.
“Open source is crucial for transforming advanced research into practical applications that people can study, refine, and expand,” stated Thomas Wolff, co-founder and chief scientific officer of Hugging Face. “With today’s NVIDIA Isaac GR00T 1.7 and Isaac TeleOp, developers can train and evaluate robots publicly using shared models, data, and workflows. The upcoming NVIDIA Cosmos 3 will further integrate frontier world models into the collaborative ecosystem.”
Open Robot Basic Models Pipeline
Hugging Face’s LeRobot is an open-source robotics library designed for training, running, and sharing robot datasets, models, policies, and workflows. Our partnership with Hugging Face connects a network of 3 million NVIDIA robot developers and 16 million AI builders, broadening access to cutting-edge physical AI tools.
By integrating NVIDIA’s physical AI capabilities with LeRobot, developers can utilize a standardized approach for data collection, model training, performance assessment, and deployment through open workflows. Key integrations include:
- NVIDIA Isaac Teleop is an open-source framework for robotic data collection, enabling developers to capture high-quality human demonstrations from external devices in a standardized format, expand datasets, and share them within the community, all directly via LeRobot.
- NVIDIA Isaac GR00T 1.7 is the first open, commercially viable robot-based model that streamlines post-training deployment through the LeRobot workflow, aiding developers in adapting GR00T for new robot tasks and embodiments with proven performance.
- NVIDIA Cosmos 3, arriving soon at LeRobot, will empower developers to generate and enhance robotics data, simulate various scenarios, and facilitate policy development in situations where real-world data is sparse or prohibitively expensive to gather.
These features complement the extensive NVIDIA resources already linked to LeRobot, supporting a holistic robot development workflow, which includes:
- the comprehensive Open Source Physics AI Dataset, downloaded over 15 million times with more than 350,000 real and simulated trajectories and 57 million grasps, offering developers a head start on their robotics projects.
- NVIDIA Isaac Sim and Isaac Lab simulation frameworks to help developers set up environments, generate robot data, test policies, and validate behaviors before transitioning to physical robots.
- NVIDIA Isaac Lab Arena within the LeRobot Environment Hub allows developers to quickly prototype complex simulations, register them with LeRobot EnvHub, and seamlessly utilize them in the LeRobot ecosystem to train and evaluate models like GR00T, Pi, and SmolVLA.
- NVIDIA Jetson Thor and LeRobot’s Reachy 2 Integration facilitates deployment of VLA models on open-source humanoid robots.
Learn more about using Isaac Teleop, Isaac GR00T 1.7, and Isaac Lab-Arena with LeRobot for comprehensive humanoid robot development. Explore the in-depth LeRobot integration workflow.
Source: blogs.nvidia.com