NVIDIA Releases GR00T 2 Robot Foundation Model Capable of Rapid Skill Learning Through Human Demonstrations

·by Henderson·Engineering
NVIDIA Releases GR00T 2 Robot Foundation Model Capable of Rapid Skill Learning Through Human Demonstrations
Key Points
  • GR00T 2 can transfer human-demonstrated skills to machines that have not performed those tasks.
  • The model's success rate can reach 95% after fine-tuning.
  • Cross-adaptation technology allows robots to learn skills from demonstrations across different devices.
  • GR00T 2 Turbo is a smaller version designed for humanoid robots, executing tasks more quickly.

NVIDIA has announced the launch of its next-generation robot foundation model, GR00T 2, which can transfer skills demonstrated by humans to machines that have never performed those tasks. As a global action model, GR00T 2 can simultaneously predict future videos and robot movements, enabling it to learn the unfolding process of physical interactions and calculate the actions needed to complete tasks. The model is designed to be applicable to various robot forms, including parallel gripper arms, dexterous five-fingered hands, and humanoid robots. NVIDIA states that GR00T 2 can learn from human videos, handheld demonstrations, and demonstrations collected through other robots.

In tests involving 50 tasks that had not been demonstrated on the target robot, GR00T 2 achieved an average success rate of 40% without specific task training. Providing five demonstrations for each task, the success rate increased to 95% after 30 minutes of fine-tuning. The robot data required for each task demonstration was less than three minutes.

Application of Cross-Adaptation Technology

NVIDIA refers to this approach as cross-adaptation, abbreviated as X-Adapt, which allows robots to transfer skills from demonstrations across different forms or devices. In one experiment, researchers demonstrated to the robot how to dispense a laboratory funnel with filter paper and moisten it with a wash bottle. This demonstration was conducted solely by humans, without remote operation of the target robot, resulting in the five-fingered robot successfully completing the task in 65% of trials. In another test, a handheld Universal Manipulation Interface (UMI) gripper demonstrated how to assemble a toy dump truck, after which another robotic arm named YAM completed the assembly in 60% of trials, despite never having received specific task demonstrations.

Researchers also used a wearable device that could mimic human finger movements to teach the robot's hands to perform bimanual tasks, including making banana yogurt, scooping oats, and striking matches to light lamps. The model's training combined 402,730 hours of first-person human videos, 10,184 hours of handheld gripper demonstrations, and 23,500 hours of robot remote operation data. NVIDIA states that its Human-1M dataset contains 100,000 hours of publicly available data.

Features of GR00T 2 Turbo

GR00T 2 can also collaborate with reasoning models to break down complex tasks into steps, monitor progress, and guide robots through application programming interfaces. In tests involving the assembly of three toys, this method completed at least 97 out of 100 trials for each toy. The system can recover from errors and interruptions during the process. Additionally, the visual guidance feature allows the reasoning model to mark the locations where the robot should grasp and release objects, increasing the success rate of short tasks involving moving objects on YAM from 40% to 92%.

NVIDIA has also developed GR00T 2 Turbo, a smaller version designed for humanoid robots aimed at achieving faster execution speeds. The model runs directly on the Unitree G1, utilizing the built-in Jetson Thor processor, with each prediction taking 124 milliseconds. Through a full-body controller, the humanoid robot successfully completed the task of providing popcorn in all 48 trials and cleared various objects from a table in 93% of trials.

NVIDIA introduced GR00T 2 at the Robotics Learning Conference held on November 9 in Austin, Texas. The company stated that the model is designed to help robots move beyond narrow training actions toward more adaptive operations suitable for industrial and everyday environments.

ItemSpecifications
Training Data402,730 hours of human videos, 10,184 hours of handheld gripper demonstrations, 23,500 hours of robot remote operation data
Prediction Time124 milliseconds
Success Rate (after fine-tuning)95%

How GR00T 2 Changes the Way Robots Learn

NVIDIA's GR00T 2 model represents a significant advancement in robot learning, particularly in skill transfer. Through human demonstrations, robots can quickly learn new tasks, enhancing efficiency and expanding the scope of robotic applications. The introduction of cross-adaptation technology allows robots of different forms to share learning experiences, which is crucial for the future development of robot collaboration and autonomy. Additionally, the launch of GR00T 2 Turbo highlights NVIDIA's focus on humanoid robot performance, indicating substantial potential for applications in industrial and everyday life.

H
About the author
Henderson