At its recent GTC 2024 event, Nvidia announced a new foundational model to build intelligent humanoid robots. Dubbed GR00T, short for Generalist Robot 00 Technology, the model will understand natural language and be able to observe human actions and emulate human movements.

According to Nvidia CEO Jensen Huang, creating intelligent humanoid robots is the most exciting AI problem today. GR00T robots will learn coordination and other skills by observing humans to be able to navigate, adapt and interact with the real world. At the conference keynote, Huang showed several demos of what GR00T is capable of at the moment, including some robots performing a number of tasks.

The GR00T model takes multimodal instructions and past interactions as input and produces the actions for the robot to execute.

To power GR00T, Nvidia has created a new family of systems-on-modules, called Jetson Thor, using the latest Blackwell graphics architecture from the company and able to provide 800 teraflops (TFLOPS) of eight-bit floating-point compute.

At the foundation of GR00T lies Nvidia Isaac Sim, an extensible, Omniverse-based platform for robotics simulation aimed to improve the way AI-based robots are designed and tested, according to the company.

To train GR00T at scale, Nvidia has also built a new compute orchestration platform, Nvidia Osmo, aimed at coordinating training and inference across several Nvidia systems, including DGX systems for training, OVX systems for simulation, and IGX and AGX systems for hardware-in-the-loop validation.

Embodied AI models require massive amounts of real and synthetic data. The new Isaac Lab is a GPU-accelerated, lightweight, performance-optimized application built on Isaac Sim specifically for running thousands of parallel simulations for robot learning.

While GR00T is still very much a work in progress, Nvidia has announced two of the building blocks that will compose it, as part of the Isaac platform: a foundational model for robotic-arm manipulators, called Isaac Manipulator, and a collection of hardware-accelerated packages for visual AI and perception, the Isaac Perceptor.

According to Nvidia, Isaac Manipulator

provides up to an 80x speedup in path planning and zero-shot perception increases efficiency and throughput, enabling developers to automate a greater number of new robotic tasks.

On the other hand, Isaac Perceptor aims to improve efficiency and safety in environments where autonomous mobile robots are used, such as in manufacturing and fulfillment operations.

Both the Manipulator and the Perceptor should become available in the next quarter, says Huang.

On a related note, Nvidia has joined the Open Source Robotics Alliance, which aims to provide financial and industry support to the Robot Operating System (ROS). The company has not detailed if they plan to use ROS for GR00T robots, though.

Link:

Nvidia Announces Robotics-Oriented AI Foundational Model - InfoQ.com