NVIDIA trains humanoid robots with Apple Vision Pro
In this neural network training method, the machine copies the operator's movements in real time, and it is Apple Vision Pro that digitizes the movements.
The developers transfer the acquired data to the NVIDIA Isaac Sim application, where they simulate movements in a virtual environment. Then, in the MimicGen NIM microservice, they generate synthetic motion data based on recorded teleoperations with Apple Vision Pro. The resulting synthetic sets are used to extend the AI database of the humanoid robot.
Training an AI humanoid robot requires a colossal amount of data and impressive finances. The Apple headset itself is necessary for NVIDIA because it solves two big problems at once: first, the helmet can generate a lot of data using teleoperation, and second, it does it relatively cheaply.