AI: Data for AI driven Robots next key area of focus. RTZ #670
AI robotics, was one of the areas Nvidia’s founder/CEO Jensen Huang spent a fair bit of time in his GTC keynote on their AI roadmap a few days ago. Particularly with ‘Blue’, a ‘R2-D2’, developed in partnership with Disney. Nvidia has a range of AI hardware and software technologies Nvidia that they’re ramping up to support generative AI technologies around robotics. Jensen’s keynote segment around AI robotics is also worth a separate watch.
As I’ve outlined before, this is an area of global innovation, including a whole host of large and small players in China. Tesla/xAI, OpenAI, Google and others are also in the fray, especially on the humanoid robots side of the experimentation. The ‘AI Race’ now definitely has robots along with self driving cars and drones in the AI mix in this AI Tech Wave, even though it’s early days.
The Information updates this emerging landscape, starting with Nvidia’s robot focused AI technologies, in “Why Apple, Meta and Google Are Buying Remote-Controlled Robot Arms”:
“During Nvidia’s conference for developers last week, CEO Jensen Huang showed off software that creates computer simulations of robots. Those simulations aim to teach robots how to perform tasks, from washing dishes to picking up household objects.”
“But some robot makers I spoke to say it’s better to train robots to do such tasks by having a person control them remotely, also known as teleoperation, and then directing an AI model to imitate those actions. It’s similar to how language models can copy the way humans write poetry or code, but for physical actions.”
A core area that will have to be focused on for AI Robotics is Data both organic and synthetic, to TEACH robots of all type to learn from AI training and inference. I discussed Nvidia’s recent synthetic data acqusition a few days ago, which starts to address this area. It’s the critical Box 4 in my AI Tech Stack below.
The Information focuses as well on this area with AI companies ramping up on this key input:
“In a sign of growing interest in teleoperation, Scale AI is considering jumping into that market, according to people who have spoken to the company’s staff.”
“Scale has an army of human contractors who create data to train AI and evaluate the performance of AI models in difficult tasks. The company has discussed using that workforce to handle teleoperation for training robots, these people said.”
“Scale AI’s interest is not surprising, given that large firms such as Tesla, OpenAI, Meta, Google and Apple are trying to develop hardware or software for humanoid or home robots. Some of those firms have hired Scale AI for other purposes in the past.”
Scale AI is a natural here given their leading position on Data software and services for a large number of the major LLM AI players.
“Scale AI, valued at $14 billion, currently helps some robotics firms label images or videos so that the AI powering their machines can better recognize a box or other objects, for instance. But some robotics customers have switched to more specialized providers of data labeling.”
“Sensei, another rival, says it wants to be “Scale AI for robotics training data” and aims to distribute cheap tele-op devices to a network of human data collectors who will perform tasks such as folding laundry on behalf of robot developers.”
“In the relatively small world of robotics, teleoperation equipment is hot.”
There are a number of other AI companies emerging in this area, in the US and China:
“Trossen Robotics, a longtime seller of robot parts, in recent years began selling Aloha, a tele-op device with four arms that allows a human operator to use two of the arms to control the other two. The device’s sensors collect information while the arms move, and the robot is trained to repeat the motions.”
“Some roboticists are collecting tele-op data using more rudimentary gear, including game controllers. Dexterity, which develops robots that pick up and stack packages in trucks and other areas, bought X-Box controllers and connected them to the robots. Human staff use the controllers to direct its machines to stack boxes, thereby improving the robots’ skills, said founding engineer Rob Sun.”
“Other robot developers tell me they accomplish the same goal using virtual reality headsets and their accompanying controllers.”
So that gets into the area of Smart AR/VR headsets and related science projects.
“Cheaper hardware and labor may help scale up tele-op data collection, but key challenges will remain for training robots. For one, the grippers on some of these devices look like crab claws, which means they don’t easily train a dexterous five-fingered hand, one roboticist told me.”
This is an area Elon Musk has focused on for his Optimus robots, with many key parts designed and built by his companies.
But it all needs to be put together in a robot, humanoid or industrial.
“And human teleoperators tend to move slowly when they’re piloting unfamiliar hardware, especially when they rely more on their vision than their sense of touch. Training AI on that tele-op data could lead to a slow-moving robot.”
China has its own AI robotics companies here like Unitree and others leading the charge. Unitree founder/CEO Wang Xingxing below is a case in point.
It’s all early days for AI robotics, but it’s a concurrent ramping development to the LLM AIs that capture the lion’s share of the attention and discussion thus far in this AI Tech Wave. It’s all just getting started. Stay tuned.
(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here)
