AI: What AI Robot Hands have to teach us about AI Robotics. RTZ #797

It looks all so easy in The Terminator and later movies. An Arnold Schwarzenegger as a robot ‘with a heart’’, and amazing capabilities at his fingers.

Like most others in this AI Tech Wave, I’m unabashedly excited about the prospects for companies small and large to be inspired by SciFi. And the possibilities to build transformative things to do with AI.

I’m just more pragmatic in my expectations on how long will it take to deliver said transformative things.

A few months ago, I explained how the current the technology realities of AI Robots, especially Humanoid ones, is roughly where self-driving cars were back in 2004. That was when DARPA kicked off the current AI EV, self driving cars and taxi thing, now two decades ago.

And we’re barely at Level 2 of Six levels of truly self driving cars. So that’s at least a few years into the future, from a technology, regulatory, and societal set of realities. Google Waymos are ‘kinda, sorta’ at Level 3, along with others like Mercedes.

Tesla is still between those two despite the latest aspirational promise of its ‘FSD’ (Full Self Driving software packages at $12,000 upgrades).

The reason is that we need to solve and execute on some very REAL tech challenges to make humanoid robots real at scale. By that I mean working to do truly useful things by robots in the millions.

The challenges range from creating the AI Data, synthetic and otherwise, to truly power the AI robots. Like all human content online to date that is converted into Data powering our LLM AIs. That’ll take years. Measured in half and full decade chunks.

The other challenge is making robots, especially humanoids, do things that we as humans take for granted. Despite Rosie the Robot from the Jetsons. ‘Simple’ things that even babies can do with two hands and five fingers each.

Turns out that is one of the most challenging things to craft into humanoid robots at scale. As Tesla and other ‘Physical AI’ startups in robotics are finding out. Both here and in China, the other leader in AI Robotics.

The Information has a couple of very good, ‘must read in full’ articles on the subject that provide great backgroud here. Let me focus on the second one titled “Why Hands Are a Handful for Tesla’s Humanoid Robots”:

“On Friday, we reported on Tesla’s Optimus humanoid robot program, which may fall well short of Elon Musk’s ambitious goal to pump out 5,000 robots by the end of the year.”

“It turns out the robots’ hands are among the big factors holding up production. In fact, some Optimus robots that are mostly complete have been sitting unused at a Tesla facility because they don’t have hands or lower forearms, we reported. Musk deserves some sympathy. Five-fingered hands are notoriously difficult for roboticists to get right, and many robotics companies steer clear of them, settling for simple pinchers instead.”

The details are mind-numblingly complex. Just like when scientists started to use the human brain as possible tech templates to develop the machine learning and now Generative LLM AI technologies that currently BARELY emulate what human babies can do by the time they are four. At a cost of hundreds of billions a year.

Robots hands with five fingers are the first of the first steps towards doing similar things with AI Robots:

“To see why, consider the number of joints in a human hand. A hand and wrist can perform about 27 different movements (in robotics-speak, that’s 27 “degrees of freedom”). To put that in perspective, a robotic arm is considered pretty agile even if it has as few as seven degrees of freedom.”

“All those degrees of freedom add up to trouble for the roboticists designing the hands. More joints require separate motors to move them, and the motors need to be powerful enough to enable the robot to do useful work. For example, the humanoid robots of 1-year-old startup Foundation can carry up to 45 pounds using their hands and arms. Still, like many other robot hands, Foundation’s are not strong enough to pinch and unscrew a bottle cap, according to someone familiar with the company.”

It’s not just the robot fingers…it’s all the stuff up and down in the humanoid arm and more, to get those fingers to do things in a coordinated way:

“What’s more, packing many wires together in the robot’s forearm can introduce thermal and electrical issues. Foundation, which is selling humanoid robots for manufacturing and logistics, is aiming to reduce the number of wires connected to its motors in the next generation of its robots.”

“Human hands are also densely packed with sensors to pick up on textures, pressure, vibrations and temperature. Endowing robot hands with similar tactile sensors to help them with dexterous tasks such as peeling an orange introduces additional engineering challenges.”

“On top of all that, robot hands need to be cheap enough to maintain and durable enough to withstand industrial labor.”

Of course, more pragmatic developers in AI robotics are eschewing five finger hands altogether:

“Because of these challenges, some robot makers are foregoing hands altogether in favor of relatively cheap pinchers. That’s the case for Physical Intelligence, a 1-year-old startup developing AI models to power a variety of robots.”

“To train its models, Pi has put together two-armed robots with relatively cheap pinchers. The company has shown that these simple pinchers are dexterous enough, when powered by the right AI, to pull off tasks that roboticists have considered challenging, including folding clothes and cardboard boxes.”

Some more examples of AI robot startups:

“Startup Ultra Robotics has put similar crab-clawed robots to work packaging e-commerce orders. Meanwhile, Dexterity’s truck-unloading robots wield a range of suction cups to pick up boxes, rather than hands. And Boston Dynamics has experimented with three-fingered hands, which give its humanoid Atlas robots seven degrees of freedom (each finger performs two to three movements.)”

The broader general argument to emulate human hands, and for humanoids in the first place, is that our physical world has ‘been built and optimized for human forms’:

“Still, the proponents of making human-like hands argue that homes and jobs have been designed around human hands, and several startups have popped up to create them (though someday they aim to build the rest of the robot bodies as well). In fact, one was founded by an ex-Tesla roboticist Jay Li, whom Tesla has sued over alleged trade theft, which he has denied. These startup founders argue that if robots ever hope to do everything that humans can, human-like hands will help.”

The other big argument for humanoid robots, and hands, is the holy grail of painfully and expensively acquiring vast amounts of training data at scale, for the AI robot’s AI models:

“Plus, proponents say that human-like hands open up the possibility to train robots by copying videos of people using their hands. For example, one startup is raising funds to develop AI glasses that can record the hand movements of factory workers. The idea is that these videos can then be used to train AI models to pilot robot hands.”

Companies ranging from Nvidia to scores of others large and small are down this rabbit hole in a deep way. I discussed Nvidia’s work here in this podcast, focusing on their Nvidia AI scientist Jim Fan. Useful background, with a ‘must watch’ link to his presentation on the subject that also lays out the long road ahead to get this all working at scale.

Tech history has taught us painfully that bottom up excitement on new tech and the resulting immediate innovations, ALWAYS take longer to implement into new tech products and services at Scale.

And Scale today is VERY different than it was in the mid-1990s, when the Internet started to be commercialized for a global audience of over a hundred million. Compared to delivering the same today to over five billion globally with smartphones and ingrained human habits. Not to mention ingrained web and internet infrastructure ecosystem, optimized for humans.

That is a critical difference vs the Internet tech wave, now THREE DECADES later.

I know, because I was there in those rooms, with front row seats.

And am now eagerly leaning into this AI thing, with those 30 years as a dress rehearsal for what’s ahead in this AI Tech Wave.

We’ll just have to wait a bit longer for all the good stuff this AI Tech Wave. And then we can applaud with our hands, real and robotic. Stay tuned.

(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here)