I just read an article from Rodney Brooks, “Why Today’s Humanoids Won’t Learn Dexterity,” and found it very interesting. I feel strongly about sharing it with you for informational purposes.
Here is a summary of the key points from the article.
Main Argument
The central claim is that today’s humanoid robots will not learn human-like dexterity, despite massive investment and hype. The belief that they will soon be “plug compatible” with humans and able to perform general-purpose manual labor is “pure fantasy.”
The Flawed Premise of Current Humanoids
-
The Goal: The entire business case for humanoids rests on the idea that one general-purpose robot can perform millions of different manual tasks currently done by humans.
-
The Requirement: This goal fundamentally requires solving human-level dexterity (the ability to manipulate objects).
-
The Problem: Dexterity has been an unsolved “hard problem” in robotics for over 65 years. Decades of research into articulated, human-like hands have produced no designs that are robust or dexterous enough for real-world applications. The industry still relies on simple, 60-year-old technology like parallel jaw grippers and suction cups.
Why the Current “Solution” Will Fail
The current strategy adopted by most humanoid companies is to use end-to-end learning, similar to what powers Large Language Models (LLMs). The plan is to:
-
Collect vast amounts of data, mostly by watching videos of humans performing tasks or by tele-operating a robot.
-
Feed this data into a large neural network, believing dexterity will “emerge.”
This approach is doomed to fail because it is collecting the wrong data.
The “Right Data” vs. the “Wrong Data”
The article argues that the success of AI models in other areas (like language and vision) came from training on data with the correct structure.
-
Vision (ConvNets): Succeeded by processing images in a way that mimics the human visual cortex, exploiting known properties of vision.
-
Language (LLMs): Succeeded by training on linear sequences of text tokens, the fundamental structure of language.
Human dexterity, however, is not primarily a visual problem. It is a multi-modal sensory problem dominated by touch and force.
-
Wrong Data (What robots are collecting): Vision (videos) and motion (joint positions). This only captures what a task looks like.
-
Right Data (What robots are missing): The article emphasizes that human dexterity is impossible without the constant, rich stream of non-visual data from our bodies, including: Tactile Sensing: A vast array of mechanoreceptors in our skin that detect texture, pressure, vibration, shape, and critical “slip” events (feeling an object start to drop). Proprioception: The “sixth sense” from muscle spindles and tendon organs that tells us our limb position, the force we are exerting, and the tension in our muscles. Forethought: Humans use vision and touch to predict an object’s properties (like weight, material, and density) before grasping, allowing us to apply the correct forces from the start.
Conclusion: By focusing only on vision and motion, today’s humanoid robot companies are ignoring the essential sensory data streams (especially touch and force) required for manipulation. They are training on the wrong data, and therefore, their models will never learn true dexterity.
Takeaways
After reading the article, I realized that Optimus is much harder than normal people thought. Even though that, I still have belief that Elon Musk will lead us to make that happen. The time horizon may be longer than I thought.
Original article: https://rodneybrooks.com/why-todays-humanoids-wont-learn-dexterity/