Conventional robots, like those used in industry and hazardous environments, are easy to model and control, but are too rigid to operate in confined spaces and uneven terrain. Soft, bio-inspired robots are far better at adapting to their environments and maneuvering in otherwise inaccessible places.
These more flexible capabilities, however, would normally require an array of on-board sensors and spatial models uniquely tailored for each individual robot design.
Taking a new and less resource-demanding approach, a team of researchers at MIT has developed a far less complex, deep-learning control system that teaches soft, bio-inspired robots to move and follow commands from just a single image.
Their results are published in the journal Nature.
By training a deep neural network on two to three hours of multi-view of video of various robots executing random commands, the researchers trained the network to reconstruct both the shape and range of mobility of a robot from just a single image.
Previous machine-learning control designs required expert customization and expensive motion-capture systems. This lack of a general-purpose control system limited their applications and made rapid prototyping far less practical.
“Our method unshackles the hardware design of robots from our ability to model them manually, which in the past has dictated precision manufacturing, costly materials, extensive sensing capabilities and reliance on conventional, rigid building blocks,” the researchers note in their paper.
The new single-camera machine-learning approach enabled high-precision control in tests on a variety of robotic systems, including a 3D-printed pneumatic hand, a soft auxetic wrist, a 16-DOF Allegro hand, and a low-cost Poppy robot arm.
These tests succeeded in achieving less than three degrees of error in joint motion and less than 4 millimeters (about 0.15 inches) of error in fingertip control. The system was also able to compensate for the robot’s motion and changes to the surrounding environment.
“This work points to a shift from programming robots to teaching robots,” notes Ph.D. student Sizhe Lester Li in an MIT web feature.
“Today, many robotics tasks require extensive engineering and coding. In the future, we envision showing a robot what to do and letting it learn how to achieve the goal autonomously.”
Since this system relies on vision alone, it may not be suitable for more nimble tasks requiring contact sensing and tactile manipulation. Its performance may also degrade in cases where visual cues are insufficient.
The researchers suggest that the addition of tactile and other sensors could enable the robots to perform more complex tasks. There is also the potential to automate the control of a wider range of robots, including those with minimal or no embedded sensors.
Written for you by our author Charles Blue,
edited by Sadie Harley, and fact-checked and reviewed by Robert Egan—this article is the result of careful human work. We rely on readers like you to keep independent science journalism alive.
If this reporting matters to you,
please consider a donation (especially monthly).
You’ll get an ad-free account as a thank-you.
More information:
Sizhe Lester Li et al, Controlling diverse robots by inferring Jacobian fields with deep networks, Nature (2025). DOI: 10.1038/s41586-025-09170-0
© 2025 Science X Network
Citation:
Deep-learning system teaches soft, bio-inspired robots to move using only a single camera (2025, July 9)
retrieved 9 July 2025
from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
Leave a comment