If you’ve ever played soccer with a robot, it’s a familiar feeling. The sun shines on your face while the smell of grass fills the air. You look around you. A four-legged robot runs towards you, haggling with determination.
While the bot doesn’t display a similar skill level to Lionel Messi, it is an impressive dribbling system in the wild. Researchers at MIT’s Improbable Artificial Intelligence Laboratory, part of the Computer Science and Artificial Intelligence Laboratory (CSAIL), have developed a robotic system with legs that can dribble a football under the same conditions as humans. The bot used a combination of onboard computing and sensing to traverse different natural terrain, including sand, gravel, mud and snow, and adapt to its varying impact on ball movement. Like any committed athlete, “DribbleBot” could get up and recover the ball after a fall.
Programming robots to play soccer has been an active area of research for some time. However, the team wanted to automatically learn how to actuate the legs during the dribble, to enable the discovery of hard-to-type skills to respond to various terrains such as snow, gravel, sand, grass, and pavement. Come in, simulation.
A robot, a ball and a field are inside the simulation: a digital twin of the natural world. You can load the bot and other assets and set physical parameters, and then handle the direct simulation of the dynamics from there. Four thousand versions of the robot are simulated in parallel in real time, allowing data to be collected 4,000 times faster than with a single robot. It’s a lot of data.
The robot starts out not knowing how to dribble the ball; he only gets a reward when he does it, or a negative reinforcement when he messes up. So, he’s essentially trying to figure out what sequence of forces he should apply with his legs. “One aspect of this reinforcement learning approach is that we need to design a good reward to make it easier for the robot to learn successful dribbling behavior,” says Gabe Margolis, an MIT doctoral student who co-led the work with Yandong Ji, an assistant research in Improbable. AI Lab. “Once we’ve designed that reward, then it’s practice time for the robot: in real time, it’s a couple of days, and in the simulator, it’s hundreds of days. Over time, it learns to get better and better at handling. of the soccer ball”. to match the desired speed”.
The bot could also navigate unfamiliar terrain and recover from crashes thanks to a recovery driver the team built into its system. This controller allows the bot to get back up after a fall and return to its dribble controller to continue chasing the ball, helping it to handle interruptions and out-of-range wickets.
“If you look around you today, most robots have wheels. But imagine there’s a disaster scenario, a flood or an earthquake, and we want robots to help humans in the search and rescue process. We need the machines to traverse terrain that is not flat, and wheeled robots cannot traverse such landscapes,” says Pulkit Agrawal, MIT professor, CSAIL Principal Investigator, and director of the Improbable AI Lab. “Our goal in developing algorithms for legged robots is to provide autonomy in challenging and complex terrain that is currently beyond the reach of robotic systems.”
The fascination with quadrupedal robots and soccer runs deep: Canadian professor Alan Mackworth first noted the idea in a paper entitled “On Robots That See,” presented at VI-92, 1992. Japanese researchers later organized a workshop on “Grand Challenges in Artificial Intelligence,” which led to discussions about using soccer to advance science and technology. The project launched as Robot J-League a year later, and worldwide fervor quickly ensued. Shortly after “RoboCup” was born.
Compared to walking alone, dribbling a soccer ball places more restrictions on DribbleBot’s movement and the terrain it can traverse. The robot must adapt its locomotion to apply forces to the ball to dribble. The interaction between the ball and the landscape could be different from the interaction between the robot and the landscape, such as thick grass or pavement. For example, a soccer ball will experience a drag force on the grass that is not present on the pavement, and a slope will apply an acceleration force, changing the ball’s typical trajectory. However, the bot’s ability to traverse different terrain is often less affected by these differences in dynamics, as long as it is not slipping, so the soccer test can be sensitive to variations in terrain than locomotion by. alone it is not.
“Previous approaches simplify the dribbling problem, making a hard, flat ground model assumption. Movement is also designed to be more static; the robot isn’t trying to run and manipulate the ball simultaneously,” Ji says. “That’s where the more difficult dynamics come into the control problem. We addressed this by extending recent advances that have enabled better locomotion outdoors into this composite task that combines aspects of locomotion and dexterous manipulation.”
On the hardware side, the robot has a set of sensors that allow it to perceive the environment, allowing it to sense where it is, “understand” its position, and “see” something in its environment. He has a set of actuators that allows him to apply forces and move himself and objects. Between the sensors and the actuators is the computer, or “brain”, in charge of converting the sensor data into actions, which it will apply through the motors. When the robot runs on the snow, it doesn’t see the snow but it can feel it through the sensors on its engine. But soccer is a more complicated feat than walking, so the team took advantage of cameras on the robot’s head and body to gain a new sensory modality of vision, in addition to new motor skill. And then we haggle.
“Our robot can go outside because it has all its sensors, cameras and computing on board. That required some innovations in terms of making the whole controller fit this onboard computing,” says Margolis. “That’s an area where learning helps because we can run a lightweight neural network and train it to process data from noisy sensors observed by the moving robot. This is in stark contrast to most robots today: typically a The robot arm is mounted on a fixed base and sits on a workbench with a giant computer attached to it. Neither the computer nor the sensors are on the robot arm! So everything is heavy, hard to move.”
There is still a long way to go to make these robots as agile as their counterparts in the wild, and some terrain was challenging for DribbleBot. Currently, the controller is not trained in simulated environments that include slopes or stairs. The robot does not perceive the geometry of the terrain; you are only estimating its contact properties with the material, such as friction. If there is a step, for example, the robot will get stuck, unable to lift the ball onto the step, an area the team wants to explore in the future. The researchers are also excited to apply the lessons learned during DribbleBot’s development to other tasks involving combined locomotion and object manipulation, rapidly transporting various objects from one place to another using the legs or arms.
“DribbleBot is an impressive demonstration of the feasibility of such a system in a complex problem space that requires dynamic whole-body control,” says Vikash Kumar, a Facebook AI Research research scientist who was not involved with the work. “The impressive thing about DribbleBot is that all sensorimotor skills are synthesized in real time in a low-cost system that uses embedded computational resources. While he exhibits remarkable agility and coordination, he is merely the “kick-off” for the next era. Game on!”
The research is supported by the DARPA Machine Common Sense Program, the MIT-IBM Watson Artificial Intelligence Laboratory, the National Science Foundation’s Institute for Artificial Intelligence and Fundamental Interactions, the Air Force Research Laboratory and the US Air Force Artificial Intelligence Accelerator. The paper will be presented at the 2023 IEEE International Conference on Robotics and Automation (ICRA).
