UMD Engineers Use AI to Give Super Mario a New Job: Driving Instructor

Within a month of the June release of Mario Kart World alongside the Nintendo Switch 2, the video game had sold nearly 6 million units, the most successful launch in Mario Kart franchise history. In the real world, Super Mario's serial crashes and decades of aggressive driving would get his license suspended faster than an approaching Bullet Bill. 

That hasn't stopped University of Maryland engineering researchers from using the mustachioed cartoon plumber who races a cart around a track as a model for responsible driving—one that could someday help keep autonomous cars safe.

Funded by the U.S. Naval Air Warfare Center Aircraft Division, the researchers are using artificial intelligence (AI) to train a computer to “play” Mario Kart on its own—in this case, the inaugural 1992 Super Nintendo version—circling the track as fast as possible while hewing to actual laws of physics and, most importantly, avoiding risk. Eventually, the UMD team hopes to provide regulators with a roadmap to certifying AI technologies in autonomous fleets that are increasingly popping up in cities across the U.S. and around the world.

The research is led by Associate Professor of aerospace engineering Mumu Xu, who studies federal safety certification for planes and cars. A few years ago, she noticed a disconnect between long-established safety norms and the wildly diverse algorithms applied to autonomous systems, making it almost impossible for engineers to actually understand how decisions are being made. Traditional vehicles have well-established processes for federal inspection, she said, “but with AI, all the old mathematics we use to show that a system is safe no longer hold because no one knows what’s under the hood.”

So Xu and her colleagues sought out a meeting point between humans and machines that everyone could understand—and it happened to wear a bushy mustache. In a study published in May in IEEE Xplore, they demonstrated how Mario Kart can be used to teach an autonomous simulator to avoid collisions through an AI technique called deep reinforcement learning. The paper's coauthors include computer science Ph.D. student Kristy Sakano, aerospace engineering doctoral student Joe Mockler and undergraduate electrical and computer engineering student Alexis Chen.

The 33-year-old Super Mario Kart, with its pixelated landscapes and A-B-A-B button pushing combos, pales in comparison to the lavish 3D graphics and gyroscopic controllers of today’s version, but its codes are simpler and available to the public. When Xu’s team reconfigured them so a computer could take the wheel, it added a reward system that offered points for whizzing past checkpoints and subtracted them for spinouts and slowdowns. The computer then raced Mario around the track millions of times. Initially, it “drove stupidly,” said Xu, hopping up and down and bouncing into walls. Eventually it learned to zip through the course without a dent. 

Using Mario Kart and other racing video games to train self-driving systems isn’t novel, but most software engineers design them for maximum speed while deprioritizing safety, Xu said. That’s despite at least 2.700 crashes and several deaths involving driver-assisted technologies over the last five years, according to the National Highway Traffic Safety Administration. (Most collisions are likely due to sensor issues, Xu said.)

The UMD project attempts to tackle “black box” technology, a term that often denotes airplane recorders but in an autonomous context means what it sounds like: a system so complex that its internal decision-making processes can't be understood even by its designers. The Mario Kart simulator can help jump-start safety conversations in the self-driving vehicle industry because of its relatability, said Bucket Costello, associate research professor and director of test and evaluation of autonomous systems for UMD's MATRIX Lab who specializes in the certification of autonomous systems. 

“Certification officials don’t really understand how autonomy works and what it does, so they’re risk-averse,” he said. “But the work that Mumu is doing can help them gain comfort in how these systems operate.” 

Xu, who collaborates regularly with the U.S. Department of Defense, launched the study partly to enrich Pentagon autonomous tech. She chose Mario Kart as a platform after being introduced to the game by her husband, who played it throughout law school. “I always ended up getting lost, but it was funny and nostalgic,” she said.

 Having demonstrated that their simulator can improve safety, the researchers will next aim to ensure that Mario’s maneuvers are mathematically feasible—can he really carve a turn at 40 mph without running off the road, for example? Those answers will help fine-tune their algorithms in a second Navy-funded study.

The research applies not just to self-driving vehicles but to any consumer product relying on algorithmic software, Xu said. 

“For anything that uses AI, whether it’s a medical device or a gadget like Amazon Echo, we hope that their algorithms are safe,” she said. “This is our attempt to figure that out.”