Not Playing Games

The room might be the strangest you’ve ever seen—soaring and palatial, with walls covered in a bizarre selection of portraits. Directly ahead is a photo of Taylor Swift, with one of Shrek to the right. Turn around to face the hawkish visage of inventor Nikola Tesla.

This isn’t a weird dream, but a virtual reality (VR) environment constructed by computer science Ph.D. student Eric Krokos and viewed through an advanced virtual reality headset. He’s testing the hypothesis that people can better recall visual information—like whose virtual portrait hangs where in the room—in realistic-looking environments, rather than when viewed in two dimensions on a flat screen.

“People are good at dealing with information spatially, but putting it on a screen makes it more abstract,” Krokos says. “Can we leverage a virtual reality headset to improve someone’s memory?”

An answer in the affirmative will have major implications for the fields of education and training. And it will illustrate how UMD researchers are helping virtual reality—now driven by gaming and entertainment—grow up a little.

After years of anticipation, VR hit the mainstream in recent weeks with the release of Facebook’s Oculus Rift headset. You need only to try the device to understand gamers’ excitement. Most of us will never drive a Formula One car or face a mob of zombies while low on ammo, but the full audio-visual immersion afforded by the Rift and competing headsets that Sony and HTC are releasing this year almost make it feel like we are.

Virtual reality game sales should reach $496 million in 2016, according to the global information firm IHS, and soar from there.

But Maryland researchers working in a unique computer visualization lab called the Augmentarium are convinced the technology has more profound uses. Virtual reality and its cousin, augmented reality—which overlays computer imagery on the real world using a heads-up display like Google Glass—could revolutionize schooling, medicine, public safety and more.

“The companies developing the technology are focused where the money is now,” says Amitabh Varshney, director of UMD’s Institute for Advanced Computer Studies (UMIACS). “We’re focused five or 10 years into the future, asking how we can use it to have a significant societal impact in critical areas.”

For instance, would cities be safer if police officers watching a disorienting array of security monitors could slip on a headset for a 3-D virtual view of the city stitched together from myriad camera feeds? UMIACS is testing just such a system outside the A.V. Williams Building using existing security cameras.

“The point is to present visual information in a more understandable way,” Varshney says. “Instead of a person appearing and disappearing on different screens, which is confusing, the system follows him.”

Medicine is another area that may be ripe for a virtual revolution. UMIACS researchers are working with doctors at the R Adams Cowley Shock Trauma Center at the University of Maryland Medical Center in Baltimore to make training films more visceral and instructive.

UMIACS researcher Sujal Bista ’05, M.S. ’10, Ph.D. ’14 last year filmed an abdominal surgery using a camera array that allows viewers to switch perspectives to see around intrusions like a doctor’s hand and view the surgery more closely. Future filming will use more capable camera setups, focusing on rare surgeries normally unseen by students. It’s the closest thing possible to participating in delicate procedures, Varshney says.

And using ultrasound, researchers are aiming to give doctors the power to look at patients and virtually see inside them, with real-time images projected on heads-up display glasses.

“The effect of much of the current imaging technology has been to drive us away from patients, even though we would prefer to be with the patient and focusing on them,” says Dr. Sarah Murthi of Shock Trauma. “Hopefully, what something like this will do is bring us back to the patient and away from the computer.”

Gaming is a great showcase for the technology, but Murthi hopes people understand how much more practical its uses can be.

“Being able to develop this technology for medicine depends on it being broadly accepted in society for uses beyond entertainment,” she says. “If we’re soon using it for navigation and many other practical things, the momentum from that could absolutely lead to a revolution in medicine.”