Visions of the Future

To the uninitiated, the scene is terrifying. A patient lies spread-eagle on an operating table, draped except for his midsection, where a trauma doctor digs for bullet fragments in a hollowed-out cavity large enough to swallow a bowling ball.

But to researchers in the University of Maryland’s Institute for Advanced Computer Studies (UMIACS) and doctors at the R Adams Cowley Shock Trauma Center at the University of Maryland Medical Center in Baltimore, it’s step one on the path to the future of emergency surgery.

The scene is playing out in 3-D on a giant screen in UMIACS’ Augmentarium, where researchers seek to graft the power of computing onto everyday life.

Postdoctoral computer science researcher Sujal Bista ’05, M.S. ’10, Ph.D. ’14 filmed the surgery at Shock Trauma with an innovative (and still-secret) video setup using cameras that present multiple viewpoints. Viewers can change perspective and see around objects in the scene, like doctors’ hands or medical equipment.

The filming experience was disturbing, Bista says—particularly the acrid smell of blood vessel cauterization. 

“If you’re not used to it, it’s very strong,” he says.

Giving viewers the same sense of immersion (minus the smell) is the first objective in this collaborative research, says Amitabh Varshney, computer science professor, UMIACS director and lead investigator for the project. The surgeries Bista is filming will be used primarily for training.

“We’re working on building virtual environments in which multiple students and teachers are immersed concurrently and can view the reconstructions in any environment with a shared perspective,” Varshney says.

Students and teachers will share virtual experiences using 3-D headsets like the Oculus Rift, which UMIACS has stocked up on, thanks to last year’s $31 million gift from Oculus VR co-founder and CEO Brendan Iribe. A $1.5 million gift from his mother, Elizabeth Iribe, will establish an endowed chair in virtual and augmented reality, fostering continued research of this type.

The environment in a trauma hospital like Shock Trauma, which established the “golden hour” ideal of saving lives through speedy care is disorienting for medical students, says Dr. Sarah Murthi, who is working with the UMD researchers.

But giving them an Oculus Rift headset and letting them acclimate by exploring the scene in virtual reality could revolutionize training, she says.

“We throw medical students and young trainees into the chaos of trauma with almost no preparation. We expect them to just pick up how to manage several patients at once, without really training them how to do that,” Murthi says. “Virtual reality will allow us a way to teach what I call spinning plates—paying attention to several things at once and not losing track of any of them.”

The next step, which will take place in coming years, is introducing virtual and augmented reality into clinical practice, Varshney says.

“If there was a virtual reality simulation where you could practice a particularly difficult surgery before you do it, the actual surgery would be faster and safer,” he says.

And during surgery, doctors could be guided by information and imagery projected on see-through goggles, perhaps listing the steps or showing systems in the body.

Further out, the researchers envision a world where remote-controlled robots on distant battlefields or in the backs of ambulances could act as the virtual hands of skilled surgeons at places like Shock Trauma, speeding the delivery of quality care dramatically.

“Augmented reality has the potential to fundamentally change and improve the practice of medicine—both in how we train and how we care for patients,” Murthi says.