How Your Laptop Could Look Beneath Your Skin

As doctors and patients try to sidestep the COVID-19 risks of in-person appointments by using Zoom and other services, University of Maryland researchers are uncovering the hidden capabilities of computer and phone cameras to help expand what doctors can diagnose in a simple video chat.

Supported by National Science Foundation Rapid Response Research grant, electrical and computer engineering Professor Min Wu and Dr. Don Milton, a professor in the Maryland Institute for Applied Environmental Health, are working to improve telemedicine, which has grown explosively during the pandemic. According to a recent report from the U.S. Department of Health and Human Services, nearly half of Medicare primary care visits switched to telehealth appointments within weeks of the beginning of the pandemic.

Wu’s research often focuses on extracting subtle signals from video. In one previous project, for instance, she showed that tiny aberrations in terrorists’ videos caused by power grids could be used to determine where in the world they were filmed. This time around, she’s developing system to identify biological signals that even basic cameras can see, but which the eye normally wouldn’t pick up. So in addition to seeing the patient normally on a webcam, the systems would also supply important health metrics to clinicians.

“For example, as the heart pumps blood to every part of our body, the subtle color changes under the skin can be captured by a video camera in terms of small changes that we call a micro signal,” she said. “The main rate of fluctuation in this small signal is at the same pace as the heart rate, and more detailed analysis on the signal can help obtain information about breathing and blood oxygen levels.”

Though such information is generally useful for doctors, it could be lifesaving during the novel coronavirus pandemic, said Milton, an expert on airborne transmission of respiratory diseases who is ramping up a major study of how COVID-19 infection occurs. He plans to test the technology Wu is developing with the cohort enrolled in his study based in the School of Public Health.

A standoff technology for clinicians to patient’s physiological signals—particularly oxygen saturation—could both aid in a COVID-19 diagnosis and caring for people who are infected and isolating themselves at home, he said.

“That could be important, because one of the really big problems with this disease in particular has been that people don’t know when they’re getting in trouble until they’re really bad off,” he said.

The technology could also be used for low-cost screening that avoids the crowds, virus transmission risks, hassles and expense of the various COVID-19 test in use now. And, Wu said, the development of technologies allowing contactless vital sign monitoring could be useful in the early detection, prevention and management of future pandemics.