Could Your Vacuum Be Listening to You?

Those cute little autonomous floor cleaners could be sweeping up more than just dust in your house, according to a team of researchers including a University of Maryland computer scientist that showed how popular robotic vacuums can be remotely hacked to collect audio, even though the devices were never designed to do so.

The researchers, including Assistant Professor Nirupam Roy, collected information from the laser-based navigation system in a popular vacuum robot and applied signal processing and deep learning techniques to recover speech and identify television programs playing in the same room as the device.

The project demonstrates the potential for any device that uses light detection and ranging (Lidar) technology to be manipulated for collecting sound. This work, which Roy co-led with Assistant Professor Jun Han at the National University of Singapore was presented today at the Association for Computing Machinery’s Conference on Embedded Networked Sensor Systems.

“We welcome these devices into our homes, and we don’t think anything about it,” said Roy, who holds a joint appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS). “But we have shown that even though these devices don’t have microphones, we can repurpose the systems they use for navigation to spy on conversations and potentially reveal private information.”

Privacy experts have suggested that the house maps the vacuum bots make using Lidar, allowing them  to avoid collisions risk potential privacy breaches. The maps, often stored in the cloud,could give advertisers access to information about such things as home size, which suggests income level, and other lifestyle-related information. Roy and his team wondered if the Lidar in these robots could also pose potential security risks as sound recording devices in users’ homes or businesses.

Sound waves cause objects to vibrate, and these vibrations cause slight variations in the light bouncing off an object. Laser microphones, used in espionage since the 1940s, are capable of converting those variations back into sound waves. But laser microphones rely on a targeted laser beam reflecting off very smooth surfaces, such as glass windows.

A vacuum Lidar, on the other hand, scans the environment with a laser and senses the light scattered back by objects that are irregular in shape and density. The scattered signal received by the vacuum’s sensor provides only a fraction of the information needed to recover sound waves. The researchers were initially unsure if a vacuum bot’s Lidar system could be manipulated to function as a microphone, and if the resulting signal could be meaningfully interpreted..

First, they hacked a robot vacuum to show they could control the position of the laser beam and send the data to their laptops through Wi-Fi without interfering with the device’s navigation.

Next, they conducted experiments with two sound sources—a human voice reciting numbers played over computer speakers, and audio from a variety of television shows played through a TV sound bar. Roy and his colleagues then captured the laser signal sensed by the vacuum’s navigation system as it bounced off a variety of objects—including a trash can, a cardboard box, a takeout container and a plastic bag—placed near the sound source.

The researchers passed the resulting signals through deep learning algorithms that were trained to either match human voices or to identify musical sequences from television shows. Their computer system, which they call LidarPhone, identified and matched spoken numbers with 90% accuracy. It also identified the shows from a minute’s worth of recording with more than 90% accuracy.

“This type of threat may be more important now than ever, when you consider that we are all ordering food over the phone and having meetings over the computer, and we are often speaking our credit card or bank information,” Roy said.

“But what is even more concerning for me is that it can reveal much more personal information” including lifestyle, work hours and political affiliation, he said. “That is crucial for someone who might want to manipulate the political elections or target very specific messages to me.”

Robot vacuum cleaners are just one example of potential vulnerability to Lidar-based spying. Many other devices could be open to similar attacks such as smartphone infrared sensors used for face recognition or passive infrared sensors used for motion detection, the researchers said.

“I believe this is significant work that will make the manufacturers aware of these possibilities and trigger the security and privacy community to come up with solutions to prevent these kinds of attacks,” Roy said.