An Ally for Access
Palm-sized Robot Navigates Touchscreens for Visually Impaired Users
As touchscreens proliferate in daily life, from coffee shops to airports, people who are blind or visually impaired are often unable to access important digital information. UMD researchers have developed a new robot that could help bridge the gap.
Photo by iStock; Toucha11y photo courtesy of Jiasheng Li
Having to tap-tap-tap away to order a sandwich or check bags at a touchscreen kiosk can be annoying. For those who are blind or visually impaired, it can be excruciating.
Soon, instead of friends, family or even strangers bridging this technological chasm, a palm-sized portable robot could help. Toucha11y, created by a team including doctoral student Jiasheng Li and computer science Assistant Professor Huaishu Peng, can stick to and press buttons on a touchscreen, communicating with a user’s phone and allowing them to use accessibility features—like voiceover and zoom—to read the options.
Federal requirements that public kiosks include accessibility features haven’t solved the problem for many users, said Peng. Even ones that are technically compliant lack standard features, blind testers told him. “We thought, if there’s a universal way for [visually impaired] people to use these machines, we could lower the learning curve a bit.”
The current prototype looks like a tiny measuring tape with a camera, computer and suction cups. It takes photos to orient itself on the screen, then uses an extendable “tape” fitted with a touch probe to press buttons. The research team tested it on screens up to 37 inches wide and on six different interfaces, and users were able to complete tasks like ordering a bubble tea, specifying ice and sugar levels, in under 90 seconds.
In future iterations, Li and Peng plan to shrink Toucha11y (which combines “touch” and “accessibility”—the “11” stands for the letters between “a” and “y” in “accessibility”) to make it easier to carry in a purse or pocket, as well as add real-time menu-scanning capabilities.
Toucha11y is part of the Maryland Initiative for Digital Accessibility, launched in July 2023 and funded by a Grand Challenges Impact Award grant, which brings together researchers across UMD, disability rights groups and tech companies to make technology accessible for all.
Topics:
Schools & Departments: