A New Starring Role for Video Captions: Language Learning

Want to learn a new language? Try watching videos in that language with the captions turned on—and slow down the speed.

A University of Maryland research team found that video captions enhanced listening comprehension for English learners, whether the videos were played at a reduced speed or even at a higher-than-normal one, and whether the student was a novice or mostly proficient. The results were published last month in the journal System.

“This offers implications for anyone learning a language,” said co-author Nihat Polat, chair of the Department of Teaching and Learning, Policy and Leadership and professor of applied linguistics. Thanks to the tremendous wealth of online language resources and to new recognition of the benefits that language learning brings, that’s a lot of people.

Content in different languages is now readily available on platforms like YouTube, Netflix and Amazon, and their captions are high-quality. A July survey by the research and analytics firm Yougov found that 30% of Americans watch TV with subtitles most or all of the time, and that 10% of those who use captions do so to learn another language.

In the past, some teachers viewed captions as another source of information that could overwhelm students who were already struggling to process language or impact their ability to develop auditory skills if they were primarily focused on the text on screen. However, this research shows that captions have specific and important benefits for comprehension, regardless of skill level, and that they can scaffold understanding rather than detract from it.

"Sometimes in a language class, teachers shy away from using [captions] because they want to focus on listening,” said lead author Laura Mahalingappa, associate professor of applied linguistics and language education. “Teachers should show a video without captions the first time so students can concentrate on listening and then watch it with captions the second time so students can fill in any information that they missed.”

Previous studies of captions in language learning did not examine how they interacted with variables, including playback speeds, the level of detail in questions or users’ language proficiency.

For this study, Mahalingappa, Polat and Jiaxuan Zong Ph.D. ’23 asked 287 university students in Turkey and China with at least low-intermediate proficiency (basic comprehension) in English to watch a TED talk in English and answer questions about its content.

Captions, when used with slower playback speed (0.75x), significantly improved the students’ ability, regardless of their English proficiency level, to answer easy comprehension questions. When the TED talk was run at normal and faster speed levels (1x and 1.25x), captions also helped students— especially those with higher English levels—answer both easy and detail-oriented questions.

Video speed can be a tool to achieve specific lesson goals when used with captions, said Polat. For example, teachers who want their students only to pick up the gist of a lesson might play a video faster to cover more material, while those who want them to comprehend details or focus on grammar might slow the video down.

Despite widespread belief that more advanced language learners do not need captions, the study found that for learners with higher proficiency, captions were valuable aids to comprehension. And while the study used English learners in a university setting, the authors noted that the results apply to learners of other languages, and both in and out of classrooms.