Creating better digital tools for students to learn to play music by ear

Learning to play music by ear is challenging for most musicians, but research from a team at the University of Waterloo may help musicians-in-training find the right notes.

The Waterloo team analyzed a range of YouTube videos that focused on learning music by ear and identified four simple ways music learning technology can better aid prospective musicians—helping people improve recall while listening, limiting playback to small chunks, identifying musical subsequences to memorize, and replaying notes indefinitely.

The research, “Helping popular musicians learn by ear: Analyzing video lessons to inform the design of memory-oriented human-recording interactions,” is published in the Proceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction.

“There are a lot of apps and electronic tools out there to help learn by ear from recorded music,” said Christopher Liscio, a recent Waterloo master’s graduate in computer science and the study’s lead author.

“But we see evidence that musicians don’t appear to use them very much, which makes us question whether these tools are truly well-suited to the task. By studying how people teach and learn how to play music by ear in YouTube videos, we can try to understand what might actually help these ear-learning musicians.”

The team studied 28 YouTube ear-learning lessons, breaking each down to examine how the instructors structured their teaching and how students would likely retain what they heard. Surprisingly, they found that very few creators or viewers were using existing digital learning tools to loop playback or manipulate playback speed despite their availability for over two decades.

“We started this research planning to build a specific tool for ear learners, but then we realized we might be reinforcing a negative pattern of building tools without knowing what users actually want,” said Dan Brown, professor of Computer Science at Waterloo. “Then we got excited when we realized YouTube could be a helpful resource for that research process.”