If your noise-canceling headphones are canceling sounds you’d actually prefer to hear, researchers at the University of Washington may have a solution worth listening to.
The UW team has developed deep-learning algorithms that let users decide which sounds to filter out from an environment and which sounds to let in. Examples include sitting in a park enjoying the sound of birds chirping, while blocking the sound of people talking. Or blocking the buzz of a vacuum being run at home while allowing a knock at the door to be heard.
The system, called “semantic hearing,” works with headphones that stream captured audio to a connected smartphone, which cancels all environmental sounds. Via voice commands or an app, a user selects sounds from 20 classes to be let in. Birds, sirens, crying babies, honking car horns — only those selected will be played through the headphones.
“Understanding what a bird sounds like and extracting it from all other sounds in an environment requires real-time intelligence that today’s noise canceling headphones haven’t achieved,” said senior author Shyam Gollakota, a UW professor in the Paul G. Allen School of Computer Science & Engineering. “The challenge is that the sounds headphone wearers hear need to sync with their visual senses. You can’t be hearing someone’s voice two seconds after they talk to you. This means the neural algorithms must process sounds in under a hundredth of a second.”
That need for speed is why the semantic hearing system processes sounds on a connected smartphone rather than through cloud servers. Because sounds from different directions arrive in people’s ears at different times, the system must also preserve delays and other spatial cues so people can still meaningfully perceive sounds in their environment, according to the UW.
Apple released a new generation of its AirPods Pro wireless buds in September that rely on improved software to intelligently decide what users need to hear when they are in noise-canceling mode.
The UW system was tested in environments such as offices, streets and parks and was able to extract sirens, bird chirps, alarms and other target sounds, while removing all other real-world noise.
The system did struggle to distinguish between sounds that share many properties, such as vocal music and human speech, and researchers said that training the models on more real-world data might improve those outcomes.
The team presented its findings Nov. 1 at UIST ’23 in San Francisco. In the future, the researchers plan to release a commercial version of the system.
