When sound waves reach the inner ear, neurons pick up the vibrations and alert the brain. Encoded in their signals is a wealth of information that allows us to follow conversations, recognize familiar voices, appreciate music, and quickly locate a ringing phone or a crying baby.
Neurons send signals by emitting spikes: brief changes in voltage that propagate along nerve fibers, also known as action potentials. Remarkably, auditory neurons can fire hundreds of spikes per second and time their spikes with exquisite precision to match the oscillations of incoming sound waves.
Using powerful new models of human hearing, scientists at MIT's McGovern Institute for Brain Research have determined that this precise timing is vital to some of the most important ways we make sense of auditory information, including voice recognition. and sound localization.
The open access findings, reported on December 4 in the magazine Nature Communicationsshow how machine learning can help neuroscientists understand how the brain uses auditory information in the real world. MIT professor and McGovern researcher Josh McDermottwho led the research, explains that his team's models better equip researchers to study the consequences of different types of hearing impairment and devise more effective interventions.
sound science
Auditory signals from the nervous system are timed so precisely that researchers have long suspected that timing is important to our perception of sound. Sound waves oscillate at speeds that determine their pitch: low-pitched sounds travel in slow waves, while high-pitched sound waves oscillate more frequently. The auditory nerve that transmits information from the hair cells that detect sound in the ear to the brain generates electrical spikes that correspond to the frequency of these oscillations. “Action potentials in an auditory nerve fire at very particular times relative to spikes in the stimulus waveform,” explains McDermott, who is also associate head of the Department of Brain and Cognitive Sciences at MIT.
This relationship, known as phase locking, requires neurons to time their spikes with sub-millisecond precision. But scientists don't really know how informative these temporal patterns are to the brain. Beyond being scientifically intriguing, McDermott says, the question has important clinical implications: “If you want to design a prosthesis that provides electrical signals to the brain to reproduce the function of hearing, you could say that it is quite important to know what kind of information is in it. the brain.” Normal hearing really matters,” he says.
This has been difficult to study experimentally; Animal models cannot offer much information about how the human brain extracts the structure of language or music, and the auditory nerve is inaccessible to study in humans. So McDermott and graduate student Mark Saddler PhD '24 turned to artificial neural networks.
Artificial hearing
Neuroscientists have long used computational models to explore how the brain might decode sensory information, but until recent advances in computing power and machine learning methods, these models were limited to simulating simple tasks. “One of the problems with these older models is that they are often too good,” says Saddler, who currently works at the Technical University of Denmark. For example, a computational model tasked with identifying the highest pitch in a pair of simple tones is likely to perform better than people asked to do the same. “This is not the kind of listening task we do every day,” Saddler notes. “The brain is not optimized to solve this very artificial task.” This mismatch limited the insights that could be extracted from this previous generation of models.
To better understand the brain, Saddler and McDermott wanted to challenge a model of hearing to do things that people use their hearing to do in the real world, like recognize words and voices. That meant developing an artificial neural network to simulate the parts of the brain that receive information from the ear. The network received input from about 32,000 simulated sound-detection sensory neurons and was then optimized for various real-world tasks.
The researchers showed that their model replicated human hearing well, better than any previous model of hearing behavior, McDermott says. In one test, the artificial neural network was asked to recognize words and voices within dozens of types of background noise, from the hum of an airplane cabin to enthusiastic applause. In all conditions, the model behaved very similarly to humans.
However, when the team degraded the timing of the spikes in the simulated ear, their model could no longer match humans' ability to recognize voices or identify the location of sounds. For example, while McDermott's team had previously shown that people use pitch to help them identify people's voices, the model revealed that this ability is lost without precisely timed cues. “You need fairly precise spike timing to account for human behavior and perform the task well,” Saddler says. This suggests that the brain uses precisely timed auditory signals because they help with these practical aspects of hearing.
The team's findings demonstrate how artificial neural networks can help neuroscientists understand how information extracted by hearing influences our perception of the world, both when hearing is intact and when it is impaired. “The ability to link patterns of auditory nerve activation with behavior opens many doors,” says McDermott.
“Now that we have these models that link neural responses in the ear to listening behavior, we can ask, 'If we simulate different types of hearing loss, what effect will that have on our hearing abilities?'” McDermott says. “That will help us better diagnose hearing loss, and we think there are also extensions to help us design better hearing aids or cochlear implants.” For example, he says: “The cochlear implant has several limitations: it can do some things and not others. What's the best way to configure that cochlear implant to allow it to mediate behaviors? In principle, you can use the models to tell you that.”
