For nearly a decade, a team of researchers at MIT's Computer Science and artificial intelligence Laboratory (CSAIL) has been trying to figure out why certain images persist in people's minds, while many others fade. To do this, they proposed to map the spatio-temporal brain dynamics involved in the recognition of a visual image. And now, for the first time, scientists have leveraged the combined strengths of magnetoencephalography (MEG), which captures the timing of brain activity, and functional magnetic resonance imaging (fMRI), which identifies active regions of the brain, to accurately determine precision when and where the brain processes. a memorable image.
Your open access study, published this month in More biology, used 78 pairs of matching images for the same concept but differing in their memorability scores: one was highly memorable and the other was easy to forget. These images were shown to 15 subjects, featuring skateboarding scenes, animals in various environments, everyday objects such as cups and chairs, natural landscapes such as forests and beaches, urban scenes of streets and buildings, and faces showing different expressions. What they discovered was that a more distributed network of brain regions than previously thought actively participates in the encoding and retention processes that support memorability.
“People tend to remember some images better than others, even when they are conceptually similar, such as different scenes of a person riding a skateboard,” says Benjamin Lahner, a CSAIL-affiliated MIT doctoral student in electrical and computer engineering and first author. from the article. study. “We have identified a brain signature of visual memorability that emerges around 300 milliseconds after viewing an image, involving areas along the ventral occipital cortex and temporal cortex, which processes information such as color perception and object recognition. “This signature indicates that highly memorable images elicit stronger and more sustained brain responses, especially in regions such as the early visual cortex, which we previously underestimated in memory processing.”
While highly memorable images maintain a higher, more sustained response for about half a second, the response to less memorable images decreases rapidly. This idea, Lahner explained, could redefine our understanding of how memories are formed and persist. The team anticipates that this research has potential for future clinical applications, particularly in the early diagnosis and treatment of memory-related disorders.
The MEG/fMRI fusion method, developed in the laboratory of CSAIL senior research scientist Aude Oliva, skillfully captures the spatial and temporal dynamics of the brain, overcoming the traditional limitations of spatial or temporal specificity. The fusion method had a little help from her friend machine learning, to better examine and compare brain activity when looking at multiple images. They created a “representational matrix,” which is like a detailed graph that shows how similar neural responses are in various regions of the brain. This graph helped them identify patterns of where and when the brain processes what we see.
Choosing pairs of conceptually similar images with high and low memorability scores was the crucial ingredient in unlocking these insights into memorability. Lahner explained the process of aggregating behavioral data to assign memorability scores to images, where they selected a diverse set of high and low memorability images with balanced representation across different visual categories.
Despite the progress made, the team observes some limitations. While this work can identify brain regions that show significant effects on memorability, it cannot elucidate the function of the regions in terms of how they contribute to better memory encoding/retrieval.
“Understanding the neural underpinnings of memorability opens interesting avenues for clinical advances, particularly in the early diagnosis and treatment of memory-related disorders,” says Oliva. “The specific brain signatures we have identified for memorability could lead to early biomarkers for Alzheimer's disease and other dementias. “This research paves the way for new intervention strategies that are precisely tailored to the individual’s neural profile, potentially transforming the therapeutic landscape for memory disorders and significantly improving patient outcomes.”
“These findings are interesting because they give us insight into what happens in the brain between seeing something and storing it in memory,” says Wilma Bainbridge, an assistant professor of psychology at the University of Chicago, who was not involved in the study. “Researchers here are detecting a cortical signal that reflects what is important to remember and what can be forgotten early on.”
Lahner and Oliva, who is also director of strategic industry engagement at the MIT Schwarzman College of Computing, director of the MIT-IBM MIT Watson ai Lab, and principal investigator at CSAIL, join Western University assistant professor Yalda Mohsenzadeh, and York University researcher Caitlin Mullin. on the paper. The team acknowledges a shared instrument grant from the National Institutes of Health, and their work was funded by the Vannevar Bush Faculty Fellowship through an Office of Naval Research grant, a National Science Foundation award, a Multidisciplinary University Research Initiative award through a grant from the Army Research Office. and the EECS MathWorks scholarship. His article is published in More biology.
