Pink Floyd song reconstructed from listeners’ brain activity: How was this done, significance | Explained News
3 min readWhat happens inside the human brain? For years, scientists have tried to figure out exactly what the brain perceives and how. A new study has made inroads in this research by successfully managing to “decode” the brain’s electrical activity to reconstitute it as recognisable music.
The researchers played Pink Floyd’s iconic “Another Brick in the Wall Part 1” to 29 subjects while analysing their brain activity using arrays of electrodes placed directly on the brain’s surface. Then, using advanced AI models, they were able to reconstruct the song simply from the electrical activity recorded.
The study results were published on August 15 in PLOS Biology.
Eavesdropping on the brain
This is not the first time scientists have managed to reconstruct what the brain perceives from recorded neural signals. In the past, there have been successful attempts at recreating words and images using signals recorded by implanted electrodes.
However, this is the first time scientists have managed to reconstruct a recognisable, even if somewhat garbled, song purely from neural recordings.
Why does this matter? The results of this study might help create better brain-computer interfaces and assistive devices that translate brainwaves into speech for those unable to speak due to paralysis caused by stroke.
Consider the late Stephen Hawking, one of the world’s most famous scientists who lost his ability to speak due to a motor-neurone disease which left him paralysed. After he was unable to speak, he communicated via a speech generating device which improved over the years. But even at its very best, the speech generated was robotic, without any inflection.
As the study’s lead author Ludovic Bellier told Scientific American, “No matter the language, speech contains melodic nuances, including tempo, stress, accents and intonation … [which] carry meaning that we can’t communicate with words alone.”
Electrodes implanted onto the brain’s surface
While promising, there are major hurdles to be passed before the kind of technology used can come to the hands of consumers – perhaps none greater than placement of electrodes on the brain itself.
Currently available technology to record the requisite brain waves is simply not sensitive enough to work on the scalp itself. This makes surgical implants a must, something that would dissuade the average consumer.
The 29 test subjects were all suffering from epilepsy and already had implants inserted onto their brains to determine the cause of the seizures. The researchers were able to get consent from these patients to carry out this study.
Moreover, even after placing electrodes directly on to the brain, the music was still somewhat garbled due to the number of electrodes that were packed in. The researchers hope that by increasing the density of electrodes on the brain (number of electrodes/surface area), they will be able to obtain even better resolution.
Focus on auditory regions for reproducing speech
Despite these limitations on its current real-world applicability, scientists have gathered some vital information during this exercise.
Crucially, researchers were able to identify which parts of the brain respond to which kind of stimuli. They found that certain portions of superior temporal gyrus – located just behind and above the ear and associated with auditory processing – respond to the onset of speech, while other areas respond to other elements of the music.
In the past, researchers have focussed on the brain’s motor cortex, associated with movements of the mouth and vocal cords to recreate the acoustics of speech. The new study can guide scientists to a different location.
The study also found that music perception relied on both hemispheres, with a preference for the right hemisphere.