An enhancement to an existing AI-powered brain decoder can translate human thoughts into text without hours of training. Scientists have made new improvements to a “brain decoder” that uses artificial intelligence (AI) to convert thoughts into text. The AI model can scan a person’s brain using non-invasive equipment and convert thoughts into written sentences—without the need for implants.
In one study, scientists developed an AI model that decoded brain signals to reproduce sentences typed by volunteers. In a second study, the same researchers used AI to map how the brain actually produces language, turning thoughts into typed sentences.
A team of researchers reported in a new study that their new converter algorithm can quickly train an existing decoder on another person’s brain. According to the scientists, these results could one day help people with aphasia, a brain disorder that affects communication. In the new study, published on February 6, 2025, in the journal Current Biology, Huth and his co-author, Jerry Tang, a graduate student at the University of Texas at Austin, explored how this communication limitation could be overcome.
“People with aphasia often have difficulty both understanding and producing speech,” study co-author Alexander Huth, a computational neuroscientist at the University of Texas at Austin (UT Austin), told Live Science. “If this is true, we may not be able to build models of their brains at all by observing their brain responses to stories they hear.”
The brain decoder uses machine learning to translate a person’s thoughts into text based on their brain’s responses to stories they’ve heard. However, previous versions of the decoder required participants to listen to stories in an MRI machine for many hours, and these decoders only worked with the people they were trained on.
A team of researchers has developed an algorithm that allows an AI-powered “brain decoder,” trained on one person, to translate the thoughts of another with minimal training. Jerry Tang/University of Texas at Austin
The researchers first trained the brain decoder on several control participants using a long-term approach—collecting functional MRI data while the participants listened to 10 hours of radio broadcasts.
They then trained two conversion algorithms on control participants and on another set of “target” participants: one using data collected when participants listened to radio stories for 70 minutes, and the other when they watched silent Pixar short films unrelated to the radio stories for 70 minutes.
Using a method called “functional alignment,” the team mapped the brain responses of control and target participants to the same audio or video stories. This information was used to train the decoder to work with the target participants’ brains, without the need to collect hours of training data.
The team then tested the decoders using a short story that none of the participants had previously heard. Although the decoder’s predictions were slightly more accurate for participants who used the original control information than for those who used the converters, the words predicted by the decoder based on each participant’s brain scans were still semantically related to the words used in the test story.
For example, in one of the test narrative fragments, someone described a job they disliked: “I’m a waitress at an ice cream parlor. So, uh, it’s not… I don’t know where I want to be, but I know it’s not it.” A decoder using a converter algorithm trained on video data predicted: “I worked at a job I found boring. I had to take orders, and I didn’t like them, so I worked on them every day.” The match isn’t exact—the decoder doesn’t capture the exact sounds people heard—but the ideas are related.
Scientists can collect data while someone watches a silent video and then use it to create a language decoder for their brain.
Researchers claim that using video converters to transfer existing decoders to people with aphasia can help them express themselves. The study also found some overlap between the ways people represent ideas from language and from visual images in the brain.
This study suggests that there is a semantic representation that is independent of the modality it originates from. In other words, this study helps us understand how the brain represents certain concepts in the same way, even when they are presented in different formats.
See also... New method for observing individual atoms confirms century-old quantum mechanical theory |
Scientists at Meta used artificial intelligence (AI) and non-invasive brain scanning to understand how thoughts are translated into printed sentences.
According to the scientists, the results of the study could in the future serve as the basis for the creation of a non-invasive brain-computer interface that will help people with brain damage or injuries communicate.
Brain-computer interfaces using similar decoding methods have been implanted in the brains of people who have lost the ability to communicate, but new research may confirm a potential path to wearable devices.
In the study, the researchers used magnetoencephalography (MEG), which measures the magnetic field generated by electrical impulses in the brain, to track neural activity as participants typed sentences. They then trained an AI language model to decode brain signals and produce sentences based on MEG data.
Two new studies shed light on how we can transform thoughts into written sentences using a digital interface. Meta
The model decoded the letters typed by the participants with 68% accuracy. Frequently encountered letters were decoded correctly more often, while less common ones, such as Z and K, had a higher error rate. When the model made errors, it tended to substitute symbols physically close to the target letter on the QWERTY keyboard, suggesting that the model uses motor signals from the brain to predict which letter the participant had typed.
The brain was found to first generate information about the context and meaning of a sentence, and then create increasingly detailed representations of each word, syllable, and letter as the participant types.
“These results support long-standing predictions that language production requires hierarchical decomposition of sentence meaning into increasingly smaller units that ultimately control motor actions,” the study authors write.
To prevent one word or letter from interfering with another, the team discovered, the brain uses a “dynamic neural code” to separate them. This code constantly shifts the representation of each piece of information in the regions of the brain responsible for language formation.
This allows the brain to associate successive letters, syllables, and words, storing information about each for a longer period of time. However, MEG experiments have been unable to pinpoint the precise regions of the brain where each of these language representations originates.
The researchers write that while the current setup is too bulky and sensitive to operate properly outside a controlled laboratory environment, advances in MEG technology could open the door to future wearable devices.
See also... Complete list of HTML/Javascript event handlers with examples List of required documents for a website in the Russian Federation |
