Linguistic content emerges in the speaker’s brain before word articulation and rapidly re-emerges in the listener’s brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models.

Summary: Researchers have discovered that brain-to-brain coupling during conversation is significantly influenced by the context in which words are used, not just by linguistic information alone. Using high-resolution electrocorticography and GPT-2 language models, they found that specific brain activity in both speakers and listeners is driven by the context-specific meaning of words.
This study highlights the importance of context in aligning brain activity during interactions and offers new insights into how our brains process language. The findings pave the way for further exploration of brain coordination and communication.

— Key facts:

• Contextual importance: Brain-to-brain coupling during conversation is largely driven by the context-specific meaning of words, enhancing communication.
• Brain activity timing: Word-specific brain activity occurs about 250 ms before speech in speakers and 250 ms after hearing in listeners, showing rapid processing.
• Advanced modelling: The study used GPT-2 language models to accurately predict shared brain activity patterns, outperforming traditional linguistic models.

When two people interact, their brain activity becomes synchronised, but it was unclear until now to what extent this “brain-to-brain coupling” is due to linguistic information or other factors, such as body language or tone of voice.

Researchers report on August 2 in the journal Neuron that brain-to-brain coupling during conversation can be modelled by considering the words used during that conversation, and the context in which they are used.

“We can see linguistic content emerge word-by-word in the speaker’s brain before they actually articulate what they’re trying to say, and the same linguistic content rapidly re/emerges in the listener’s brain after they hear it,” says first author and neuroscientist Zaid Zada of Princeton University.

To communicate verbally, we must agree on the definitions of different words, but these definitions can change depending on the context. For example, without context, it would be impossible to know whether the word “cold” refers to temperature, a personality trait, or a respiratory infection.

“The contextual meaning of words as they occur in a particular sentence, or in a particular conversation, is really important for the way that we understand each other,” says neuroscientist and co-senior author Samuel Nastase of Princeton University. “We wanted to test the importance of context in aligning brain activity between speaker and listener to try to quantify what is shared between brains during conversation.”

To examine the role of context in driving brain coupling, the team collected brain activity data and conversation transcripts from pairs of epilepsy patients during natural conversations.
The patients were undergoing intracranial monitoring using electrocorticography for unrelated clinical purposes at the New York University School of Medicine Comprehensive Epilepsy Center. Compared to less invasive methods like fMRI, electrocorticography records extremely high-resolution brain activity because electrodes are placed in direct contact with the surface of the brain.
Next, the researchers used the large language model GPT-2 to extract the context surrounding each of the words used in the conversations, and then used this information to train a model to predict how brain activity changes as information flows from speaker to listener during conversation.

Using the model, the researchers were able to observe brain activity associated with the context-specific meaning of words in the brains of both speaker and listener. They showed that word-specific brain activity peaked in the speaker’s brain around 250 ms before they spoke each word, and corresponding spikes in brain activity associated with the same words appeared in the listener’s brain approximately 250 ms after they heard them.

Compared to previous work on speaker–listener brain coupling, the team’s context-based approach model was better able to predict shared patterns in brain activity.
“This shows just how important context is, because it best explains the brain data”, says Zada.
“Large language models take all these different elements of linguistics like syntax and semantics and represent them in a single high-dimensional vector. We show that this type of unified model is able to outperform other hand-engineered models from linguistics.”

In the future, the researchers plan to expand on their study by applying the model to other types of brain activity data, for example fMRI data, to investigate how parts of the brain not accessible with electrocorticography operate during conversations

“There’s a lot of exciting future work to be done looking at how different brain areas coordinate with each other at different timescales and with different kinds of content,” says Nastase.

 

Author of this article, published 3 August 2024 on Neuroscience News,
is Kristopher Benke – Source: Cell Press