How the brain deciphers the melody of speech

“It’s not what you say, it’s how you say it.” And new research backs it up.

A first-of-its-kind study from Northwestern University’s School of Communication, the University of Pittsburgh and the University of Wisconsin-Madison reveals a region of the brain, long known for early auditory processing, plays a far greater role in interpreting speech than previously understood.

What’s new

The multidisciplinary study being published Monday, March 3 in the journal “Nature Communications” found a brain region known as Heschl’s gyrus doesn’t just process sounds — it transforms subtle changes in pitch, known as prosody, into meaningful linguistic information that guides how humans understand emphasis, intent and focus in conversation.

For years, scientists believed that all aspects of prosody were primarily processed in the superior temporal gyrus, a brain region known for speech perception. Bharath Chandrasekaran, the study’s co-principal investigator and professor and chair of the Roxelyn and Richard Pepper Department of Communication Sciences and Disorders at Northwestern, said the findings challenge long-held assumptions about how, where and the speed at which prosody is processed in the brain.

“The results redefine our understanding of the architecture of speech perception,” Chandrasekaran said. “We’ve spent a few decades researching the nuances of how speech is abstracted in the brain, but this is the first study to investigate how subtle variations in pitch that also communicate meaning is processed in the brain.”

A rare set of research participants

Chandrasekaran partnered with Dr. Taylor Abel, chief of pediatric neurosurgery at the University of Pittsburgh School of Medicine to study auditory information processing in 11 adolescent patients who were receiving neurosurgery treatment for severe epilepsy. They all had electrodes implanted deep in the cortex of the brain that is critical for key language function.

“Typically, communication and linguistics research rely on non-invasive recordings from the surface of the skin, which makes it accessible but not very precise,” Abel said. “A collaboration between neurosurgeon-scientists and neuroscientists, like ours, allowed us to collect high-quality recordings of brain activity that would not have been possible otherwise, and learn about the mechanisms of brain processing in a completely new way.”

To explore how the brain deciphers the melody of speech, researchers worked with the rare group of patients who had electrodes implanted in their brains as part of epilepsy treatment. While these patients actively listened to an audiobook recording of “Alice in Wonderland,” scientists tracked activity in multiple brain regions in real time.

What researchers found

Using the intracerebral recordings from the electrodes deep in the patient’s brain, researchers noted the Heschl’s gyrus section processed subtle changes in voice pitch — not just as sound, but as meaningful linguistic units. The brain encoded pitch accents separately from the sounds that make up words.

“Our study challenges the long-standing assumptions how and where the brain picks up on the natural melody in speech — those subtle pitch changes that help convey meaning and intent,” said G. Nike Gnanataja of UW-Madison’s Department of Communication Sciences and Disorders and co-first author of the study. “Even though these pitch patterns vary each time we speak, our brains create stable representations to understand them.”

Gnanataja says the research also revealed that the hidden layer of meaning carried by prosodic contours — the rise and fall of speech — is encoded much earlier in auditory processing than previously thought.

Similar research was conducted in non-human primates, but researchers found those brains lacked this abstraction, despite processing the same acoustic cues.

What’s next

By unlocking the hidden layer of speech, Chandrasekaran and his team discovered how the brain processes pitch accents, revealing profound implications for various fields.

“Our findings could transform speech rehabilitation, AI-powered voice assistants, and our understanding of what makes human communication unique,” he said.

Understanding early prosodic processing could lead to new interventions for speech and language disorders, such as autism, dysprosody in patients who have had a stroke, and language-based learning differences.

The study also highlights the unique role of linguistic experience in human communication, as non-human primates lack the ability to process pitch accents as abstract categories.

Additionally, these findings could significantly enhance AI-driven voice recognition systems by enabling them to better handle prosody, bringing natural language processing closer to mimicking human speech perception.