Scientists have been working on technologies that can turn a person’s thoughts into spoken words. Two new reports show how far the field has come.
AILSA CHANG, HOST:
An injury or disease that affects the brain can leave a person unable to speak. So scientists have been working on technologies that can turn a person’s thoughts into spoken words. NPR’s Jon Hamilton has more on two new reports in the journal Nature that speak to how far the field has come.
JON HAMILTON, BYLINE: There are five words in the sentence how do I do it. And for Pat Bennett, each one of those words is a struggle.
PAT BENNETT: (Vocalizing words).
HAMILTON: Bennett is 68 and has ALS, or Lou Gehrig’s disease, which attacks the nerve cells that control muscles. Dr. Jaimie Henderson, a professor of neurosurgery at Stanford, says the disease has disabled her vocal tract.
JAIMIE HENDERSON: She’s able to make vocalizations, but she’s not able to speak intelligibly.
HAMILTON: So Bennett volunteered to try an experimental system known as a brain computer interface. Henderson says it monitors signals from her brain as she tries to form words.
HENDERSON: The brain is still representing that activity. It just isn’t getting past the blockage.
HAMILTON: The system relies on tiny sensors implanted in Bennett’s brain. They’re connected to a computer that recognizes the patterns of brain activity associated with specific speech sounds, or phonemes. Henderson says those phonemes are then processed by what’s known as a language model.
HENDERSON: The language model is essentially a sophisticated autocorrect. So it takes all of those phonemes which have been turned into words and then decides which of those words are the most appropriate ones in context.
HAMILTON: The language model has a vocabulary of 125,000 words. And the system allows Bennett to produce more than 60 words a minute, a skill she demonstrates during a session in the lab.
BENNETT: I am thirsty. Bring my glasses here.
UNIDENTIFIED PERSON #1: Nice.
UNIDENTIFIED PERSON #2: That was Good.
HAMILTON: Still, Henderson says the system is far from perfect.
HENDERSON: She’s able to do a very good job with it over short stretches. But eventually, there are errors that creep in. So about 1 in 4 words on average is incorrect.
HAMILTON: A second report using a slightly different approach comes from a team at the University of California, San Francisco. Dr. Eddie Chang, a professor of neurological surgery, says instead of implanting electrodes in the brain, he places them on the brain’s surface. In a previous experiment, Chang’s team showed that they could turn a person’s thoughts into text on a computer screen. Chang says the team’s latest effort is a big improvement.
EDDIE CHANG: We worked with a new participant with a new device and got a lot better performance.
HAMILTON: Instead of just 15 words a minute, the new system does more than 70, using a much larger vocabulary. Also, the system has a voice designed to sound the way the participant did before she had a stroke. And it includes an avatar, a digital face that appears to speak as the woman silently thinks about speaking.
UNIDENTIFIED PERSON #3: Do not be afraid to ask me questions.
HAMILTON: Chang says those additions make the new system much more engaging.
CHANG: Hearing someone’s voice, of course, and then seeing someone’s face actually move when they speak. And so those are the things that we gain from talking in person as opposed to just texting.
HAMILTON: Chang says those features help make the new system about more than just communication.
CHANG: I think it goes beyond just restoring the words and the text. There is this aspect of it that is, to some degree, restoring identity and personhood.
HAMILTON: Turning thoughts into speech is still something that’s possible only in a lab. But scientists expect the technology to reach people’s homes in the next few years.
Jon Hamilton, NPR News.
(SOUNDBITE OF AKEMI FOX SONG, “SO FINE”)
Copyright © 2023 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
