Loading ...

How fast can you type on your phone while texting someone? It's possible someone with paralysis will soon be typing faster than you — and with fewer typos.

A novel brain implant technology recently helped two people with paralysis communicate by using their thoughts to type on a virtual QWERTY keyboard. A study in Nature Neuroscience describes the research, which included one participant with amyotrophic lateral sclerosis (ALS) and another with a spinal cord injury. One came close to the typical smartphone typing speed of a person without paralysis, what the study authors say is a significant scientific advance.

Over the past 20 years, researchers' understanding of how the cortex encodes movement has improved as computers have. "Our machine learning models have gotten more sophisticated, and the hardware has caught up to support them," said study co-author Daniel Rubin, MD, PhD, a critical care neurologist at Mass General Hospital and assistant professor of neurology at Harvard Medical School. "We've been able to decode ever more complex and nuanced movements directly from cortical activity. And that brings us to this study." 

How They Did It 

Mass General Brigham Neuroscience Institute and Brown University researchers used an implantable intracortical brain-computer interface (iBCI) and artificial intelligence (AI) to translate neural activity into keystroke movements.

They embedded microelectrodes in the motor cortex of each participant's brain, part of the frontal lobe that controls voluntary movement. The participants were asked to use their thoughts to perform three distinct finger movements — up, down, and curled into the palm — for each of their 10 fingers. This allowed the researchers to distinguish the unique neural activity linked to 30 finger movements required for QWERTY keyboard typing. The electrodes "listened" to the brain activity taking place, essentially decoding it and translating it into text. An AI program helped predict the letters and words that the participants intended based on their neural activity. 

Their investigation was part of a larger, longer-term study, the BrainGate clinical trial, which has been exploring the use of brain-computer interface (BCI) technology to help people with paralysis of the limbs control computers and robotic limbs with their thoughts. The BrainGate clinical trial has been ongoing since 2004, according to Rubin.

"It's changed structure a little bit over the years," he said, but the long-term goal has been to work with people who have paralysis — either due to brainstem stroke, cervical spinal cord injury, or neurodegenerative conditions like ALS that cause progressive weakness — to find a way to use BCI technology to help restore communication, mobility, and functional independence. 

One of the most devastating symptoms of severe paralysis is the loss of the ability to communicate, Rubin said. Astrophysicist Stephen Hawking is often cited as a person with ALS who used technology to speak. But Hawking relied on assistive and augmentative communication technology, which lets users select one letter at a time using, for example, eye gaze tracker technology to communicate. Eye gaze technology requires a person to look at a computer screen with a camera embedded in it, and the camera tracks what their eyes are looking at and infers which letter they're selecting. "But those technologies are really slow. And they're very exhausting to use because your eyes are open for a long period of time," said Rubin. 

Typing with eye gaze technology produces anywhere from two to 15 words per minute. It's slow for the speaker and slow for the listener, said Rubin, adding, "There are also a lot of people with paralysis who can't use those systems at all because the paralysis affects their eye movements." 

Expand

Some Fast Typing 

In the study, the participant with a spinal cord injury was able to type up to 110 characters per minute and as many as 22 words per minute — faster than his typing speed prior to the injury — with 95% accuracy, Rubin said. The participant with ALS was able to type up to 47 characters per minute with 81% accuracy. For perspective, average keyboard typing speed in the general population is 52 words per minute. 

The typing speed of the faster participant came close to typical smartphone texting speed, which is about 27 words per minute, said Rubin. 

"This approach led to an improvement in communication rates compared to previous BCI studies," said Jennifer L. Collinger, PhD, a professor of physical medicine and rehabilitation at the University of Pittsburgh who studies neuroprosthetics that restore function for people with upper limb paralysis or loss. "The study demonstrates highly accurate decoding of bilateral finger movements to enable BCI control of typing on a QWERTY keyboard." 

The fact that one of the participants came close to the typing speed of people without disabilities was notable, added Collinger, who was not involved with the study. 

When the Motor Cortex Meets BCI 

In the study participants, the injury to the neural pathway occurred somewhere downstream from the cerebral cortex, "the part of the brain that does all of our thinking," said Rubin. "Thoughts, emotions, attention, personality, memories are all intact in these conditions." The connection between the cerebral cortex and the muscles is the issue. In the patient with a spinal cord injury, damage to the cord interrupts message-sending from the brain to the limbs. With ALS, degeneration of the motor neurons that carry the message from the cortex to the muscles causes paralysis. 

BCI attempts to use a sensor — either implanted or not — to record the neural activity coming directly from the motor cortex. Computer algorithms decode that neural activity to identify what someone is trying to do, even if they've lost the ability to do so, said Rubin. "So when people with ALS or cervical spinal cord injury think about moving their hand or think about moving their fingers, there's electrical activity in their motor cortex that looks more or less the same as it does in a person who doesn't have paralysis. So, we just decoded that." 

Then, in real time, they used computers to take that decoded signal and "do something useful," Rubin said. In this case, control a communication device. 

The technology grew out of the past 50 years' research of nonhuman systems. "It was really in the early 2000s that our understanding of how the motor cortex encodes simple movements works — how it does things like movement of the arm in a two-dimensional plane," said Rubin. Now, they're working on how the nervous system encodes that information and combining it with AI to process data in real time. 

Building Access to Deeper Communication 

Many of the people with paralysis who the scientists work with and speak with have taught them what it's like to try to communicate from their perspective. "The eye tracking systems are really slow, so people don't write out exactly what they're thinking word for word. They're sort of telegraphing their speech. They'll write out a single word or two words because they know that a lot of people that they're having conversations with don't have the patience to wait for them to type out a long message," Rubin said. 

Slow communication modes lead people with paralysis to be less fully themselves, he said. "Communication is more than just sort of letting someone know that you have pain or that you need something done for them. It is about sharing part of yourself and sharing your story and your personality. So, being able to have basic conversation is really important." 

Working With QWERTY 

The scientists chose the QWERTY keyboard, which includes 26 letters plus some punctuation, because it's familiar. It's the standard computer keyboard layout used by English speakers. "Most folks have seen a QWERTY keyboard and have used a QWERTY keyboard, which is designed to somewhat intentionally have letters next to each other and near each other that are very unlikely to appear next to or near each other in actual English words," Rubin said.

QWERTY also interfaces well with AI to predict the correct word a user types, even if they make a mistake and their eye is tracking near but not on the correct letter. "The deliberate design of the QWERTY keyboard allows AI to clean up a lot of errors so that people can type fast and accurately at the same time," Rubin said.

To help the computer understand how the individual participants use a QWERTY keyboard, the researchers did some data gathering sessions with each of them prior to seeing how well their iBCI system worked as a communication tool. They put up a prompt on the screen and then asked the participants to attempt an action. T he electrodes — the sensors on the motor cortex — listened to the activity in different parts of the cortex.

"We asked them to move their left pinky up, move your right pointer finger down, curl your left thumb into your finger, or into your palm — a series of instructed movements. Both participants have very severe processes, so neither of them can move their hands at all. So there wasn't anything to sort of see in the room. But they were working with us, sitting in front of a computer screen, and we were giving these instructions and they faithfully attempted each of these movements with their thoughts," he said.

As they gathered data from the sensors, the researchers saw that there were very distinct neural patterns of activity that differentiated each of the participants' movements. "We were pleasantly surprised at how well we could decode these different movements," said Rubin.

He said one of the other things that was "quite incredible about this data set" is that, over time, one participant's typing speed went up by 30 characters a minute. "He learned how to touch type using this interface, which is really cool," he said.

Collinger was impressed as well. "Neither of the participants was a particularly skilled typist prior to their injury, so that does not seem to be a prerequisite for the effectiveness of this approach," she said. "Skilled typists may achieve even faster communication rates."

What the Future Holds 

The researchers are excited to look at how the technology is working at the level of individual neurons, "at the acquisition of dexterity — learned movement at a precise level," said Rubin. He hopes it will teach them more about how to think about designing systems that decode ever more complex movements to help restore function to people with paralysis. 

Rubin said he looks forward to the day when he can prescribe BCI devices to his patients with paralysis for at-home communication. "In much the same way a cardiologist prescribes a pacemaker to someone with arrhythmia, I'm going to prescribe a device for a patient to help them speak, to communicate," he said. 

Collinger reported no conflicts. Rubin reported that the MGH Translational Research Center has a clinical research support agreement with Paradromics, for which DBR provides consultative input. Disclosure information for study authors is available in the original study publication.