Bridging the Gap Between the Brain and Communication

Posted by Susan Lubejko on August 12, 2021 in Language, Neurodegenerative disorders, Neuroscience | 1 Comment

What is your preferred mode of communication? For many of us, we feel comfortable speaking, writing, or typing to share our thoughts and connect with others. Our practiced texting thumbs and typing fingers have allowed us to document written language with incredible speed. In some cases, however, patients may find themselves with the inability to communicate with such ease. Rare cases of disorders such as locked-in syndrome – which can be caused by stroke, drug overdose, or traumatic brain injury and renders patients completely paralyzed but conscious – or amyotrophic lateral sclerosis (ALS) – which progressively causes damage to the nerves that control movement – can leave patients with the desire to communicate but the physical inability to speak, write, or type. While these disorders once left patients with little hope of regaining skills used in daily activities, new discoveries in neuroscience are making it possible for these patients to communicate. What kinds of neuroscientific and other technologies have been developed to restore this ability in these patients? What are the hurdles that remain to be overcome in supporting these new forms of communication?

When thinking about examples of people that have used technology to overcome an inability to speak, perhaps the first to come to mind is the prolific and influential physicist Stephen Hawking. Although a diagnosis of early onset ALS gradually left this brilliant scientist with the inability to move or speak, Hawking continued to write books, give speeches, and proceed with his research until his death in 2018. How was he able to do this? 

Through a bit of fascinating engineering, his glasses were outfitted with a motion-sensing infrared beam that was able to detect the slightest movement of his cheek. To speak, he would watch a computer monitor perched on his wheelchair as a cursor scrolled through the letters of the alphabet, numbers, and symbols, and could stop the cursor on his target with the smallest facial twitch. As he built these letters together to form words, a computerized voice would speak them. While this is an amazing technological development, you may imagine that using this process to speak is painstaking and slow. At times, this system would only allow Hawking to speak a word or two per minute. Other technologies built to help patients communicate use cameras to track eye movements across a screen in a similar fashion and with similar frustrations. 

To better deal with the limitations of this technology, many scientists and engineers have turned to neuroscience. Specifically, brain-computer interfaces have become an attractive option for helping paralyzed patients communicate. Brain-computer interfaces (BCIs) are defined as systems that “acquire brain signals, analyze them, and translate them into commands that are relayed to output devices” [1]. In the case of patients with trouble communicating due to paralysis, BCI technology can make it possible for the decoding of brain activity by computers to bypass the motor movements that are typically required for communication, even the small facial twitches or eye movements required with previous technologies. 

Earlier, yet effective, attempts to use BCIs for communication technology use electroencephalography (EEG) to record brain signals that can control computer output. EEG allows for the totally non-invasive recording of brain electrical activity through electrodes placed on the scalp. In the experiments done to validate these techniques, researchers flashed a grid of letters and numbers on a screen. With each subsequent “flash” of the grid, a letter or number would disappear. The researchers were able to define an electrical signature in the brain of each participant that corresponded to the sudden disappearance of their goal letter during a flash [2,3]. While these brain recording and signal processing techniques were stable and accurate, patients participating in the experiments still experienced limitations in the speed of their communication, with different versions of this technology allowing for 2.1-60 character choices per minute.

Although the use of EEG is an attractive option to record brain signals while avoiding the need for costly or risky brain surgery, more recent attempts at improving the communication of paralyzed individuals have gone right to the source of movement in the brain, the motor cortex. The motor cortex is a strip of brain tissue that sits on the surface of the cerebral cortex and is responsible for executing motor movements through coordination between its different parts that function to plan the movement and send signals to the muscles. Interestingly, it is thought that individuals that have become paralyzed can still generate brain activity corresponding to attempted or imagined movement in this part of the brain, as studies in healthy individuals have shown that brain activity in parts of the motor cortex are similar whether imagining or actually completing a movement [4,5]. In patients with paralysis, despite the fact that these signals can’t be sent to the corresponding muscles to produce the movement due to nerve damage, the types of BCI used here instead capture those signals at the level of the brain and send them elsewhere, like to a computer.

In a 2016 report by researchers in the Netherlands, a woman with ALS had surgery to implant electrodes that sat on the surface of the motor cortex in her brain to directly measure brain activity generated by her attempting to move her right hand. With practice, the woman was able to move a cursor across a computer screen and click on different characters to spell words by attempting to clench and unclench her fist. Although her motor cortex was no longer able to actually produce these movements in her hand, the brain activity remained effective at controlling the computerized device. This system was effective, was able to translate to home-use, and reportedly was well-liked and increased the quality of life for this patient. However, she was only able to produce one letter for every 30 seconds of effort, a process that appears even slower than the previously described technologies [6].

Excitingly, a 2021 research publication [7] appears to have built upon these previous technological innovations to improve the speed of information transfer by patient users. This study focuses on a male patient with a spinal cord injury that caused paralysis from the neck down. Similarly to the last study, the researchers implanted an electrode array on the surface of his brain in the motor cortex. However, instead of moving a cursor across a screen or focusing on a grid of letters and numbers, the patient was instructed to imagine he was actually writing the individual letters while holding a pen on lined paper. The researchers used an algorithm to train the computer to recognize the brain activity elicited when the patient imagined writing each letter and were able to use that information to predict future instances of the patient imagining writing their own sentences. Incredibly, this method was over 94% accurate during use (reaching 99% accuracy with autocorrect after the fact) and improved the communication speed to 90 characters per minute, which the authors of the study compare to smartphone typing speeds of 115 characters per minute in similarly aged patients without paralysis. While the authors are careful to point out that this study simply stands as a proof of concept in one patient, the results were long-lasting and effective, suggesting future studies on other patients with different movement disabilities are warranted. 

Clear steps forward have been taken in the technology that allows patients with paralysis or locked-in syndrome to use computers and communicate by spelling or writing. Fascinatingly, the use of BCIs for these patients doesn’t stop there, with other scientists and engineers using BCIs to lay the groundwork for allowing patients to speak and move through the help of computers as well. Recently, BCIs targeting the motor cortex have been used to decode the brain activity that produces the physical movements in the muscles of the mouth and throat that lead to song in songbirds [8] and speech in humans [9], which could allow for a future with computer-synthesized speech through the imagined movement of these muscles in patients with the inability to speak. Additionally, other neuroscientists and engineers are tackling the inability to move in paralyzed patients by using BCIs to decode activity in the motor cortex to control robotic arms [10] or even allow patients with paralysis to once again move their own arm via motor cortex brain activity decoding by computers to control stimulation of electrodes placed on or in the muscles controlling arm movement [11,12].

For humans, much of our interactions with the world, including communication through speech and writing, inherently require voluntary movement, leaving patients with disorders that prevent movement with disadvantages and frustrations. Recent neuroscience and bioengineering feats, however, aim to reduce these disparities by harnessing the power of the brain itself in conjunction with our ever-improving computer capabilities. These studies point to a future where patients with paralysis or locked-in syndrome will have access to a panel of these technologies to ameliorate communication and movement challenges. 

References

1. Shih JJ, Krusienski DJ, Wolpaw JR (2012) Brain-computer interfaces in medicine. Mayo Clinic Proceedings, 87(3): 268-279
2. Nijboer F, Sellers EW, Mellinger J, et al. (2008) A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clinical Neurophysiology, 119:1909-1916
3. Chen X, Wang Y, Nakanishi M, Gao X, Jung T, Gao S (2015) High-speed spelling with a noninvasive brain-computer interface. Proceedings of the National Academy of Sciences, E6058-E6067
4. Lotze M, Montoya P, Erb M, Hulsmann E (1999) Activation of cortical and cerebellar motor areas during executed and imagined hand movements: an fMRI study. Journal of Cognitive Neuroscience, 11(5): 491-501
5. Dechent P, Merboldt KD, Frahm J (2004) Is the human primary motor cortex involved in motor imagery? Cognitive Brain Research, 19: 138-144
6. Vansteensel MJ, Pels EGM, Bleichner MG, et al. (2016) Fully implanted brain-computer interface in a locked-in patient with ALS. New England Journal of Medicine, 375:2060-2066
7. Willett FR, Avansino DT, Hochberg LR, Henderson JM, Shenoy KV (2021) High-performance brain-to-text communication via handwriting. Nature, 593: 249-254
8. Arneodo EM, Chen S, Brown DE, Filha V, Gentner TQ (2021) Neurally driven synthesis of learned, complex vocalizations. Current Biology, 31:3419-3425
9. Anumanchipalli GK, Chartier J, Chang EF (2019) Speech synthesis from neural decoding of spoken sentences. Nature, 568:493-498
10. Hochberg LR, Bacher D, Jarosiewicz B, et al. (2012) Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 485:372-375
11. Bouton CE, Shaikhouni A, Annetta NV, et al. (2016) Restoring cortical control of functional movement in a human with quadriplegia. Nature, 533: 247-250
12. Ajiboye AB, Willett FR, Young DR, et al. (2017) Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration. Lancet, 389:1821-30