A 2007 spinal cord injury left a man paralysed from the neck down. He has since demonstrated that he can communicate his thoughts using a brain implant system. It translates his handwriting into text.
BrainGate, a long-standing research collaboration, is the source of this device. It's a braincomputer interface (BCI) that uses artificial intelligence to interpret signals from neural activity during handwriting.
The man T5 (65 years old at the time) was not able to write because his hand and all of his limbs had been paralyzed over several years.
The experiment was published in Nature earlier this year. During it, the man focused as if writing well, thinking about how to make the letters with a pen and paper.
While he was doing this, electrodes in his motor cortex recorded signals from his brain activity. These signals were then interpreted on an external computer by algorithms decoding T5's imaginary pen trajectory. This mental traced 26 letters of the alphabet as well as some basic punctuation marks.
Frank Willett from Stanford University, first author of the study, says that the new system combines both rich neural activity captured by intracortical electrodes with the power of language models. These models, when applied to neurally decoded letter, can create fast and accurate text.
Similar systems have been transcribing brain activity into text since the BrainGate's inception. However, many interfaces before that were developed have used different cerebral metaphors to denote which characters to write. For example, point-and-click typing using a cursor controlled by the mind.
However, it was not known how well neural representations of handwriting, a faster and more dexterous motor skill, might be retained in brain, nor how they might be used to communicate with a Brain-computer Interface (or BCI).
T5 demonstrated the promise of a virtual handwriting program for those who have lost almost all their independent movement.
Diagram showing how the system works. (F. Willett et al., Nature, 2021, Erika Woodrum)
The man was tested and achieved writing speeds of approximately 90 characters per hour (or 18 words per min) with an accuracy of about 94 percent. Autocorrect enabled can increase this accuracy to up to 99 percent.
Researchers say that this rate is significantly faster than any previous BCI experiments using virtual keyboards. It's also almost equal to the speed at which smartphone users type in the men's age group, which averages 115 characters per minute or 23 words per hour.
Willett states, "We have learned that the brain can still prescribe fine movements even after the body loses the ability to do so,"
"And we've learned that complicated intended motions involving changing speeds and curved trajectories, like handwriting, can be interpreted more easily and more rapidly by the artificial-intelligence algorithms we're using than can simpler intended motions like moving a cursor in a straight path at a steady speed."
The researchers claim that alphabetical letters can be very different in their shape. This means that the AI can decode user's intent faster than other BCI systems which don't use dozens of inputs at once.
The system interprets the man's handwriting as it appears in his imagination. (Frank Willett)
The potential for this new technology is amazing, but the researchers stress that it is still a proof-of-concept. It has only been tested with one participant so it is not yet a clinically feasible product.
Next steps could include teaching others how to use the interface, expanding it to include symbols (such capital letters), improving the system's sensitivity, and providing more advanced editing tools to the user.
Although there is still much to be done, we may be able to see an exciting new development that will allow us to communicate with people who have lost it.
Researchers write that their results "open a new approach to BCIs" and show the possibility of decoding precise rapid, dexterous movements many years after paralysis.
"We believe the future intracortical BCIs are bright."
Nature reports the findings.
