Bravo-6, a 60-year-old man with ALS whose vocal tract is progressively paralyzed, tried to say a phrase and make a hand gesture in the same moment, and a full-body avatar on a screen did both. The system behind it is a single grid of 253 electrodes laid over the brain’s sensorimotor cortex, built by Edward Chang’s team at the University of California, San Francisco. Its results appeared Sept. 14, 2026, in Nature Neuroscience.
Chang, the senior author, sums up the premise this way: “Conversation is about much more than the words being spoken.”
Three participants, two avatars
The study followed three people, and the paper in Nature Neuroscience identifies each by code. Bravo-1r is 41 and has spastic quadriparesis and anarthria after a bilateral pontine stroke. Bravo-3 is 47, with quadriplegia and anarthria after a pontine stroke. Bravo-6 is the 60-year-old with ALS. The NIH release describes them as three patients with varying degrees of vocal-tract and bodily paralysis, and the paper traces the paralysis to two brainstem strokes and one case of ALS. Volume 29, issue 10 of the journal carries the work on pages 2445 to 2455, titled “Simultaneous Speech and Gesture Decoding for Multimodal Communication in Paralysis.”
The avatar results cover only two of them. Samantha Brosler, the lead author, and her colleagues write that the full-body avatar experiments were conducted after Bravo-3 withdrew from the trial, so every avatar demonstration and every simultaneous speech-and-gesture result involves Bravo-1r and Bravo-6. The NIH release on the work says the same thing in plainer terms: three patients with paralysis, two of whom controlled the avatar.
253 electrodes and a decoder trained on both tasks
The implant is an electrocorticography array of 253 disk-shaped electrodes at 3-millimeter spacing, each with a 1-millimeter recording contact. It sits on the surface of the brain rather than penetrating it. Participants attempted to say specific phrases and perform gestures such as hand waves and thumbs-up signs, sometimes separately and sometimes together, while machine-learning decoders translated the signals into avatar commands.
The central finding concerns training data. Models trained only on isolated speech or isolated gestures did not automatically carry over to simultaneous attempts, the paper reports, but training on both isolated and simultaneous data improved performance across every behavioral context. In Chang’s NIH quote the point is neurological: speaking while gesturing is “a multilayered, dynamic process involving the whole motor cortex.”
Nature Research’s own summary of the paper puts it as training the system with speech and gestures together improving performance and reducing mistakes, and it describes two participants animating the avatar by attempting gestures while answering conversational prompts.
Accuracy figures for Bravo-6 and Bravo-1r
The numbers differ sharply between participants, and the paper reports them separately. In the simultaneous task, with 10 gestures and 10 phrases, Bravo-6 reached 70.0 percent speech accuracy and 66.0 percent gesture accuracy in real time. In a gesture-only task with 10 gestures plus rest he scored 81.8 percent, and in a more conversational paradigm he reached 85.0 percent for gestures and 75.0 percent for speech.
Bravo-1r’s scores ran in opposite directions. In the conversation paradigm both decoders hit 100 percent, while a gesture-only task with just four gestures plus rest produced 61.7 percent.
The 66.0 percent gesture figure in Bravo-6’s simultaneous task sits below his 81.8 percent in the gesture-only task, though the tasks are not identical, since the gesture-only set includes a rest class and the simultaneous task pairs each gesture with a phrase. The sets are also small: 10 phrases and 10 gestures for Bravo-6, and only four gestures plus rest in Bravo-1r’s gesture-only task. Nature Research’s summary states the limitation plainly, saying that testing in more participants and with larger sets of words and gestures will be needed before the technology can be used in practice.
From the 2023 avatar to a wireless version
The project builds on Chang’s earlier avatar work. In 2023, UCSF and UC Berkeley reported a system that let Ann, a stroke survivor paralyzed for 18 years, communicate through a digital avatar at nearly 80 words per minute, almost six times faster than her previous 14-words-per-minute device. That system decoded 39 phonemes through 253 electrodes and animated the avatar’s face; the new work adds gestures from the same motor cortex, at the same time as speech.
The NIH release also notes that a fully implantable wireless version is planned for testing, and a Sept. 15 summary adds that eye-tracking alternatives remain slow and limited.
What the record cannot yet show is how the system performs outside a scripted task. Bravo-3 withdrew before any avatar sessions, which leaves two participants, 70.0 percent speech accuracy for the one with the larger task set, and a conversational result of 100 percent for the other.
This article was produced with the assistance of AI and reviewed by Morning Overview editors prior to publication.
More from Morning Overview
- The second-largest U.S. reservoir just fell to its lowest level ever recorded
- Security experts still urge phone owners to switch off one location-tracking setting
- The FBI warns a new phishing trick can hijack your account without your password
- The NSA says three phone features should be off whenever you aren’t using them
