Video Game Technology Enables Paralysed Woman to Speak

Published on August 27, 2023

Researchers at UC San Francisco (UCSF) and UC Berkeley have achieved a groundbreaking feat by using technology commonly found in video games to allow a paralysed woman to speak again. The woman had been left paralysed following a stroke.

The team utilized a brain-computer interface, the same technology often used for driving facial animation in video games, to convert brain signals into electronic speech and facial expressions, as reported by The Independent.

This innovative breakthrough not only holds immense promise for individuals affected by paralysis but also showcases the remarkable potential of merging cutting-edge technology with medical research.

Video Game Technology Enables Paralysed Woman to Speak

Published on August 27, 2023

Researchers at UC San Francisco (UCSF) and UC Berkeley have achieved a groundbreaking feat by using technology commonly found in video games to allow a paralysed woman to speak again. The woman had been left paralysed following a stroke.

The team utilized a brain-computer interface, the same technology often used for driving facial animation in video games, to convert brain signals into electronic speech and facial expressions, as reported by The Independent.

Signals that would have controlled muscle movements in the woman's tongue, jaw, voice box, and face were intercepted. These signals were then connected to computers powered by artificial intelligence (AI) which could analyze brain activity and translate it into communication.

The researchers were able to decode the signals into three forms of communication: text, synthetic voice, and facial animation on a digital avatar. This included lip sync and emotional expressions. Notably, this marks the first instance of facial animation being synthesized from brain waves.

"Our goal is to restore a full, embodied way of communicating, which is really the most natural way for us to talk with others. These advancements bring us much closer to making this a real patient solution," stated Edward Chang, Chairman of neurological surgery at UCSF.

Michael Berger, CTO and co-founder of Speech Graphics company, commented, "Creating a digital avatar that can speak, emote, and articulate in real-time, connected directly to the subject's brain, shows the potential for AI-driven faces well beyond video games."

The study, published in the journal Nature, detailed that the woman could even cause the avatar to express specific emotions and move individual muscles. While this technology is currently a scientific proof of concept, it represents a significant step towards restoring communication to people with paralysis who can't speak.