Japanese Creator with ALS Performs with His Eyes and Brain Waves

Old & New video

By Ryuzo Suzuki / Yomiuri Shimbun Senior Photographer

12:23 JST, October 15, 2023

Masatane Muto, also known as EYE VDJ MASA, is a creator with amyotrophic lateral sclerosis (ALS) who operates music and video through eye tracking. Muto performed while successfully moving robotic arms with his brain waves at a free public-private-academic collaborative event called Change Tomorrow 2023 held in September in Minato Ward, Tokyo.

ALS is an intractable disease that gradually weakens the body’s muscles. Muto cannot speak on his own or move his limbs for the most part. He performed during the opening ceremony of the Tokyo Paralympics in the summer of 2021.

During the Change Tomorrow event, Muto closed his eyes to the music while increasing his concentration. Then, two robotic arms placed behind him and three robotic arms on the stage smoothly moved to start clapping, appearing as if Muto was doing so and encouraging the audience to clap along. When a new song played, the robotic arms were raised and lowered to the beat.

The robotic arm system produces three types of faint sounds in Muto’s ears. By combining two of three sounds and transmitting the selected pattern twice with his brain waves, nine commands are available to perform nine different actions.

When Muto increased his concentration, the 16 electrodes attached to his head would activate. Depending on the strength of his brain waves and other factors, the signal sent to the robotic arms would cause one of the commands to be executed.

After the performance, Muto held a handshake session using a different robotic arm. He selected with his brain waves one of the three options: a fist bump, high five or handshake, for the event attendees who wanted to participate in the session.

Keio University Prof. Kouta Minamizawa, a physical informatics specialist who created the robotic arm system, said, “I want to make it usable for daily life in the future.” Minamizawa studies “body augmentation,” which includes moving a robot in a remote location as if it were a part of the user’s body. He made many improvements on the system based on Muto’s feedback.

Muto played three of his original songs at the event. The music and images were controlled and provided by eye tracking which was conducted almost simultaneously with the robotic arms.

“It was very moving,” Muto said in a synthesized voice based on his own voice after the successful event. “I have been trying hard to change the stereotype of ALS as a progressive intractable disease that stops the body moving. I think the audience had a sense of hope in this respect.”

He added: “I want to continue research and development with colleagues by using the power of technology and creativity to open up a new future of not only supplementing lost bodily functions but also augmenting bodily functions.”