Meta unveils wristband for controlling computers with hand gestures

Straits Times

2 days ago

Designed by researchers at Meta, this experimental technology reads the electrical signals that pulse through your muscles when you move your fingers.
SAN FRANCISCO – The prototype looks like a giant rectangular wristwatch. But it doesn't tell the time: It lets you control a computer from across the room simply by moving your hand.
With a gentle turn of the wrist, you can push a cursor across your laptop screen. If you tap your thumb against your forefinger, you can open an app on your desktop computer. And when you write your name in the air, as if you were holding a pencil, the letters will appear on your smartphone.
Designed by researchers at Meta, the tech giant that owns Facebook, Instagram and WhatsApp, this experimental technology reads the electrical signals that pulse through your muscles when you move your fingers.
These signals, generated by commands sent from your brain, can reveal what you are about to do even before you do it, as the company detailed in a research paper published on July 23 in the journal Nature.
With a little practise, you can even move your laptop cursor simply by producing the right thought. 'You don't have to actually move,' Dr Thomas Reardon, the Meta vice-president of research who leads the project, said in an interview. 'You just have to intend the move.'
Meta's wristband is part of a sweeping effort to develop technologies that let wearers control their personal devices without touching them. The aim is to provide simpler, quicker and less awkward ways of interacting with everything from laptops to smartphones – and maybe even to develop new digital devices that replace what we all use today.
Most of these technologies are years away from widespread use. They typically involve tiny devices surgically implanted in the body, which is a complicated and risky endeavor. These implants are tested solely with disabled people who cannot move their arms and hands, and need new ways of using computers or smartphones.
Top stories
Swipe. Select. Stay informed.
Asia Live: Thailand, Cambodia clash with jets, rockets, artillery in deadly border row
Asia 9 Thai civilians killed as Thai and Cambodian militaries clash at disputed border: Reports
Singapore Boy, 15, charged after being caught with vapes 5 times; ordered to stay 2 years in S'pore Boys' Home
Business MOM probing work injury claim flagged by late Sumo Salad boss Jane Lee: MOS Dinesh
Business Marina Bay Sands sees record second quarter with revenue up 37% to $1.77 billion
Singapore What's key to a good life? Most Singapore residents choose emotional and mental well-being
Singapore Over 2 years' jail for man who worked with wife to cheat her then boyfriend of $220k
Asia South Korea police raid offices of BTS' agency Hybe over share probe
Neuralink, a Silicon Valley start-up founded by Mr Elon Musk, aims to implant chips under the skull, beside the brain. Synchron, run by an Australian neurologist, hopes to implant devices inside blood vessels in the neck. These efforts, like those of many other start-ups, aim to read brain activity directly – an enormously complex process that must be tailored to the individual.
Meta's wristband uses a technique called electromyography, or EMG, to gather electrical signals from muscles in the forearm. These signals are produced by neurons in the spinal cord – called alpha motor neurons – that connect to individual muscle fibres.
Because these neurons connect directly to the muscle fibres, the electrical signals are particularly strong – so strong that they can be read from outside the skin. The signal also moves much faster than the muscles. If a device like Meta's wristband can read the signals, it can type much faster than your fingers.
'We can see the electrical signal before you finger even moves,' Dr Reardon said.
EMG has long provided a way for amputees to control prosthetic hands. But technologies that use the technique as a computer interface are only beginning to mature.
In 2012, three Canadian entrepreneurs founded a company called Thalmic Labs, which built an armband called Myo that sent simple computer commands using hand gestures. With the swipe of a hand, for instance, you could switch to a new slide in a PowerPoint presentation. But the company discontinued the product several years later.
Dr Reardon started similar research after founding a company called Ctrl Labs with two other neuroscientists he had met in a PhD programme at Columbia University. In 2019, the start-up was acquired by Meta, where it now operates within a research operation called Reality Labs.
Although Dr Reardon and his colleagues have been privately demonstrating their technology for years, they are only now beginning to publicly share their work because it is now mature enough for the marketplace. The key development is the use of AI techniques to analyse the EMG signals.
After collecting these signals from 10,000 people who agreed to test the prototype, Dr Reardon used a machine learning system called a neural network – the same breed of AI that drives ChatGPT – to identify common patterns in this data. Now, it can look for these same patterns even when a different person is using the device.
'Out of the box, it can work with a new user it has never seen data for,' said Dr Patrick Kaifosh, director of research science at Reality Labs and one of the neuroscientists that founded Ctrl Labs.
According to Dr Reardon, who is also known as the founding father of the Internet Explorer web browser at Microsoft, Meta plans to fold the technology into products over the next few years.
In 2024, the company demonstrated how its wristband could be used to control an experimental version of its smart glasses, which can take photos, record videos, play music and verbally describe the world around you.
Like technologies from Neuralink and Synchron, Meta's wristband could also provide new options for disabled people. Researchers at Carnegie Mellon are testing the wristband with people who have spinal cord injuries, allowing them to use smartphones and computers even though they do not have full use of their arms or hands.
Most people with these types of injuries retain the ability to activate at least some of their muscle fibres. This allows the device to read what their brain is trying to do, even though their bodies can't actually make it happen.
'We can see their intention to type,' said Dr Douglas Weber, a professor of mechanical engineering and neuroscience at Carnegie Mellon.
In a similar way, Meta's wristband lets you control a computer with the appropriate thought. Merely thinking about a movement is not enough. But if you intend to make a movement, the wristband can pick up on what you aim to do – even if you do not physically move.
'It feels like the device is reading your mind, but it is not,' Dr Reardon said. 'It is just translating your intention. It sees what you are about to do.' NYTIMES