The tech conglomerate Samsung has unveiled pioneering smart TV software that can be controlled with your brainwaves at the Samsung Developer Conference in San Francisco.
Samsung started Project Pontis three months ago, teaming up with the Center of Neuroprosthetics of the Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland.
The research, called Project Pontis, aims to make Samsung’s televisions s more accessible for people with physical disabilities like quadriplegia. The company wants to enable “users with physical limitations to change channels and adjust sound volume with their brains.”
“How can we provide accessibility to people who cannot move or who have extreme limitations on their movements,” Ricardo Chavarriaga, a senior scientist at EPFL who’s working on the project with Samsung, said during a panel at Samsung Developer Conference.
“We’re making tech that is more complex, that is more intelligent, but we should not forget this tech is being made to interface with humans,” he added.
The first step in making the brainwave-controlled TV software is to collect a sample of how the brain behaves when the user wants to do something like select a movie. Samsung and EPFL combine indicators from both the environment and brain scans to build a model and apply machine learning to let the user select shows using eye movements and brainwaves.
To collect the brainwaves in the prototype, a user wears a headset covered with 64 sensors while looking at an eye tracker. The headset is connected to a computer that’s mirrored to the TV.
The current prototype uses eye tracking to determine when a user has selected a particular movie. The system then builds a profile of videos the user gravitates toward, making it easier to provide lists of content in the future. The user ultimately makes a selection using eye tracking.
Samsung explained that with the aid of EPFL the company is also working on a system that goes further and relies on brain signals alone for users who aren’t able to control their eyes or other muscles reliably.
“One thing we have to take in account is everybody is different,” he said. Currently, the technology has to be tailored to each person because of variations in brains. “We believe we have to do the best for the person, so we have to personalize,” Chavarriaga explained.
Brain power
While developers aren’t yet making apps that can be controlled with the brain, Samsung’s doing research into the area. And it’s not the only company trying to use brainwaves to control devices. SpaceX and Tesla CEO Elon Musk in March 2017 launched Neuralink, a company dedicated to creating “neural lace,” which involves installing tiny electrodes in the brain to transmit thoughts.
And neuroscientists around the globe have been researching ways to make a digital interface for the brain. The technology is still in its early days, but it could one day replace touch screens and voice assistants in devices. Currently, most brain-computer interfaces (BCIs) are currently being created only for people who have suffered debilitating injuries that left them partially or completely paralyzed.
While Samsung’s first prototype also is targeted at accessibility, it’s too soon to say whether we’ll all one day be controlling our devices with our brainwaves, said Martin Kathriner, head of public affairs for Samsung Electronics Switzerland GmbH. There are limitations with the current hardware. The sensor helmet requires a layer of gel applied to the head, something consumers likely aren’t going to do at home.
“To us it’s an accessibility idea,” he said. “If it’s applicable to us one day as pro couch potatoes, I have no idea.”
Samsung initially considered building the technology into a smartphone but opted for the TV in part because of its bigger screen and because most homes have a TV, Kathriner said. He added that TVs also can be used as smart home hubs, which could be attractive for the brainwave technology.
Samsung plans to work on its second prototype through the first quarter of 2019 and then start tests in Swiss hospitals “where we start to explore how this situation, currently a prototype, is perceived by patients,” Kathriner said.
Samsung this week has been hosting its annual developer conference in San Francisco. SDC reflects Samsung’s big push to get developers to make software specifically for its devices. In the past, that’s meant making apps that work on the edge of Samsung’s curved smartphone displays or take advantage of its S Pen stylus. This year, that focus has turned to Bixby and artificial intelligence. But Samsung also has pushed developers to make apps for its other products, like its TVs and home appliances.
