Copeland is part of cutting-edge research into brain-computer interfaces at the University of Pittsburgh, recently awarded over $8 million by the National Institutes of Health. The team’s experiments are a peek into a potential transhumanist future more commonly associated with cyberpunk movies “The Matrix” and “Ghost in the Shell.” Since 2015, Copeland has lived with a transistor-like chip, known as a multi-electrode array, surgically implanted directly into his brain. Copeland’s chip records the rapid-firing of cellular neurons — an almost inscrutably complex neurological signal — which is ferried over to a computer for what’s referred to as “decoding.” This signal is subsequently “translated” into the desired, seemingly telekinetic actions of its user.
To date, one of the team’s biggest successes has been decoding the complicated neural signals to allow Copeland to control a nimble robotic arm. As such, the chip has obvious implications for those like Copeland who can regain lost abilities through the interface, and the research done with the brain-computer interface is likely to have a huge impact on future advances in accessibility. But there are tangential areas where the impact of the chip could be similarly massive.
When he wasn’t completing tasks set by the research team, Copeland put the BCI to recreational use, immersing himself in video games. The deft movements required for arcade-shooter “Raiden V,” or the mouse clicks crucial to hit card game “Slay The Spire” were simply thought into existence.
According to Andrew Schwartz, a leading neuroscientist at the University of Pittsburgh, the technology is straightforward to use with video games. Movement direction, on-off, yes-no — the inputs Copeland uses to play two-dimensional games such as “Sonic the Hedgehog 2” — are all “strongly represented” in brain activity and thus “simple” to decode. By contrast, the signals required to control the nimble robotic arm (one of the team’s big successes) and bigger, more cumbersome video games are more difficult to interpret. Nevertheless, Copeland, a self-confessed fan of role-playing games, has managed to play “Final Fantasy XIV” using a BCI. The process is “complicated,” he says, but not impossible.
Still, the video game industry has already shown its interest in acting as proof of concept for the technology. Celebrities and tech companies ranging from the game developer Valve to Elon Musk are invested in its revolutionary potential. But even as BCI research holds remarkable promise for Copeland and those like him, its implementation in the video game industry may spawn an ethical minefield.
Commercializing brain-computer interfaces
BCIs present life-changing possibilities, particularly for those who are paralyzed. But it’s the broader, nonmedical applications that have led to technology companies jockeying for position. In 2017, start-up Neurable made headlines with a video game controlled using a noninvasive BCI: the team attached an electroencephalogy (EEG) device — an assembly of sensors that sit just above the head — to a modified HTC Vive virtual reality headset. In a demo, players were tasked with escaping a futuristic laboratory without using their hands. Objects hovered in the air while participants cycled through them subconsciously, directing their mind to the object they wished to select and subsequently use.
“It felt like the computer guessing what was important to you,” says Neurable co-founder Adam Molnar, explaining their technology in terms of it “hacking” a neural response. “In an immersive environment you start to feel like this is a different kind of interaction than what we’ve felt before.” Despite hands-on success, few companies were interested in partnering with Neurable to commercialize the expensive consumer-oriented technology, and Neurable eventually pivoted its operations. Molnar remains tight-lipped on future details, but says the company is working in partnership with the military to develop neurological feedback for training purposes. Not quite the new horizon of everyday brain-controlled gaming the demo seemed to suggest.
Valve, the multibillion dollar video game company behind the digital store Steam and the “Half-Life” franchise is the highest-profile video game entity to publicly declare its own interest in BCIs. “We’re way closer to ‘The Matrix’ than people realize,” Gabe Newell, the studio’s president, gushed to IGN in April, stressing a post-BCI world will feel weirder than a movie could ever convey while transforming the video game industry. “It’s going to have a huge impact on the kinds of experiences we can create for people.”
What Newell actually imagines and how the technology could impact video games remains unclear, although Valve, a company with deep pockets, is ideally positioned to explore the nascent field. Mike Ambinder, an experimental psychologist at the studio, confirmed via email that the company is actively researching the technology and “engaging with external companies” it considers to be doing “appealing work in the space.”
Ambinder, however, posits a gradual phase of development toward his boss Newell’s science fiction end goal. Over the next few years, noninvasive BCIs could be incorporated into playtesting — an aspect of video game production that allows developers to see how players respond to the game before it’s released. The sector already utilizes physiological signals such as eye-tracking (to index attention) and electrodermal activity (to measure physiological arousal). Improvements for this application, he said, will be “relatively straightforward to implement” with current technology.
Further down the line is adaptive gameplay, a technological bridge between BCI playtesting and mind-controlled gaming. It’s this area which excites Ambinder and his colleagues most — the potential for video game experiences to respond to players’ feelings in real-time rather than relying on traditional inputs such as the keyboard, mouse or gamepad. What if a horror title knew you were scared and thus intensified its sound design to compound the sensation? Perhaps in-game weather would shift according to your mood; a rumbling storm to mirror a foul feeling. Or maybe a story-rich adventure could track your emotional response to non-playable characters and thus offer comforting interactions. Ambinder suggests incorporating such intimate feelings into games will lead to “qualitatively different” experiences. “Opportunities like this do not come along very often,” he wrote.
Navigating the ethics of mind-controlled gaming
The most remarkable element of the University of Pittsburgh’s laboratory work into BCIs could present the biggest ethical dilemmas. Copeland’s neurological signals don’t just travel from his brain to the robotic arm. Sensors on its mechanized fingertips also trigger signals in the opposite direction, capable of generating what he describes as tingling, pressure, warmth, tapping, and vibrations — a crude sense of touch in other words. This means participants using invasive BCIs, either now or in the future, aren’t just outputting their own data but opening up their brains to alteration from artificially-generated neuron firings.
Nita Farahany, a professor at Duke University who specializes in neuro-ethics, says this aspect of BCIs poses huge questions. “We have no idea what the implications of making those alterations are long term, and how much behavior can actually be manipulated,” she says. “Can you stimulate craving or can you stimulate addiction? Could you stimulate particular preferences?”
The answers to these questions could signal a dark future relationship between BCIs and video games. The industry has long sought to induce behaviors, including those with the potential to become compulsive. These instances range from the 1980s when teenagers parted with quarters at the arcade, to modern online titles which derive at least part of their income from loot boxes (often referred to by critics — and, increasingly, government regulators — as a form of gambling). Broader design principles often prioritize making players feel good, be that via the satisfaction of leveling up or the thrill of satisfying combat. The pleasures they already impart to sometimes problematic degrees — such as reports of players spending thousands of dollars in hit role-playing title “Genshin Impact” — that could be exacerbated by BCIs in the future.
More pressing than these hypotheticals is the data churned out by EEGs or invasive devices, and collected by the corporations behind them. Issues of privacy and access are key, particularly in regards to information that might track and record responses to particular stimuli, emotional states, and even decisions made. Farahany says that first, principles on the permissible uses and misuses of neurotechnology need to be established, followed by the drafting of a user bill of rights. But technological bills of rights, while much discussed, have proved mostly ineffectual, even in the established fields of AI and data, which already shape millions of lives. More concretely, Farahany wants to see privacy built into BCI devices and their user interfaces — an on-off switch for data transmission and options to toggle how different types of data is shared.
With or without an agreed BCI bill of rights, or really any clarity on the ethical questions, companies are set to continue investing in the technology. And video games, easily manipulatable and visually dynamic pieces of computer software, are well suited to demonstrate the technological potential, even if the ultimate outcomes of their commingling with neural interfaces leave many feeling justifiably uneasy. Even so, big barriers remain between Copeland’s activities and the technology reaching the wider public — none more formidable than the neuroscience itself.
Copeland, however, is understandably focused on the positives of a technology, which afford him greater autonomy. “If you’re trying to think of bad stuff that can happen, you can think of that, but I look at the good you can do with it,” he says. Far from merely anticipating greater degrees of freedom, Copeland believes in a future where BCIs are widely used.
“Eventually, it will get to the stage where someone who doesn’t ‘need’ [a BCI] could have it,” he says. “That will open up a whole new genre or space for people to be gaming in.”
Lewis Gordon is a video game and culture writer. His work has appeared in outlets such as VICE, The Verge, The Nation, and The A.V. Club. Follow him on Twitter @lewis_gordon.
Post a Comment