Brain Computer Interface

BCI Outline BCI Concept: rebuilding instead of repairing BCI overview: actions from thoughts BCI Story: from fiction to reality BCI apps: Silent Talk

BCI Concept: direct communication pathway between a brain and an external device. Often aimed at assisting, augmenting or repairing human cognitive or sensory-motor functions. A brain–computer interface (BCI), or a brain–machine interface, is a direct communication pathway between a brain and an external device. Research on BCIs began in the 1970s at the University of California Los Angeles (UCLA) under a grant from the National Science Foundation, followed by a contract from DARPA. The papers published after this research also mark the first appearance of the expression brain–computer interface in scientific literature. The field of BCI took momentum after that, mostly toward neuroprosthetics applications that aim at restoring damaged hearing, sight and movement. Signals from implanted prostheses can, after adaptation, be handled by the brain like a natural sensory organ. Following years of animal experimentation, the first neuroprosthetic devices implanted in humans appeared in the mid-nineties.

BCI motivation: In USA, more than 200,000 patients live with the motor sequelae (consequences) of serious injury. There are two ways to help them restore some motor function: Repair the damaged nerve axons Build neuroprosthetic device

BCI Principle: (a) In healthy subjects, primary motor area sends movement commands to muscles via spinal cord. (b) But in paralyzed people this pathway is interrupted. (c) A Computer based decoder is used, which translates this activity into commands for muscle control.

BCI versus neuroprosthetics: uses artificial devices to replace the function of impaired nervous systems or sensory organs. connect the nervous system to a device E.g. cochlear implants, retinal implants. BCI: connect the brain (or central nervous system) with a computer system. E.g. EEG, Neuroprosthetics is an area of neuroscience concerned with neural prostheses—using artificial devices to replace the function of impaired sensory organs. The most widely used neuroprosthetic device is the cochlear implant, which, as of 2006, has been implanted in approximately 100,000 people worldwide. There are also several neuroprosthetic devices that aim to restore vision, including retinal implants. The differences between BCIs and neuroprosthetics are mostly in the ways the terms are used: neuroprosthetics typically connect the nervous system to a device, whereas BCIs usually connect the brain (or nervous system) with a computer system. Practical neuroprosthetics can be linked to any part of the nervous system—for example, peripheral nerves—while the term "BCI" usually designates a narrower class of systems which interface with the central nervous system. The terms are sometimes used interchangeably, and for good reason. Neuroprosthetics and BCIs seek to achieve the same aims, such as restoring sight, hearing, movement, ability to communicate, and even cognitive function. Both use similar experimental methods and surgical techniques.

BCI Input and Output: f Consider the potential to manipulate computers or machinery with nothing more than a thought. It isn't about convenience -- for severely disabled people, development of a brain-computer interface (BCI) could be the most important technological breakthrough in decades.

Brain “Features” User must be able to control the output: use a feature of the continuous EEG output that the user can reliably modify (waves), or evoke an EEG response with an external stimulus (evoked potential) Brain waves transformations wave-form averaging over several trials auto-adjustment with a known signal Fourier transforms to detect relative amplitude at different frequencies

Alpha and Beta Waves Studied since 1920s Found in Parietal and Frontal Cortex Relaxed - Alpha has high amplitude Excited - Beta has high amplitude So, Relaxed -> Excited means Alpha -> Beta

Mu Waves Studied since 1930s Found in Motor Cortex Amplitude suppressed by Physical Movements, or intent to move physically (Wolpaw, et al 1991) trained subjects to control the mu rhythm by visualizing motor tasks to move a cursor up and down (1D)

Mu Waves

Mu and Beta Waves (Wolpaw and McFarland 2004) used a linear combination of Mu and Beta waves to control a 2D cursor. Weights were learned from the users in real time. Cursor moved every 50ms (20 Hz) 92% “hit rate” in average 1.9 sec

P300 (Evoked Potentials) occurs in response to a significant but low-probability event 300 milliseconds after the onset of the target stimulus found in 1965 by (Sutton et al., 1965; Walter, 1965) focus specific

P300 Experiments (Farwell and Donchin 1988) 95% accuracy at 1 character per 26s

BCI System: Non-Invasive BCI Non-Invasive BCIs: easy to wear produce poor signal resolution because the skull dampens signals Signals recorded in this way have been used to power muscle implants and restore partial movement E.g. EEG. http://www.youtube.com/watch?v=G0cz- q1g6go&feature=player_embedded Signals recorded in this way have been used to power muscle implants and restore partial movement in an experimental volunteer. Although they are easy to wear, non-invasive implants produce poor signal resolution because the skull dampens signals, dispersing and blurring the electromagnetic waves created by the neurons. Although the waves can still be detected it is more difficult to determine the area of the brain that created them or the actions of individual neurons. video clip: A brain-computer interface based on EEG signals, developed by MMSPG. It allows to play a video game without any device like joystick.

Model Generalization (time) EEG models so far haven’t adjusted to fit the changing nature of the user. (Curran et al 2004) have proposed using Adaptive Filtering algorithms to deal with this.

Model Generalization (users) Many manual adjustments still must be made for each person (such as EEG placement) Currently, users have to adapt to the system rather than the system adapting to the users. Current techniques learn a separate model for each user.

Model Generalization (users) (Müller 2004) applied typical machine learning techniques to reduce the need for training data. Support Vector Machines (SVM) and Regularized Linear Discriminant Analysis (RLDA) This is only the beginning of applying machine learning to BCIs!

BCI System: Partially-Invasive BCI Electrocorticography (ECoG) with a sensor tapping directly into the brain's cortex is a very promising intermediate BCI modality. higher spatial resolution, better signal-to-noise ratio, wider frequency range first trialed on humans in 2004 on a teenage boy suffering from epilepsy to play Space Invaders. The units of speech known (phonemes) allow signals of a particular "discrete" nature: e.g. phonemes - "oo", "ah", "ee" and "eh" produce signals that reliably move a cursor on a computer screen. http://www.braingate2.org/systemOverview.html ECoG is a very promising intermediate BCI modality because it has higher spatial resolution, better signal-to-noise ratio, wider frequency range, and lesser training requirements than scalp-recorded EEG, and at the same time has lower technical difficulty, lower clinical risk, and probably superior long-term stability than intracortical single-neuron recording. This feature profile and recent evidence of the high level of control with minimal training requirements shows potential for real world application for people with motor disabilities.

Working of BCI: Every time we think, move, feel or remember something, our neurons are at work. That work is carried out by small electric signals that zip from neuron to neuron as fast as 250 mph some of the electric signal escapes, which can be detected, interpret and use them to direct a device of some kind. The reason a BCI works at all is because of the way our brains function. Our brains are filled with neurons, individual nerve cells connected to one another by dendrites and axons. Every time we think, move, feel or remember something, our neurons are at work. That work is carried out by small electric signals that zip from neuron to neuron as fast as 250 mph [source: Walker]. The signals are generated by differences in electric potential carried by ions on the membrane of each neuron. Although the paths the signals take are insulated by something called myelin, some of the electric signal escapes. Scientists can detect those signals, interpret what they mean and use them to direct a device of some kind. It can also work the other way around. For example, researchers could figure out what signals are sent to the brain by the optic nerve when someone sees the color red. They could rig a camera that would send those exact signals into someone's brain whenever the camera saw red, allowing a blind person to "see" without eyes.

Monkey thinks, Robot does! Experiments with monkey operating a robotic arm with its mind at the Pittsburgh University Medical Center http://www.makeahistory.com/index.php/your-details/222-the-worlds-first-commercial-brain- computer-interface- There has been rapid development in BCIs since the mid-1990s. Several groups have been able to capture complex brain signals using recordings from groups of neurons and use these to control external devices. They have managed to record signals from monkey and rat cerebral cortices in order to operate BCIs to carry out movement. Monkeys have navigated computer cursors on screen and commanded robotic arms to perform simple tasks simply by thinking about the task and without any motor output. The monkeys were trained to reach and grasp objects on a computer screen by manipulating a joystick while corresponding movements by a robot arm were hidden. The monkeys were later shown the robot directly and learned to control it by viewing its movements.

BCI System:

Cell-culture BCIs: Aim of the experiment: to study how brain cells function as a network and to learn more about one of the most complex devices in the known universe: the human brain. to find out exactly how the neurons do what they do and extract those rules and apply them in software or hardware for novel types of computing By watching the brain cells interact, scientists hope to understand what causes neural disorders, such as epilepsy. The research may also help the researchers in their quest to build "living" computers that combine neural and silicon systems. Computers lack the flexibility and adaptability of the human brain and perform poorly at pattern recognition tasks. A Florida scientist has developed a "brain" in a glass dish that is capable of flying a virtual fighter plane and could enhance medical understanding of neural disorders such as epilepsy. "We're interested in studying how brains compute," "If you think about your brain, and learning and the memory process, I can ask you questions about when you were five-years-old and you can retrieve information. That's a tremendous capacity for memory. In fact, you perform fairly simple tasks that you would think a computer would easily be able to accomplish, but in fact it can't." ............ Pattern recognition. Although computers can perform certain tasks extremely quickly, they lack the flexibility and adaptability of the human brain and perform particularly poorly at pattern recognition tasks. If we extract the rules of how these neural networks are doing computations like pattern recognition we can apply that to create computing systems

Cell-culture BCIs: Experiment at Univ. of Florida 25,000 neurons taken from the brain of a rat that are connected to a computer via 60 electrodes. rapidly began to reconnect themselves to form a living neural network. To put the experimental brain to the test, it is connected to a jet flight simulator via the electrode grid and a desktop computer. If you take these cells out of the cortex and you put them into one of these dishes, you remove all of the inputs—sensory systems like vision or hearing—that they would normally have. The only thing that's going on is the spontaneous activity of reconnecting. Experiment: The "living computer" was grown from 25,000 neurons extracted from a rat's brain and arranged over a grid of 60 electrodes in a Petri dish. There's a lot of data out there that will tell you that the computation that's going on here isn't based on just one neuron. The computational property is actually an emergent property of hundreds of thousands of neurons cooperating to produce the amazing processing power of the brain. It's essentially a dish with 60 electrodes arranged in a dish at the bottom. The brain cells then started to reconnect themselves, forming microscopic interconnections. If you take these cells out of the cortex and you put them into one of these dishes, you remove all of the inputs—sensory systems like vision or hearing—that they would normally have," DeMarse said. "The only thing that's going on is the spontaneous activity of reconnecting.. But as the neurons begin to receive information from the computer about flight conditions—similar to how neurons receive and interpret signals from each other to control our bodies—the brain gradually learns to fly the aircraft. The neurons will analyze data from the computer, like whether the plane is flying level or is tilted to one side," DeMarse said. "The neurons respond by sending signals to the plane's controls to alter the flight path. New information is sent back to the neurons, creating a feedback system.

Cell-culture BCIs: Warwick et al.: Controlling a mobile robot with a biological brain (Jan 2010) Closed-loop adaptive feedback system with approximately 105 neurons receiving information about robot position from the computer about flight conditions Information is sent back and forth creating a feedback system Are we creating Living Computers or Hybrid computers here? Silicon-based computers are very accurate and fast at processing some kinds of information, but they have none of the flexibility of the human brain. Brains can easily make certain kinds of computations that computers are unable to do, such as answering open-ended questions about what happened sometime in the past. To do a search like that in silicon is pretty difficult, unless you program [a computer] to specifically answer that question," DeMarse said. "Yet these neurons are able to do this in rats and in humans all the time." Understanding how neurons distribute information and encode it would allow scientists to take those rules and develop a silicon system that operates similar to the neurons, yet has the retention capacity of a silicon computer.

In Review… Brain Computer Interfaces Allow those with poor muscle control to communicate and control physical devices High Precision (can be used reliably) Requires somewhat invasive sensors Requires extensive training (poor generalization) Low bandwidth (today 24 bits/minute, or at most 5 characters/minute)

Future Work Improving physical methods for gathering EEGs Improving generalization Improving knowledge of how to interpret waves (not just the “new phrenology”)

Commercialization and companies: Ciberkinetics Honda Neural Signals Starlab NASA A few companies are pioneers in the field of BCI. Most of them are still in the research stages, though a few products are offered commercially. Neural Signals is developing technology to restore speech to disabled people. An implant in an area of the brain associated with speech (Broca's area) would transmit signals to a computer and then to a speaker. With training, the subject could learn to think each of the 39 phonemes in the English language and reconstruct speech through the computer and speaker [source: Neural Signals]. NASA has researched a similar system, although it reads electric signals from the nerves in the mouth and throat area, rather than directly from the brain. They succeeded in performing a Web search by mentally "typing" the term "NASA" into Google [source: New Scientist]. Cyberkinetics Neurotechnology Systems is marketing the BrainGate, a neural interface system that allows disabled people to control a wheelchair, robotic prosthesis or computer cursor [source: Cyberkinetics]. Japanese researchers have developed a preliminary BCI that allows the user to control their avatar in the online world Second Life [source: Ars Technica].

BCI video In this short talk, Dr. Shenoy describes how his team of Stanford researchers has built a system that achieves typing at 15 words-per-minute, just by "thinking about it". http://www.youtube.com/watch?v=I7lmJe_EXEU&feature=related

References: http://mlpr.wikidot.com/brain-computer-interface http://www.makeahistory.com/index.php/your-details/222- the-worlds-first-commercial-brain-computer-interface- http://www.wireheading.com/ http://news.nationalgeographic.com/news/2004/11/1119_0 41119_brain_petri_dish_2.html http://www.wired.com/dangerroom/2009/05/pentagon- preps-soldier-telepathy-push http://en.wikipedia.org/wiki/Brain-computer_interface http://www.wired.com/wired/archive/9.08/assist_pr.html http://www.youtube.com/watch?v=I7lmJe_EXEU&feature =related