Graz-Brain-Computer Interface: State of Research By Hyun Sang Suh

Overview: BCI systems The user performs a certain task, which has a distinct EEG signature The specific features are extracted from the EEG A pattern classification system uses these EEG features to determine which task the user performed The BCI presents feedback to the user, and forms a message or command

Motor execution vs. Movement imagination Imagination Execution ERD ERS 500ms time Subject 1, g3 Subject 2, f4

How can we discriminate four motor imagery tasks? Tongue Left Hand Right hand Foot

The mu-wave BCI n Mu wave activity occurs around roughly 12 Hz. n Alpha waves are strongest over the visual areas in the occipital lobe, But mu waves are strongest over the motor areas in the frontal lobe. n Mu activity changes as people perform or imagine movement. You have ERD/ ERS patterns depending on the motor imagery tasks n Mu wave activity occurs around roughly 12 Hz. n Alpha waves are strongest over the visual areas in the occipital lobe, But mu waves are strongest over the motor areas in the frontal lobe. n Mu activity changes as people perform or imagine movement. You have ERD/ ERS patterns depending on the motor imagery tasks Time

Subjects and experimental paradigm n Participants: Six female and three male healthy right- handed subjects. n Remain relaxed and avoid any motion during experiment. n Imagine the experience of movement (kinesthetic, MIK). n The arrow pointing represent one of the four different tasks (left hand, right hand, both feet and tongue). n EEG signal were recorded from 60 electrodes referenced to the left mastoid. n Participants: Six female and three male healthy right- handed subjects. n Remain relaxed and avoid any motion during experiment. n Imagine the experience of movement (kinesthetic, MIK). n The arrow pointing represent one of the four different tasks (left hand, right hand, both feet and tongue). n EEG signal were recorded from 60 electrodes referenced to the left mastoid.

Quantification of ERD/ ERS n First, band-pass filtering of each trial. n Second, squaring of samples (with smoothing) n Third, averaging of N trials. n The ERD/ ERS pattern is defined as the percentage power decrease (ERD) or power increase (ERS) comparison to one-second reference interval ( sec). n First, band-pass filtering of each trial. n Second, squaring of samples (with smoothing) n Third, averaging of N trials. n The ERD/ ERS pattern is defined as the percentage power decrease (ERD) or power increase (ERS) comparison to one-second reference interval ( sec).

Kappa coefficient and ITV n Kappa coefficient - To measure distinctiveness n Kappa coefficient - To measure distinctiveness Where acc is the accuracy derived by confusion matrix, n is the number of classes Where acc is the accuracy derived by confusion matrix, n is the number of classes n Intertask variability (ITV) - standard deviation of averaged ERD/ ERS n Intertask variability (ITV) - standard deviation of averaged ERD/ ERS

Frequencies and band power changes

Time-frequency maps displaying ERD/ ERS time

Maps displaying the topographical distribution of averaged band power High ITV Low ITV Intertask variability: ITV

Brainloop Interface for Google R. Scherer, G. Pfurtscheller. The self-paced Graz brain-computer interface: methods and applications. Computational Intelligence and Neuroscience 2007, 79825, 2007.

Mu vs. P300 BCIs n Requiring training n Work in real-time n 2D control possible n Continuous control n Affected by movement n Requiring training n Work in real-time n 2D control possible n Continuous control n Affected by movement n Requiring no training n Require averaging n 1D control only n Discrete control n Affected by distraction n Requiring no training n Require averaging n 1D control only n Discrete control n Affected by distraction Mu BCI P300 BCI

Phase Synchronization Features n Currently, BCIs system is not considered the relationships between EEG signals measure at different electrode recording. n We can obtain the additional information from this relationships. n Phase Locking value (PLV) is one of the method to quantify such relationships. n The PLV can measure the level of phase synchronization between pairs of EEG signals. n The PLV value of 1 means that the two channels are highly synchronized, whereas a value of 0 means no phase synchronization. n Currently, BCIs system is not considered the relationships between EEG signals measure at different electrode recording. n We can obtain the additional information from this relationships. n Phase Locking value (PLV) is one of the method to quantify such relationships. n The PLV can measure the level of phase synchronization between pairs of EEG signals. n The PLV value of 1 means that the two channels are highly synchronized, whereas a value of 0 means no phase synchronization.

Phase Synchronization Features

BCI Applications

Patient with Spinal Cord Injury n Spinal Cord Injury (SCI) - Damage or trauma to the spinal cord that result in a loss or impaired function - The effects of SCI depend on type of injury (i.e, a car accident, falls, sports injuries, or a disease) n Spinal Cord Injury (SCI) - Damage or trauma to the spinal cord that result in a loss or impaired function - The effects of SCI depend on type of injury (i.e, a car accident, falls, sports injuries, or a disease)

Restoration of hand movement in SCI patient

Functional Electrical Stimulation

BCI controlled FES G. Pfurtscheller, G. R. Müller, J. Pfurtscheller, H. J. Gerner, Rüdiger Rupp. 'Thought'- control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia. Neuroscience Letters 351, 33-36, 2003.G. Pfurtscheller, G. R. Müller, J. Pfurtscheller, H. J. Gerner, Rüdiger Rupp. 'Thought'- control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia. Neuroscience Letters 351, 33-36,

What is the Neuroprosthese? n It is a device which replaces nerve function lost as a result of disease or injury. n The neuroprosthetics can act as a bridge between functioning elements of the nervous system and damaged nerves. n It can be used in the spinal cord to allow standing in paraplegics. n It is a device which replaces nerve function lost as a result of disease or injury. n The neuroprosthetics can act as a bridge between functioning elements of the nervous system and damaged nerves. n It can be used in the spinal cord to allow standing in paraplegics. Hand prostheses

AUDITORY PROSTHETICS n most successful example of sensory prosthetic is the cochlear implant. n lack the cochlear hair cells that transduce sound into neural activity. n Extended to direct stimulation of the brainstem for those with dysfunctional cochlear nerves. n most successful example of sensory prosthetic is the cochlear implant. n lack the cochlear hair cells that transduce sound into neural activity. n Extended to direct stimulation of the brainstem for those with dysfunctional cochlear nerves.

VISUAL PROSTHETICS n The device uses electrical signals to bypass dead photoreceptors and stimulate remaining viable cells of the retina. n Images come from the external video camera worn behind the patient’s glasses. n The images are transmitted through a computer to electrodes attached to the retina n Reproduce the visual image in the occipital lobe. n The device uses electrical signals to bypass dead photoreceptors and stimulate remaining viable cells of the retina. n Images come from the external video camera worn behind the patient’s glasses. n The images are transmitted through a computer to electrodes attached to the retina n Reproduce the visual image in the occipital lobe.

BCI controlled Neuroprosthese n The BCI system is implanted his right hand and arm n Detect brain pattern (ERD/ ERS) of left hand foot imagery movement n Provide two graps patterns n The BCI system is implanted his right hand and arm n Detect brain pattern (ERD/ ERS) of left hand foot imagery movement n Provide two graps patterns

BCI controlled Neuroprosthesis G. R. Müller-Putz, R. Scherer, G. Pfurtscheller, R. Rupp. EEG-based neuroprosthesis control: a step towards clinical practice. Neuroscience Letters 382, , 2005.

BCI controlled Game

Thank you for your attention