EEG for Consumer Products Todor Mladenov CSNL, GIST 2011

Outline Introduction EEG Signals Measuring EEG Signals EEG Examples Emotiv EPOC System Emotiv EPOC Applications Emotiv Demonstration Conclusions

Introduction The goals of today’s talk: Brief look at the generation of EEG signals. Investigate simple principles for measuring EEG signals Look at a some examples of EEG signals reflecting face motions Introduction to a user friendly BCI system BCI Examples Short demonstration

EEG Signals (1) The electroencephalogram (EEG) measures the activity of large numbers (populations) of neurons. First recorded by Hans Berger in 1929. EEG recordings are noninvasive, painless, do not interfere much with a human subject’s ability to move or perceive stimuli, are relatively low-cost. Electrodes measure voltage-differences at the scalp in the microvolt (μV) range. Voltage-traces are recorded with millisecond resolution – great advantage over brain imaging (fMRI or PET).

EEG Signals (2) EEGs require electrodes attached to the scalp with sticky gel for good contact Require physical connection to the machine

EEG Signals (3) Standard “10-20 System” Spaced apart 10-20% Nasion – point between the forehead and the skull Inion – Bump at the back of the skull Letter for region F - Frontal Lobe T - Temporal Lobe C - Center O - Occipital Lobe Number for exact position Odd numbers - left Even numbers - right

EEG Signals (4) Practical approach: electrical fields Electroencephalography (EEG) Electrocorticogram (ECoG) Local field potential or single neuron Scalp Soft Tissue Skull Dura Cortex 5 mm [Wolpaw04]

EEG Signals (5) Diagrammatic comparison of the electrical responses of the axon and the dendrites of a large cortical neuron. Current flow to and from active synaptic knobs on the dendrites produces wave activity, while AP are transmitted along the axon.

EEG Signals (6) Normal brain function involves continuous electrical activity Patterns of neuronal electrical activity recorded are called brain waves Brain waves change with age, sensory stimuli, brain disease, and the chemical state of the body An electroencephalogram (EEG) records this activity EEGs can be used to diagnose and localize brain lesions, tumors, infarcts, infections, abscesses, and epileptic lesions A flat EEG (no electrical activity) is clinical evidence of death

EEG Signals (7) Many neurons need to sum their activity in order to be detected by EEG electrodes. The timing of their activity is crucial. Synchronized neural activity produces larger signals.

EEG Signals (8) EEG potentials are good indicators of global brain state. They often display rhythmic patterns at characteristic frequencies

EEG Signals (9) EEG suffers from poor current source localization and the “inverse problem”- It is mathematically impossible to reconstruct a unique intracranial current source for a given EEG signal, as some currents produce potentials that cancel each other out.

EEG Signals (10) Somatosensory and Somatomotor cortex

Measuring EEG Signals (1) How do we check electrical charge in one spatial point? We need to compare The signal is distorted Brain Cerebrospinal fluid ( CSF ) Meninges Bone Muscle Fat

Measuring EEG Signals (2) Channel: Recording from a pair of electrodes (here with a common reference: A1 – left ear) If two electrodes are “active”, it is called “bipolar” recording. If one electrode is “silent”, it is called “monopolar” recording. The reference sites: ear lobe, mastoid, nose.

Measuring EEG Signals (3) Bipolar vs. monopolar recordings Monopolar recording is used in research, because it enables the researcher to localize the event of interest. EEG biofeedback (AKA neurofeedback) has unquestionably proven over the past 35 years to be the best "brain" training method to solve many brain problems such as the ADD-Autism continuum, Depression, Insomnia, Epilepsy, and many similar brain problems. Bipolar recording is used in BF (biofeedback), because it reduces shared artifacts. Electrodes should be placed on the sites with the strongest gradients of the potentials under training.

Measuring EEG Signals (4) Quality control of EEG recording EEG amplifiers must be calibrated with daily checks Acquisition parameters must be checked daily and keep the same The same procedures must be employed in all individuals All artifacts must be eliminated or taken into account prior to spectral analysis.

Measuring EEG Signals (5) Physiologic artifacts Eye movement Muscle activity ECG artifacts Skin artifacts Extraphysiologic artifacts Electrodes Alternating current (60 Hz) artifact Movements in the enviroment

Measuring EEG Signals (6) Montages Longitudinal Transverse Referencial Ear Cz Average (CAR)

Measuring EEG Signals (7) Polarity

EEG Examples (1) EEG Records During Epileptic Seizure Epilepsy is characterized by uncontrolled excessive activity of either a part or all of the central nervous system. Grand mal epilepsy: characterized by extreme neuronal discharges in all areas of the brain, last from a few seconds to 3 to 4 minutes. Petit mal epilepsy: Characterized by 3 to 30 seconds of unconsciousness or diminished consciousness during which the person has several twitch-like contractions of the muscle.

EEG Examples (2) Chewing

EEG Examples (3) Vertical Eye Roll

EEG Examples (4) Talking and moving head

EEG Examples (5) Yawn

EEG Examples (6) Eye closure and reopening

EEG Examples (7) Blink and triple blink

Emotiv EPOC System (1) A new form of man-machine interface A consumer product that offers a uniquely personal experience in the areas of: Games & Entertainment MMOGs – massively multiplayer online games Online Virtual Worlds Accessibility Semantic Application Interface

Emotiv EPOC System (2) Combines low powered wireless communication with a high fidelity signal acquisition system to produce a practical, light weight consumer product Self adjusts to fit most head shapes and sizes Features optimal sensor positioning that offers accurate, spatial resolution of the brain Incorporates high performance wireless connectivity A gyroscope to control avatar head or cursor movement

Emotiv EPOC System (3) Interact directly with elements on screen using the power of your thoughts Facial expressions are detected naturally, in real-time – allowing Avatars come to life without conscious involvement A player’s emotional experience can be monitored to dynamically adjusted content based on their excitement, engagement, calmness, tension, and frustration Player’s can manipulate objects by thought – giving them the sensation of “the Force” Designed to complement existing controllers, Emotiv provides a intuitive, highly immersive and natural extension of the user experience

Emotiv EPOC System (4) Emotiv Headset features: Saline based feltpads. Hi-performance wireless 14 Channel Raw EEG data 2 Axis Gyroscope EEGLAB support SDK for .NET programmers

Emotiv EPOC System (5) Channel names based on the International 10-20 locations are: AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4, References (P3, P4)

Emotiv EPOC System (6) Emotiv SDK provides decrypted raw EEG data. Sampling rate is 128sps. Each time data for 14 channel is achieved. C# application is written using Emotiv SDK for acquisition for EEG data. Also offline data are recorded and can be used in EEGLAB or offline classifier.

Emotiv EPOC System (7) Emotiv Procesing Panels Expressiv Panel- Used to mimic user’s facial expressions . Used for: MMOG, Social networks, AI characters – become more humanly interactive Affectiv Panel- Used to detect certain emotions or levels of interest. Used for: feedback loop – enables media to individual emotional engagement and adjust play accordingly. Cognitiv Panel- Used to control the movement of a virtual object. Classifies conscious, active intent. Used for: telekinesis – lift rotate, push

Emotiv EPOC System (8) Emotive Control Panel

Emotiv EPOC System (9) Cognitive Suite

Emotiv EPOC System (10) Expressive Suite

Emotiv EPOC System (11) Affectiv Suit

Emotiv EPOC System (12) EmoState Events

Emotiv EPOC System (13) Emotiv EPOC Specification

Emotiv EPOC System (14) Input Propagation

Emotiv EPOC System (15) Emotiv SDK

Emotiv EPOC System (16) Games - Multiplayer

Emotiv EPOC Applications (1) EMOTIV EPOC™ IN THE IMPLEMENTATION OF A SMART ROOM, Juan Miguel L. Andres, Marika Gianina H. Fernandez GOAL: Be able to aid paraplegic individuals in manoeuvring around a room via the Emotiv EPOC™

Emotiv EPOC Applications (2) Video - Brain Controlled Car

Emotiv EPOC Applications (3) Video – 5 axis robot arm – Alex Blainey

Emotiv Demonstration

Thank you! Q&A

References “Adaptation of Hybrid Human-Computer Interaction Systems using EEG Error-Related Potentials”, Ricardo Chavarriaga, Andrea Biasiucci, Killian F¨orster, Daniel Roggen, Gerhard Tr¨oster and Jos´e del R. Mill´an “Translating Thoughts Into Actions by Finding Patterns in Brainwaves”, Charles W. Anderson and Jeshua A. Bratman “Towards Emotion Recognition from Electroencephalographic Signals”, Kristina Schaaff and Tanja Schultz “A user-friendly SSVEP-based brain–computer interface using a time-domain classifier”, An Luo and Thomas J Sullivan Pfurtscheller G., et al., 1993, Brain Computer Interface a new communication device for handicapped people. Journal of Microcomputer Applications, 16:293-299. Neuper, C. et al., 1999. Motor imagery and ERD. Related Desyncronization, Handbook of Electroencepalography and Clinical Neurophysiology Vol. 6. Elsevier, Amsterdam, pp. 303-525. Grabner, R. H., Stern, E., & Neubauer, A. C. (2003). When intelligence loses is impact: neural efficiency during reasoning in a familiar area. International Journal of Psychophysiology, 49, 89-98. Brain-Computer Interfaces, Fabien Huske, Markus A. Kollotzek, Alexander Behm EEG/MEG: Experimental Design & Preprocessing, Lena Kastner, Thomas Ditye Classic EEG (ERP) / Advanced EEG, Quentin Noirhomme The ElectroEncephaloGram, Cognitive Neuropsychology, January 16th, 2001 Brain-Computer Interface, Overview, methods and opportunities, Emtiyaz (Emt), CS, UBC The emerging world of motor neuroprosthetics: a neurosurgical perspective, Neurosurgery. 2006 Jul; 59(1):1-14. Workshop on direct brain/computer interface & control, Febo Cincotti, August 2006

List of Abbreviations EEG – Electroencephalography ERP – Event Related Potential ERS – Event-Related Synchronization ERD – Event-Related Desynchronization HCI – Human Computer Interface BCI – Brain Computer interface VEP – Visual Evoked Potential SSVEP – Steady-State Visual Evoked Potential P300 – An ERP signal type ECG – Electrocardiography EMG – Electromyography fMRI – Functional Magnetic Resonance Imaging MEG – Magnetoencephalogy