1. Introduction to Brain Computer Interface (BCI) Systems
Todor Mladenov
CSNL, GIST 2011

2. Outline Introduction Background EEG signals Noninvasive EEG methods
BCI Applications
Communication Issues
Research Issues
Conclusions

3. Introduction Machines that could be controlled by one's thoughts.
Brain computer interface devices (BCI) detect and translate neural activity into command sequences for computers and prostheses.
Electrodes recording from the brain are used to send information to computers so that mechanical functions can be performed.
BCI devices aim to restore function in patients suffering from loss of motor control e.g. stroke, spinal cord injury, multiple sclerosis (MS) and amyotrophic lateral sclerosis (ALS).
BCI will broaden repertoire of neurosurgical treatments available to patients previously treated by non-surgical specialists

4. Background
Richard Caton discovered electrical properties of exposed cerebral hemispheres of rabbits and monkeys.
Marxow discovers evoked potentials.
German Psychiatrist Hans Berger discovered alpha waves in humans and invented the term “electroencephalogram”(EEG).
Berger records electrical activity from the skull.
Gray Walter finds abnormal activity with tumors.
1950s - Grey Walter developed “EEG topography” - mapping electrical activity of the brain.
1970s - Research that developed algorithms to reconstruct movements from motor cortex neurons, which control movement.
1980s - Johns Hopkins researchers found a mathematical relationship between electrical responses of single motor-cortex neurons in rhesus macaque monkeys and the direction that monkeys moved their arms (based on a cosine function).
1990s - Several groups able to capture complex brain motor centre signals using recordings from neurons and use these to control external devices.

5. EEG Pioneers In 1929, Hans Berger
Recorded brain activity from the closed skull
Reportet brain activity changes according to the functional state of the brain
Sleep
Hypnothesis
Pathological states (epilepsy)
In 1957, Gray Walter
Makes recordings with large numbers of electrodes
Visualizes brain activity with the toposcope
Shows that brain rhythms change according to the mental task demanded

6. What is EEG
An electroencephalogram (EEG) is a measure of the brain's voltage fluctuations as detected from scalp electrodes.
It is an approximation of the cumulative electrical activity of neurons.

7. Physical Mechanism
EEGs require electrodes attached to the scalp with sticky gel
Require physical connection to the machine

8. Electrode Placement (1)
Standard “10-20 System”
Spaced apart 10-20%
Nasion – point between the forehead and the skull
Inion – Bump at the back of the skull
Letter for region
F - Frontal Lobe
T - Temporal Lobe
C - Center
O - Occipital Lobe
Number for exact position
Odd numbers - left
Even numbers - right

9. Electrode Placement (2)
International 10/20 system

10. Electrode Placement (3)
The EEG measures
not action potentials
not summation of action potentials
but summation of graded Post Synaptic Potentials (PSPs) (only pyramidal cells: dipoles between soma and apical dendrites)

11. EEG Channels
Channel: Recording from a pair of electrodes (here with a common reference: A1 – left ear) Multichannel EEG recording: up to 40 channels recorded in parallel

12. Brain “Features” User must be able to control the output:
use a feature of the continuous EEG output that the user can reliably modify (waves), or
evoke an EEG response with an external stimulus (evoked potential)

13. Brain “Features”
Generally grouped by frequency: (amplitudes are about 100µV max)
Type
Frequency
Location
Use
Delta
<4 Hz
everywhere
occur during sleep, coma
Theta
5-7 Hz
temporal and parietal
correlated with emotional stress
(frustration & disappointment)
Alpha
8-12 Hz
occipital and parietal
reduce amplitude with sensory stimulation or mental imagery
Beta
12-36 Hz
parietal and frontal
can increase amplitude during intense mental activity Mu
9-11 Hz
frontal (motor cortex)
diminishes with movement or intention of movement
Lambda
sharp, jagged
occipital
correlated with visual attention
Vertex
higher incidence in patients with epilepsy or encephalopathy

14. Brain Wave Transforms Wave-form averaging over several trials
Auto-adjustment with a known signal
Fourier transforms to detect relative amplitude at different frequencies
seperates spontaneous EEG signal to component frequencies and amplitudes
high frequency resolution demands long (in the range of seconds) analysis windows

15. Alpha Rhythm Frequency: 8 – 12 Hz Amplitude: 5 – 100 microVolt
Location: Occipital, Parietal
State of Mind: Alert Restfulness
Source: Oscillating thalamic pacemaker neurons
Alpha blockade occurs when new stimulus is processed

16. Beta Rhythm Frequency: 12 – 36 Hz Amplitude: 2 – 20 microVolt
Location: Frontal
State of Mind: Mental Activity
Reflects specific information processing between cortex and thalamus

17. Delta Rhythm
Frequency: 1 – 4 Hz Amplitude: 20 – 200 microVolt Location: Variable State of Mind: Deep sleep Oscillations in Thalamus and deep cortical layers Usually inibited by ARAS (Ascending Reticular Activation System)

18. Theta Rhythm Frequency: 5 – 7 Hz Amplitude: 5 – 100 microVolt
Location: Frontal, Temporal
State of Mind: Sleepiness
Nucleus reticularis slows oscillating thalamic neurons
Therefore diminished sensory throughput to cortex

19. Mu Waves Studied since 1930s Found in Motor Cortex
Amplitude suppressed by Physical Movements, or intent to move physically
(Wolpaw, et al 1991) trained subjects to control the mu rhythm by visualizing motor tasks to move a cursor up and down (1D)
(Wolpaw and McFarland 2004) used a linear combination of Mu and Beta waves to control a 2D cursor.
Weights were learned from the users in real time.
Cursor moved every 50ms (20 Hz)
92% “hit rate” in average 1.9 sec

20. Alpha and Beta Waves Studied since 1920s
Found in Parietal and Frontal Cortex
Relaxed - Alpha has high amplitude
Excited - Beta has high amplitude
So, Relaxed -> Excited
means Alpha -> Beta

21. Importance of EEG for HCI (1)
People with disabilities may have no other option for HCI than BCI.
Combining EEG with motion information to improve wearable activity recognition systems.
Performance improvement in both humans and artificial systems strongly relies in the ability of recognizing erroneous behavior or decisions. EEG activity evoked by erroneous gesture recognition can be used as a feedback for HCI. [1]
Hybrid Human Computer Interaction Systems (Hybrid – HCI).
Thoughts recognition [2]
Emotions recognition [3]

22. Importance of EEG for HCI (2)
Recognizing the emotional state of a person and its thoughts are directly applicable to the automotive industry – automatic braking, sleep detection, tiredness level monitoring, anxiety and tension monitoring.
Cars controlled by thoughts - AutoNOMOS Labs at Freie Universität Berlin.
New possibilities for gaming experience utilizing EEG controls.
Application of EEG systems in education. Example are the apps based on NeuroSky’s ( light a comfortable EEG sensors with custom fully integrated ASIC chip) equipment.

23. Reading the Brain (1) Direct Neural Contact ( Invasive )
Most accurate method
Highly invasive
Not possible with current technologies
Perhaps possible in future with e.g. nanobots

24. Reading the Brain (2) Electroencephalogy (EEG) Non-invasive
Measures electical activity in brain
Non-invasive
Susceptible to noise
Easy to use + low cost + portable
Most commonly used device in BCIs

25. Reading the Brain (3) Magnetoencephalogy (MEG)
Measures magnetic fields produced by electrical activity in brain
Non-invasive
Very accurate
High equiment requirements and maintenance costs

26. Reading the Brain (4) Functional Magnetic Resonance Imaging (fMRI)
Measures blood flow in brain using MRI (haemodynamics)
Blood flow correlates to neural activity
Studies the brain‘s function
Very accurate
Very high costs due to MRI

27. Direct (noninvasive) interfaces in EEG
The on-going electrical activity of the brain measured from scalp electrodes is called the electroencephalogram or EEG.
An event-related potential (ERP) is any measured brain response that is directly the result of a thought or perception. More formally, it is any stereotyped electrophysiological response to an internal or external stimulus.
Direct Interfaces via EEG
VEP – Visual Evoked Potential
SSVEP – Steady-State Visual Evoked Potential
P300 – ERP elicited by infrequent, task-relevant stimuli.
ERS/ERD – Event related synchronization/desynchronization

28. Event Related Potentials (ERP)
Averaging of trials following a stimulus
Noise reduction: The noise decreases by the squareroot of the number of trials
Far field potentials require up to 1000 measurements
Assumption: no habituation occurs (participants don‘t get used to stimulation)

29. Visual Evoked Potential (VEP)
Caused by Visual Stimulus
Occurs with flashing lights (3-5 Hz)
Have been used to monitor function during surgery for lesions involving the pituitary gland, optic nerve and chiasma.
Application:

30. Steady-State Visual Evoked Potential (SSVEP)
SSVEP are signals that are natural responses to visual stimulation at specific frequencies. When the retina is excited by a visual stimulus ranging from 3.5 Hz to 75 Hz, the brain generates electrical activity at the same (or multiples of) frequency of the visual stimulus.
Excellent signal-to-noise ratio and relative immunity to artifacts.
Applications:
SSVEP-controlled robots (Boston University)
User-friendly interface ( NeuroSky) [4]

31. P300 (1)
P300 is thought to reflect processes involved in stimulus evaluation or categorization.
It is usually elicited using the oddball paradigm in which low-probability target items are inter-mixed with high-probability non-target (or "standard") items.
Results in a positive curve on EEG after 300ms.
Strongest signal at parental lobe.

32. P300 (2) (Farwell and Donchin 1988)
95% accuracy at 1 character per 26s

33. ERS/ERD (1)
Event-related desynchronization (ERD) and event-related synchronization (ERS) is the change of signal's power occurring in a given band, relative to a reference interval.
People have naturally occurring brain rhythms over areas of the brain concerned with touch and movement. When people imagine moving, these brain rhythms first become weaker, then stronger. These two changes are called ERD and ERS, respectively.
ERS
oscillatory power increase
associated with activity decrease?
ERD
oscillatory power decrease
associated with activity increase?

34. ERS/ERD (2)
The imagination of either a left or right hand movement results in(5):
An amplitude attenuation (Event-Related Desynchronization (ERD)) of μ (8-12Hz) and central beta EEG-rhythms (13-30Hz) at the contralateral sensorial motor representation area and,
in some cases, in an amplitude increase (Event-Related Synchronization (ERS)) within the γ-band (30-40Hz) at the ipsi-lateral hemisphere(6).
EEG recorded from C3 electrode.

35. ERS/ERD (3)

36. ERS/ERD (4) Time Time Pre-Stimulus Post-Stimulus Pre-Stimulus
Typical Theta response
Apha response
ERS
Amplitude
Theta
Amplitude
Alpha
ERD
Time
Time
Pre-Stimulus
Post-Stimulus
Pre-Stimulus
Post-Stimulus
Stimulus
Stimulus

37. ERS/ERD (5) ERD ERS Theta Alpha Decrease Increase Power
Test power
Reference
power
Decrease Increase
Power
Decrease Increase
Power
ERD
ERS
Reference
power
Test power

38. ERS/ERD (6)
Decrease Increase
Power

39. ERS/ERD (7) Decrease Increase Power Visuo-spatial semantic LTM task
Semantic task: upper alpha relevant High activity (ERD red) is related to good performance
Visuo-spatial
semantic LTM task
Decrease Increase
Power
Working and short term memory task upper alpha is less relevant
Inhibition (ERS) of irrelevant semantic processes leads to good performance
Visuo-spatial
learning WM task

40. Communication Issues
Typical training time versus communication bitrate for the three main types of noninvasive BCIs.

41. BCI Applications (1)
For people with certain disabilities ( paraplegia, amyotrophia,.. ) BCI might be the only way of communication.
Human enhancements:
Cybernetic Organisms
Brainwave Synchronization
Exocortex ( intelligence booster)
Human manipulation
Mind-control
“Neurohacking”: unwanted reading of information from brain.
Neuroprosthetics:
Surgically implanted devices used as replacement for damaged neurons.

42. BCI Applications (2)
BCI – operated robot

43. BCI Applications (3)

44. BCI Applications (4)

45. BCI Applications (5)

46. Research Issues (1)
1. Make BCI easy, convenient and fun to use for the most popular consumer devices and apps.

47. Research Issues (2)
2. Energy budget analysis – Is it feasible to spend energy preprocessing data at the BCI headset in order to reduce the amount of transmitted data and save energy from the communication link. 3. Improve the signal to noise ration (SNR) for dry sensor electrodes. 4. Deliver EEG signals with medical systems quality for consumer applications and low price. New signal processing methods and algorithms.

48. Research Issues (3)
5. Currently, research is only beginning to crack the electrical information encoding the information in a human subject's thoughts. Understanding this “neural code” can have significant impact in augmenting function for those with various forms of motor disabilities.

49. Conclusions
BCI research has tremendous implications to the field of medecine.
Lot’s of on-going active research driven by the enormous interest.
Consumer Electronics companies such as Emotive and Neurosky are coming up with user friendly headsets.
New areas of application outside of medecine – gaming, control, education.

50. Thank you!
Q&A

51. References
“Adaptation of Hybrid Human-Computer Interaction Systems using EEG Error-Related Potentials”, Ricardo Chavarriaga, Andrea Biasiucci, Killian F¨orster, Daniel Roggen, Gerhard Tr¨oster and Jos´e del R. Mill´an
“Translating Thoughts Into Actions by Finding Patterns in Brainwaves”, Charles W. Anderson and Jeshua A. Bratman
“Towards Emotion Recognition from Electroencephalographic Signals”, Kristina Schaaff and Tanja Schultz
“A user-friendly SSVEP-based brain–computer interface using a time-domain classifier”, An Luo and Thomas J Sullivan
Pfurtscheller G., et al., 1993, Brain Computer Interface a new communication device for handicapped people. Journal of Microcomputer Applications, 16:
Neuper, C. et al., Motor imagery and ERD. Related Desyncronization, Handbook of Electroencepalography and Clinical Neurophysiology Vol. 6. Elsevier, Amsterdam, pp
Grabner, R. H., Stern, E., & Neubauer, A. C. (2003). When intelligence loses is impact: neural efficiency during reasoning in a familiar area. International Journal of Psychophysiology, 49,
Brain-Computer Interfaces, Fabien Huske, Markus A. Kollotzek, Alexander Behm
EEG/MEG: Experimental Design & Preprocessing, Lena Kastner, Thomas Ditye
Classic EEG (ERP) / Advanced EEG, Quentin Noirhomme
The ElectroEncephaloGram, Cognitive Neuropsychology, January 16th, 2001
Brain-Computer Interface, Overview, methods and opportunities, Emtiyaz (Emt), CS, UBC
The emerging world of motor neuroprosthetics: a neurosurgical perspective, Neurosurgery Jul; 59(1):1-14.
Workshop on direct brain/computer interface & control, Febo Cincotti, August 2006

52. List of Abbreviations EEG – Electroencephalography
ERP – Event Related Potential
ERS – Event-Related Synchronization
ERD – Event-Related Desynchronization
HCI – Human Computer Interface
BCI – Brain Computer interface
VEP – Visual Evoked Potential
SSVEP – Steady-State Visual Evoked Potential
P300 – An ERP signal type
ECG – Electrocardiography
EMG – Electromyography
fMRI – Functional Magnetic Resonance Imaging
MEG – Magnetoencephalogy