Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen Pieter Laurens Baljon December 14, :30-13:00

Overview What is a BCI? EEG-based BCI –Preprocess, extract features, classify –Functional correlates of features Our BCI Setup –Online, offline and simulation Clinical- or theoretical relevance (or both?)

What is a BCI Interface between the brain and computer –Normally: hands and arms, voice –Could be deficient through stroke or ALS A BCI: –“must not depend on the brain’s normal output pathways of peripheral nerves and muscles” 1 Prosthesis connected to nerve endings is not a BCI

What is a BCI Adapted from Carmena et al. 2003, in PLoS Biology 1(2)

What is a BCI (Spelling example) YouTube:

What is a BCI (Pong example) YouTube:

What is a BCI Brain signal can come from –Invasive electrodes –Non-invasive measurements EEG, fMRI, etc. Underlying assumption –Intentions have discernible counterpart in brain signal

EEG-based BCI Sub fields of EEG-based BCI: –Signal processing on the EEG –Cognitive task for the subject (psychology) –Designing computer application (HMS) Typical pattern-recognition pipeline 1.Preprocessing 2.Feature extraction 3.Classification (not considered here)

The EEG: Preprocessing Preprocessing –Recombining electrodes can improve SNR 1. Spatial Filtering –Laplacian filters Subtract surrounding electrodes Vary distance to surrounding electrodes 2. Statistical recombination –Independent-Component Analysis –Common-Spatial Patterns

The EEG: Feature Extraction Signal is recorded in 2 or more conditions –Features should correlate with condition. –They must be detectable in single trial Two principal approaches: –Brute force machine learning Combine all imaginable features –Features with a functional correlate Potential shifts:Bereitschafts potential Rhythms:Alpha, mu, beta, etc. P300:Particular waveform

The EEG: Sensorimotor Rhythm (SMR) Function of periodical brain activity The predominance of a function –Expressed by spectral power Many rhythms are ‘idling-rhythms’. –Alpha rhythm over occipetal lobe (~10Hz) –Mu rhythm over motor cortex (~10 Hz)

The EEG: Sensorimotor Rhythm (SMR) University college, London & TU Graz VR application, controlling a wheelchair

The EEG: (SCP) & P300 Slow cortical potentials: –Low-pass filtered signal –E.g. Bereitschafts potential Ability to self regulate –Also used for neurofeedback –To treat ADHD P300 is ‘evoked potential’ –Less training –Indicate attended target Tetraplegic operating a speller application Outline of a P300 speller application. When target row/column is highlighted, it evokes a P300.

Training Subject: biofeedback –learning to control physiological ‘parameters’ –E.g. Heartrate, EEG-components System: any Pattern Recognition method –BCI competition: Different sorts of data Complexity of classifier –Reduces ‘meaningfulnes’ of transformation?

Training No ‘continuous mutual learning’. –Mostly epoch based –Update the system in between sessions –Danger of oscillations in feedback loop. There is no between-subjects design yet –Due to large inter-subject variability (?) –Could elucidate Effect of non-linear vs. linear feedback on EEG

Our BCI Setup (online) General purpose framework: BCI2000 Modular setup for –Amplifier driver –Signal processing –Application Open-source Borland C++ Large community: over 100 labs Initial problems running BCI experiments

Our BCI Setup (offline) Offline analysis in MatLab –Framework to test pattern recognition Setup similar to BCI2000 Simple addition of new features, thus far: –Preprocessing:ICA, CSP –Features:Spectral power, Hjorth –Classification:HMM, kNN, LDA, SVM

Our BCI Setup (simulation) Addition to BCI2000. Signal source can model SMR changes Collaboration with developers of BCI2000 Simulation in order to: –reverse engineer inner workings of BCI2000 –pretest settings for adaptivity

Clinical- & Theoretical relevance Most of the research is on healthy subjects Clinical research poses problems: –Proper operation requires extensive training –ALS Patients are only to learn control if they had it before the injury. –Small body of potential subjects Birbaumer reports a “significant increase in quality of life” They normally cannot communicate at all.

References [1] J. R. Wolpaw, N. Birbaumer, W. J. Heetderks, D. J. McFarland, P. H. Peckham, G. Schalk, E. Donchin, L. A. Quatrano, C. J. Robinson and T. M. Vaughan, “Brain-computer interface technology: A review of the first international meeting,” IEEE Transactions on rehabilitation engineering, vol. 8, pp. 164–173, Slide 1. Cover of the book Mathilda, about a telekinetic girl. Illustration: Quentin Blake Slide 3. PL Baljon (author) operating a BCI. Private collection. Photo: Mark Span. Slide 5, 6. Movies from youtube, filmed at CeBIT from Fraunhofer BCI, Berlin BCI. Slide 7. “Hans-Peter Salzmann gelang es 1996 erst nach monatelangem Training mit dem Thought Translation Device, den Cursor zu steuern.” Source : University of Tübingen Slide 12. “Controlling a wheelchair in a VR application” Source: University college, London & TU Graz. Slide 13. Tetraplegic operating a speller device: Source: NIBIB, Letter grid is taken from the BCI2000 manual. It is an excerpt from a trial with a P300 speller application.