Directeur : Mr S. PERREY (PR). Improving Usability in Human Computer Interfaces: an investigation into cognitive fatigue and its influence on the performance of hybrid brain computer interfaces Presented by Gérard Derosière 06/14/2012

I. Supervisors of the project and partners 2 Codirection in the context of a PhD thesis : SupervisorsPartnersSpecifities / CompetencesLogo Tomas Ward Department of Electronic Engineering, National University of Ireland Maynooth. Specialized in the development of Brain Computer Interfaces (BCIs). Stéphane Perrey Movement to Health (M2H) Laboratory, Montpellier 1 University. Understanding of the cerebral correlates during : - A motor task with fatigue occurrence; - The cognitive tasks.

II. Introduction 3 Brain-Computer Interfaces (BCIs): allow the control of an electrical system without voluntary movement needed, just by the mean of the cerebral activity (Gouy-Pailler, 2010). The user have to : - Do mental imagery or mental arithmetic; - Focus his attention to a light. How is it possible? These tasks require sustained attention. One main characteristic of the sustained attention:  The time-on-task (TOT) effect. Characterized by: - A progressive decrease of the attentional resources with time  measured through the increase of the reaction times (RTs) (Lim et al., 2010) ; - A modification of the cerebral activation (e.g., Paus et al., 1997)  non-stationarity. Attentional resources

4 II. Introduction TOT effect must be reduced during the use of a BCI for 2 reasons: - Decrease of attentional capacities  exhaustion, inability to use the BCI; - Modification of cerebral activation  ↓ of classification accuracy.  Proposed solutions (Baï et al., 2010) : to develop BCIs requiring minimal training, and thus, less mental load Pfurtscheller et al. (2010): the hybrid BCIs (hBCIs)  allow to enhance classification accuracy. Allison et al. (2010): development of a simultaneous hBCI.  Event-Related Desynchronization (ERD): mental imagery.  Steady-State Visual Evoked Potential (SSVEP): focalization of attention to a light.  Enhancement of the classification accuracy in the simultaneous condition.

5 III. Definition of the problematic Summary  BCIs require sustained attention;  Sustained attention is characterized by a "Time-On-Task" effect;  TOT effect must be reduced during the use of a BCI;  A new type of BCI to enhance classification accuracy: the simultaneous hBCI; Incompatibility between simultaneous hBCI and the aim to reduce TOT effect. Questions: - Does the simultaneous condition generate an early TOT effect? - If so, is it linked to an early modification of the cerebral activation? - Finally, does it generate an early decrease of the classification accuracy? Simultaneous hBCIs require divided attention  the mental load = higher than a simple condition.

6 IV. Proposed protocol PASAT RTs... Time RTs... Time SSVEP... Time SSVEP + PASAT RTs Session 2: Steady State Visual Evoked Potential (SSVEP) only Session 3: hybrid condition Session 1: Mental arithmetic only PASAT: Paced Auditory Serial Addition Test (mental arithmetic task). RTs: Reaction times. Concomitant measurements: Electroencephalograp hy (EEG), functional NIRS. Subjective aspects of fatigue. Classification accuracy tested in the three conditions, in a monomadal and multimodal manner. RTs Comments:

7 V. Expected results and conclusion Expected results: RTs TOT SSVEP + Mental arithmetic Mental arithmetic SSVEP SSVEP + Mental arithmetic Mental arithmetic SSVEP TOT Classification accuracy Project partly financed by the LABEX NUMEV  this will conduct to new development for pathological people. Classification accuracy TOT Multimodal (EEG+fNIRS) Monomodal fNIRS Monomodal EEG Tested in the three conditons:

Directeur : Mr S. PERREY (PR). Thank you for your (sustained) attention.