1

2 Abstract - Two experimental paradigms : - EEG-based system that is able to detect high mental workload in drivers operating under real traffic condition ① An auditory workload scheme ② A mental calculation task - performance is strongly subject-dependent; however, the results are good to excellent for the majority of subjects

3 Introduction - Important issue in fields : pilots, flight controllers, operators of industrial plants - Be integrated with existing systems to maximize the performance - Approach #1 : A closed-loop system The system’s interaction with the operator is adjusted According to the operator’s mental workload - Approach #2 : The workload detector as an objective measure of mental workload To develop improved modes of human-machine interaction - Driving a car while performing additional tasks that model interaction with the car’s systems To obtain a system that is able to measure and mitigate mental workload (1) in real time and (2) in a real operational environment

4 The Experimental Setup - A system that is able to measure and mitigate mental workload in real time and in real operational environments. ⊙ the execution of secondary tasks not related to driving ⊙ Twelve male and five female subjects age 20 to 32 years old ⊙ Approximately 100 km/h on the highway in moderate traffic conditions ⊙ Not to speak during the experiment in order to avoid additional workload - Perform three types of tasks : #1 Driving the vehicle #2 An auditory reaction time task #3 A High mental workload task

5 #3 - 2 An auditory reaction time task #3 - 1 A mental calculation task ① German words links (left) and rechts (right) were randomly presented every 7.5 s ② Asked to count down in steps of twenty-seven ③ After two minutes, the subjects were asked for the final result ① A female news reader and a male voice reciting from a book ② The subject were instructed to follow the latter. ( To verify whether the subjects were engaged or not, they had to answer related questions) Subject had to their attention to one of two simultaneously presented voice recordings, replicating a situation in which several vehicle occupants are talking at the same time. #2 An auditory reaction time task ① Give three digit random number (between 800 and 999). ② Press corresponding buttons

6 A crucial purpose of the experiment is to investigate whether the output of the workload detector can be used to control the secondary task

7 Online Detection of Mental Workload #1 The Workload detector

8 ⊙ Artifact channel removal Such broad-band differences are characteristic for muscle artifacts (> 20 Hz) or eye artifacts (< 6 Hz). The channels that exhibit those differences are excluded. ⊙ Selection subset channel Subset is one of four candidate sets that potentially include frontal, occipital, and temporal scalp positions. ⊙ Spatial filtering Each of the selected channels is normalized by the common median reference signal (the median of all channels is subtracted from each channel) ⊙ Bandpass filtering bandpass filter using one of the bands listed above

9 ⊙ Classification Linear model whose parameters are computed by standard linear discriminant analysis (LDA) of the feature vectors The output is scalar value : Low workload (values below zero) High workload (values above zero) high and low workload, by means of a threshold scheme that employs a hysteresis, which makes the classification substantially more robust (m l < m h )

10 ⊙ Parameter Calibration To find suitable values for all the previously mentioned subject- and task-specific parameters, they use the well known cross-validation Technique. This procedure is performed for each possible combination of parameter candidates in the feature extraction part (EEG channel subset, spatial filter, frequency band, window length)

11 Result #1 Neurophysiological Interpretation

12 #2 Accuracy of the Workload Detector Auditory workload : (95.6% correct) Mental calculation workload : (91.8% correct)

13 #3 Performance Improvement

14

15 Abstract Methodology for relating neural variability to response variability, describing studies for response accuracy and response latency during visual target detection. Introduction - Analysis of trial-averaged ERP in EEG has enabled one to assess the speed of visual recognition and discrimination in terms of the timing of the underlying neural processes. - More recent work has used single-trial analysis of EEG to characterize the neural activity directly correlated with behavioral variability during tasks involving rapid visual discrimination. - These results suggest that components extracted from the EEG can capture the neural correlates of the visual recognition and decision making processing on a trial-by-trial basis. ‘ Cortically-coupled computer vision’

16 Linear methods for single-trial analysis - The goal of a BCI system is to detect neuronal activity associated with cognitive events. - Identify only one type of event (visual target recognition, differentiate this from other visual processing) - 64channels, 1000Hz, time window is 0.5s (32000 samples, dimensional feature vector) Sampling1000Hz (up to 500Hz) => L=50 time window, 20Hz (up to10Hz), 640 dimension - The task is a binary classification - The EEG activity is recorded as D x T values : D is the number of channels T is the number of samples - For reasonable classification : (1) Reduce the trial-to-trial variability by filtering the signal to remove 60Hz interference and slow drifts. (2) Reduce the dimentionality of the problem by stepping classification window every L-th sample. L = 50, the signal of interest is at 10Hz while faster signal variation are considered noise. 1, 2, ,499,500 D T 1, 2, ,9,10 D T

17 X is projected onto a single dimension y. (X is a matrix of channels by samples, and y is a row vector containing multiple samples). In this equation the inner product computes the average over trials and samples. When the intensity y averaged within the specified time window is used as classification criteria we achieve on this data an Az-value of 0.84.

18 EEG correlates of perceptual decision making - Using single-trial linear discrimination analysis To identify the cortical correlates of decision making during rapid discrimination of images - A series of target (faces) and non-target (cars) trials are presented in rapid succession - Stimulus evidence is varied by manipulating the phase coherence of the images - Using a set of 12 face (Max Planck Institute face database) and 12 car grayscale images (image size 512 x 512 pixels, 8-bits/pixel).

19 Early component ( = 170 ms following stimulus) Late component ( > 300 ms following stimulus)

20 Identifying cortical processes leading to response time variability - Studying visual target detection using an RSVP paradigm - During this task, participants are presented with a continuous sequence of natural scenes. RSVP : rapid serial visual presentation - 4 blocks of 50 sequences - Each sequence has a 50% chance of containing one target image with one or more people in a natural scene - These target images can only appear within the middle 30 images of each 50 image sequence. - The Each image was presented for 100 ms

21 - Gaussian profile : - Linear regression coefficients : latency of the component activity