1. Titel van de presentatie
:44
A tool to estimate STRESS (MENTAl state) in real time
Anne-Marie Brouwer
Maarten Hogervorst
Rob van de Pijpekamp
Jan van Erp

2. BACKgrond: estimating mental state using neurophysiology
Titel van de presentatie
:44
BACKgrond: estimating mental state using neurophysiology
Estimating mental (affective or cognitive) state of one individual at one moment in time
(Neuro)physiological effects of mental processes and not caused by confounds
EEG patterns looks different for difficult than easy scenarios, because:
mental stress differs
or:
.. more movements, causing motion artefacts
.. more visual information processing, leading to more active visual cortex ..
Almost 10 years ago, we started working on bci: controlling computers.. From there we moved to monitoring mental state from bio signals.
I will shortly describe a study that shows how we investigate the ‘real’ relation between neurophysio and mental processes, so that we know which variables actually carry information about the mental state under study, and then I describe an approach, or tool, that may be useful for out of the lab situations and is built on the methods used in the lab study.

3. LAB APPROACH: N-Back taSk
Titel van de presentatie
:44
LAB APPROACH: N-Back taSk
Difficulty varies across conditions while visual input and movement remain the same
‘Is the presented letter a target or not?’
0-1-2 back in 2- minute blocks
500 ms
2000 ms
0-back-target
1-back-target
2-back-target

4. Titel van de presentatie
:44
classification
Individually trained model (support vector machine)
Training on first part of data, testing on last (unseen) part of data

5. Relative sensitivity of different measures
Titel van de presentatie
:44
Relative sensitivity of different measures
2-vs-0-back, 2 minutes of data
EEG:
Most measures around 85% correct
One electrode suffices
Physiology:
Skin conductance, ECG around significance level (60%)
Breathing frequency: 68%
Eye:
Blinkduration: 50%, blink rate: 63%,
Pupilsize: 75%

6. Titel van de presentatie
:44
CombinatiON
EEG: 85% Physiology: 75% Eye: 75% All sensors: 91%
Hogervorst, Brouwer and van Erp JBF (2014) Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload. Frontiers in Neuroscience 8:322

7. Titel van de presentatie
:44
IF we want to apply this..
Generalization across situations
Generalization across subjects (training data)
Deal with movements and confounds
Idea of potential variables of interest

8. Titel van de presentatie
:44
ESTIMERE
Tool to estimate mental state in real time, out of the lab
No worries about confounds: ‘get all information that is possibly out there’
Automatically adapts to changing situation
Met Estimere willen we nu real time en in praktijksituatie mental state inschatten.
Praktijksituaties: altijd sprake van confounds. Die kun je echter ook als informatiebron behandelen. Als beweging en werklast voor een bepaald scenario altijd gecorreleerd zijn kun je daar gebruik van maken. Waarbij dan de vraag is of je die beweging het meest handig via ruis in EEGkanalen kunt meten of op een andere manier.

9. Titel van de presentatie
:44
CURRENT setup
Pupilsize Blinkfrequency Eyelid opening
Heart rate
Estimated stress
Classifier
mio
Keyboard hits mouse movements
Match?
Ground truth:
Subjectively experienced stress level

10. Titel van de presentatie
:44
CURRENT setup

11. Titel van de presentatie
:44
Flexible system
Easy to adapt:
Type of mental state
Sensors
Variables
Type of classification model
Selection of retraining data
Frequency of the probe question
Another type of ground truth …

12. Titel van de presentatie
:44
tool
To monitor users for adaptive automation or evaluation
For research:
How well can we estimate certain states In certain situations?
Which sensors and variables are most valuable?
How well does it generalize over persons and situations?
Tool to monitor users for adaptive automation or evaluation

13. Titel van de presentatie
:44
Lab -> real life
Pick an ideal mental state monitoring case:
Few body movements
Added value: patients that do not/cannot convey information well in other ways
Research in the same context as the application
1.12
Non-Emotional
Emotional
Few body movements, at least at time of measurement/interpretation
Omzeilen van generalisatie issue: je traint het model met data zo dicht mogelijk bij de situatie en persoon ‘of interest’
1.08
R to R interval
1.04
1.00
Brouwer, Hogervorst, Reuderink, van der Werf, van Erp (2015) Physiological signals distinguish between reading emotional and non-emotional sections in a novel. Brain-Computer Interfaces, 1-14. 20 40 60 80
100
120
Page number