1. Human-Operator Monitoring System
- Linda J

2. Motivation
Human error - a frequent reason of catastrophes and ecological disasters
Temporal indisposition
Weariness
oversight
Monitoring of conscious brain involvement
Capable sensors available
Lack of complex solution exploiting wireless link technology, database systems and AI algorithms
Linda Emotion Detection

3. How to achieve Emotion Detection in Computer Systems?
Biometrics
Techniques like
1) EYE GAZE Tracking
2) Emotion Mouse
3) Facial Expression Detection
4) Artificial Intelligent Speech Recognition
These techniques combined to give
The BLUEEYES Technology
Linda Emotion Detection

4. What is BlueEyes ? What is BlueEyes not?
The complex solution for human-operator monitoring:
conscious brain involvement monitoring
triggering user-defined actions
data recording
What is BlueEyes not?
Doesn’t predict nor interfere with operator’s thoughts
Cannot force directly the operator to work
Linda Emotion Detection

5. BlueEyes is for ? Generic control rooms Power station Captain bridge
Flight control centers
Operating theatres – anesthesiologists
Common application
A simpler system version for drivers
Linda Emotion Detection

6. BlueEyes - benefits Prevention from dangerous incidents
Minimization of
ecological consequences
financial loss
a threat to a human life
The reconstruction of the course of operator’s work
Linda Emotion Detection

7. Face Recognition Consists of two steps : 1) Face recognition
2) Expression Detection
Of these, Expression Detection plays role
Expression glasses measures the movement of face muscles.
The movmt. is then compared with reference index to judge the emotion.
Linda Emotion Detection

8. Speech Recognition
Speech’ processed by complex voice processing algorithms.
Speech signal ==> set of words (voice prints).
voice prints(vp) generated - compared with reference index of vp corresponding to emotions.
The tone of the voice’s given priority.
Thus, emotion identified.
Linda Emotion Detection

9. Emotion Mouse A non-invasive way to judge emotion is through “Touch”.
Human touch – got by a mouse, having sensors on it.
The physiological attributes are then correlated to emotions using some models.
IR sensor – heart beat from the finger tips.
thermosister chips- body temperatures.
Linda Emotion Detection

10. Eye Gaze Tracking Saccades presence  User conscious
Retina - central vision (macula)
Central vision and the conscious brain
Saccade - a rapid eye jump to a new location which directly give indication abt visual attention.
Saccades presence  User conscious
Linda Emotion Detection

11. The eye movement sensor
Eye position measuring - direct infrared oculography
Oxy- and deoxyhemoglobin measurment
Two axial accelerometer
Ambient light sensor
Jazz Multisensor
Linda Emotion Detection

12. Wires, wires, wires... A serious limitation of: mobility comfort
safety
Linda Emotion Detection

13. Personal Area Network ! Solution: A wireless Personal Area Network
Linda Emotion Detection

14. Design objectives A mobile data acquisition device
A reliable real-time buffering, processing and recording
A clear visualization interface
Mechanisms for creating custom data analyzers
To ensure a possibility of distributing the processing among a number of computers
Linda Emotion Detection

15. System overview
Data Acquisition Unit
Central System Unit
Bluetooth
Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system.
Connection
Manager
Visualization
Module
Atmel 89C52
microcontroller
Data Analysis
Data
Logger
Jazz
Multisensor
Linda Emotion Detection

16. DAU - features Lightweight Runs on batteries - low power consumption
Data Acquisition Unit
Lightweight
Runs on batteries - low power consumption
Easy to use - does not disturb the operator working
ID cards for operator authorization
Voice transmission using hardware PCM codec
Atmel 89C52
microcontroller
Bluetooth
Jazz
Multisensor
Linda Emotion Detection

17. CSU - features Connection management Data processing Visualization
Central System Unit
Connection management
Data processing
Visualization
Data recording
Connection
Manager
Connection
Manager
Visualization
Module
Bluetooth
Data Analysis
Data
Logger
Access verification
System maintenance
Linda Emotion Detection

18. CSU - saccade detector
Saccade detector and conscious brain involvement level calculation
Raw eye
movement data
saccade
Differentiation
Conscious brain involvement level
Linda Emotion Detection

19. CSU - Visualization Module
Raw and processed data visualization using:
VU-meters
pie-charts
time series
audio and video data playback
Linda Emotion Detection

20. Implementation - DAU Microcontroller board PCM codec board
Batteries and voltage level monitor
Microcontroller board
Bluetooth module
PCM codec board
Linda Emotion Detection

21. Human-Operator Monitoring System
System demonstration
Linda Emotion Detection

22. DAU - components Atmel 89C52 microcontroller Bluetooth module BlueEyes
Data Acquisition Unit
microphone
earphone MC
PCM codec
Atmel 89C52
microcontroller
Bluetooth
module
Jazz Multisensor
Beeper
UART
LCD display
MAX232
UART
LED indicators
ID card
interface
ID card
Simple
keyboard
Linda Emotion Detection

23. Thank you for your attention
- Linda J
Linda Emotion Detection