1 Computerized Recognition of Emotions from Physiological Signals Physiological Signals Dec Mr. Amit Peled Mr. Hilel Polak Instructor : Mr. Eyal Braiman Physiological Signal Processing Laboratory

2 Introduction The assumption is that human emotions can be detected through physiological signals. The assumption is that human emotions can be detected through physiological signals. Different emotions have deferent influences on physiological signals. Different emotions have deferent influences on physiological signals. Physiological signals : Physiological signals : Heart rate Heart rate Body temperature Body temperature Body resistance Body resistance Physiological Signal Processing Laboratory

3 Project Goals To prove or disprove the assumption that human emotions have influences on physiologic signals. To prove or disprove the assumption that human emotions have influences on physiologic signals. That different human emotions have different influences on physiologic signals. That different human emotions have different influences on physiologic signals. To determine the most effective configuration for analysis and classification: To determine the most effective configuration for analysis and classification: The SVM algorithm parameters (including Kernel-Type). The SVM algorithm parameters (including Kernel-Type). The kinds of signals that should be recorded. The kinds of signals that should be recorded. Physiological Signal Processing Laboratory

4 Project Plan Sample physiological signals from participants and record it. Sample physiological signals from participants and record it. Process the data. Process the data. Activate the SVM algorithm in several configurations for training and classification. Activate the SVM algorithm in several configurations for training and classification. Analyze the classification results and its errors. Analyze the classification results and its errors. Physiological Signal Processing Laboratory

5 Sampling Technique The electrodes and sensors were attached to the examinee. The electrodes and sensors were attached to the examinee. The examinee was explained about the procedure. The examinee was explained about the procedure. The examinee was shown a movie clip without knowing in advance the type of emotion associated with it. The examinee was shown a movie clip without knowing in advance the type of emotion associated with it. While watching the clip, the examinee Physiological Signals were recorded. While watching the clip, the examinee Physiological Signals were recorded. Physiological Signal Processing Laboratory

6 Sampling Technique After each one of the five clips the examinee was asked about the strongest emotion he experienced while watching. Then, he was asked to grade the emotion intensity from 1 to 5. After each one of the five clips the examinee was asked about the strongest emotion he experienced while watching. Then, he was asked to grade the emotion intensity from 1 to 5. Physiological Signal Processing Laboratory

7 Sampling Technology We recorded three physiological signal: We recorded three physiological signal: Heart rate - NORAV E.C.G. and NORAV Software. Heart rate - NORAV E.C.G. and NORAV Software. Four electrodes were attached to the examinee and he was told to avoid movements while watching the clips. Four electrodes were attached to the examinee and he was told to avoid movements while watching the clips.  Temperature - a sensor that was connected to an amplifier and a LABVIEW application.  Resistance - two sensors that were connected to a 1 KOhm resistance in order to protect the examinee and a LABVIEW application. Physiological Signal Processing Laboratory

8 Sampling Technology We created a LABVIEW application that enabled us to sample and record the temperature and resistance data as txt files : We created a LABVIEW application that enabled us to sample and record the temperature and resistance data as txt files : Physiological Signal Processing Laboratory

9 Data Processing The data processing procedure main goal is to create an equal comparison base for the recorded data depending on the type of emotion and the type of signal. The data processing procedure main goal is to create an equal comparison base for the recorded data depending on the type of emotion and the type of signal. The process stages are: The process stages are: Phase 1 Phase 1 All signals : Equal length Equal length Normalization (different range and relative intensity) Normalization (different range and relative intensity) Heart rate only : Smooth and fix the signal with HPF Smooth and fix the signal with HPF Physiological Signal Processing Laboratory

10 Data Processing Phase 2 Phase 2 Production of all reasonable statistical data that can be gained from the recordings: Heart rate: Minimum, Maximum, Average, Standard Deviation, Variance. Heart rate: Minimum, Maximum, Average, Standard Deviation, Variance. Resistance: Minimum, Maximum, Average, Standard Deviation, Variance, Middle. Resistance: Minimum, Maximum, Average, Standard Deviation, Variance, Middle. Temperature: Minimum, Maximum, Average, Standard Deviation, Variance. Temperature: Minimum, Maximum, Average, Standard Deviation, Variance. This data is used as an input to the SVM algorithm ! Physiological Signal Processing Laboratory

11 SVM Algorithm The SVM algorithm is used for classification. The SVM algorithm is used for classification. There are several Kernel-Types. Each type is compatible to a different type of separation. There are several Kernel-Types. Each type is compatible to a different type of separation. The classification process includes 3 Stages: The classification process includes 3 Stages: Training Training Classification Classification Test Test Physiological Signal Processing Laboratory

12 SVM Algorithm We chose to run the algorithm with the Gaussian kernel : We chose to run the algorithm with the Gaussian kernel : This type of kernel was the only one who could make the classification. All other algorithm parameters had default values. All other algorithm parameters had default values. Physiological Signal Processing Laboratory

13 Classification The SVM algorithm was activated in the The SVM algorithm was activated in the “ Leave One Out ” method in order to increase its reliability. The SVM algorithm was activated in three modes: The SVM algorithm was activated in three modes: “ One against all ” (Two labels) “ One against all ” (Two labels) “ Multi class ” (Five labels) “ Multi class ” (Five labels) “ Decision value check ” (Two labels according to the Max. decision value variable) “ Decision value check ” (Two labels according to the Max. decision value variable) Physiological Signal Processing Laboratory

14 Classification All three modes were activated in 7 different input configuration : All three modes were activated in 7 different input configuration : Heart rate only Heart rate only Temperature only Temperature only Resistance only Resistance only Heart rata & Temperature Heart rata & Temperature Heart rata & Resistance Heart rata & Resistance Temperature & Resistance Temperature & Resistance Heart rate & Temperature & Resistance Heart rate & Temperature & Resistance Physiological Signal Processing Laboratory

15 Results Physiological Signal Processing Laboratory Decision Value Ckeck Multi Class One against All 180,26.66,26.66,16.66, ,100,100,100, ,100,100,100,53.33 H.R. & Temp & Res ,6.66,60,6.66, ,76.66,90,36.66, ,100,100,100,100H.R. 100,10,13.33,20, ,100,100,100, ,100,100,100,53.33Temp ,10,30,33.33, ,100,100,100, ,100,100,100,100Res 120,6.66,23.33,10, ,100,93.33,100, ,100,100,100,53.33 H.R. & Temp 240,20,46.66,16.66, ,100,100,100, ,100,100,100,100 H.R. & Res 180,23.33,20,36.66, ,100,100,100, ,100,100,100,53.33 Temp &Res

16 Conclusions Generally, the best results, that is to say, with the smallest general and specific errors, were received in the decision value check mode. Generally, the best results, that is to say, with the smallest general and specific errors, were received in the decision value check mode. Within that type of check, the best results were received while checking the Temperature data only. Within that type of check, the best results were received while checking the Temperature data only. Physiological Signal Processing Laboratory

17 Improvement Suggestions Use of other physiological signals. Use of other physiological signals. Use of advertisements and commercials instead of clips from known movies. Use of advertisements and commercials instead of clips from known movies. Check participants in different ages. Check participants in different ages. Check if the gender has influence on the results (Differences between men and women). Check if the gender has influence on the results (Differences between men and women). Physiological Signal Processing Laboratory

18 The End Thank you for your attention ! Physiological Signal Processing Laboratory