1. Emotional response to sound Influence of spatial determinants Master’s Thesis in the Master’s programme in Sound and Vibration Fredrik Hagman Presented by. Taylor Roh “ I believe in the power of failure “

2. Contents Introduction Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic Emotions and Hearing - Measure of Emotion 1. Subjective Measures 2. Physiological Measures

3. Introduction Negative feeling Positive feeling Auditory system : Main Channel for emotion induction & one of the main receptors

4. Introduction Previous - > evolutionary viewpoint human perception through the years of evolution (due to the cultural, geographical, developmental reason) Increasing acoustic intensity Sound source  ----  listener Tone, bursts, noise < - ecological stimuli Free field condition without any interference from boundaries

5. Introduction :: Main objective :: Room acoustic  -----  emotional perception Sound Known from Evolutionary view point Tested in Various acoustic environments “How our acoustic surroundings are influencing our emotional reaction to sound”

6. THEORY Part 1.

7. Spatial hearing Ear input signal => Spatial hearing 1. Theory Hearing mechanism >> structure = pinna ( highly irregular shape ) Pinna( essential role : linear filter > due to resonances) -> Ear canal ( travel down the external ear canal ) ->eardrum( transfer middle and inner ear ) Hearing mechanism & structure pinna -> individual 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

8. Spatial hearing Sound influenced by the head & torso [ crucial for the localization ability ] ITD( interaural time differences ) ILD( interaural level differences ) 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic Sound event (Psychoacoustic term) not sound Sound event ( perceived sound ) EX) ultrasound = without presence of a sound event Govern the spatial hearing Influence by head & torso

9. Localization and perception Human hearing = binaural 1. Theory 16cm The ability of hearing => Localizing sounds from different directions 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

10. Frequencies below 2kHz Human hearing -> phase difference for localization Higher -> time differences For example, 3-4kHz no longer diffracted -> ILD is substantial Localization and perception 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

11. Human ability ( determine the distance ) > much less accurate Underestimate the distance ( real : far away ) Overestimate the distance( real : closer than 1m ) Distance Cues 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

12. Distance judgment more accurate in reverberant environment compared to an anechoic environment Distance Cues 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

13. ~ 1960, The degree of spaciousness of room  solely  reverberation The late of 1960s, spatial phenomenon  early / lateral reflection, spatial impression, SI Perception of room acoustic 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

14. Reflections = localization of the source gets more cumbersome. The reverberation time = the energy density of the room to decline 60dB Direct sound = reflections + reverberation The direct sound in combination with reflections builds up a perceived auditory spaciousness around the listener Perception of room acoustic 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

15. SI(Spatial impression) estimated by measuring the level of difference between direct sound and reverberant sound referred to as H To fully characterize the SI, the interaural cross-correlation (IACC) cross-correlation : the left and right channel from binaural impluse response. IACC : correlation between the left & right ear To achieve as high SI as possible, the correlation should be kept low Perception of room acoustic 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

16. Ambisonic 4 channel, 4 sound pressure signals Ambisonic tests performed with headphones [ Head phone -> loud speakers ] To fully achieve a perfect auralization by loudspeakers, the influence of the listening room needs to be negligible. => Anechoic environment Ambisonics 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

17. Draw conclusions between different acoustic treatments on emotional response => Analysis of variance (ANOVA) Statistics 1. Theory 1. Spatial hearing 2. Localization and perception 3. Distance Cues 4. Perception of room acoustic 5. Ambisonics 6. Statistic

18. EMOTIONS AND HEARING Part 2.

19. IF environment is changed, the meaning and interpretation of the sound could be different. “What if the flushing toilet is heard in a reverberant cave?” “What if the flushing toilet is heard in a reverberant cave?” Emotions and hearing 2. Emotions and hearing

20. Room acoustic parameters -> alter our behavioural pattern or attitudes towards sound events Cognition – reason or thinking Emotion – soul, feeling or passion Traditional approach : cognition & emotion -> counterparts Today ‘s thesis : cognition & emotion -> co-operate informational system ( attract or pull us away from sth/ working as an alarm system ) Emotions and hearing 2. Emotions and hearing

21. Emotions --- change-  behavioural or physiological processes which further influence our perception, cognition, attention Emotional process 1. primary emotions : emotional responses by positive, negative or neutral 2. Secondary emotions : previous experiences from memory -> potential benefits or threats Many Thesis : how we process emotions which includes different levels This thesis : emotional processing  subjective behavioural / physiological processes. Emotions and hearing 2. Emotions and hearing Not included

22. It has been suggested that the human mind and body are connected when it comes to emotional induction. (Niedenthal, 2007 thesis) Studies -> the different reverberation times, a spatial determinant alter the emotional response to sound (Vastfjall,Larsson, & Kleiner, 2002 ) This thesis -> The further establishment of these determinants are the main purpose of this thesis. The sound source from IADS( International Affective Digitized sound) database Emotions and hearing 2. Emotions and hearing

23. : Bipolar scales : valence and arousal The valence dimension : From pleasant to unpleasant The arousal dimension : From activated to calm Subjective measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

24. Further emotional states- circumplex model of affection Subjective measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

25. Self reports – Verbal or visual SAM ( Self-Assessment Manikin) 9-points scale Graphical representations of emotions ( from pleasant to unpleasant ) Subjective measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

26. Graphical representations of emotions ( from calm to aroused) Subjective measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

27. Physiological measures – objective way of measuring emotion ex) heartbeat, breathing pattern or EEG This Thesis – electrodermal activity(EDA) Facial electronmyogrphy(EMG) Physiological measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

28. Electrodermal Activity(EDA) - all electrical phenomena in the skin EDA : good indicator of emotional arousal Linked to the human sweat glands sweat = emotionally arousing / stressful states This Thesis – electrodes => index and middle finger of the non-dominant hand Physiological measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

29. Each sharp peak = Skin conductor response(SCR) = discrete event -> change the skin conductance Stimuli onset and starting – latency / peak value / recovery half time Physiological measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

30. Skin conductor response(SCR) => Linked to a specific event and be masked by preceding responses related to something else A method to be able to separate skin conductive responses Were developed by with the intention to decrease the time Between stimuli and still be able to analyze their event relation => This method is adapted. Physiological measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

31. Facial electromyography(EMG) -EDA : arousal dimension of affect ( positive or negative ) EMG : facial expressions (precise measure) Two muscles ; Corugator Supercilii(CS) Zygomaticus Major(ZM) Physiological measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

32. Facial electromyography(EMG) Corugator Supercilii(CS) - expression related to unpleasant like frownig - measured by placing electrodes just above the eyebrow Zygomaticus Major(ZM) - linked to pleasant emotions such as smiling - This activity : the corner of the mouth to the lobe of the ear High ZM = happy / pleasant High CS = unpleasant Physiological measures 2.1. Measure of Emotion 2.1 Measure of Emotion 2.1.1. Subjective measures 2.1.2 Physiological measures

33. QNA Feel free to ask

34. THANK YOU To be continued by Ji-eun