2. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University "Culture-dependent emotional reactions to music Auditory roughness, cultural background, and musical tension-release judgments Pantelis N. Vassilakis, Ph.D. DePaul University School of Music

3. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Prelude Meaning is neither a property of a stimulus, nor the result of a physiological response to such stimulus, nor a contextual side- effect, nor a product of 'the mind.' Rather, it is a manifestation of the intersection among the stimulus as a physical entity, context, our physiology, and our mind and is partially received from, partially attached to the stimulus. Therefore, meanings and emotions associated with objects and events are not based on the isolated objects/events but, rather, on our relationship (as observers) to objects/events.

4. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University 1.Auditory roughness Concept & Models 2.SRA - Spectral Analysis New versus old methods 3.Roughness & musical tension Dependence on cultural background At A Glance

5. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Auditory Roughness Auditory Roughness: Term introduced in the late 19 th century by Helmholtz to describe the harsh, raspy sound quality of narrow harmonic intervals. E.g. An acoustic/sensory dimension of dissonance. One of the perceptual manifestations of interference, expressed as a function of a complex signal’s spectral distribution; an attribute of timbre _ Adding two sine signals with frequencies f 1 and f 2 results in a complex signal whose amplitude fluctuates between a minimum and a maximum value at a rate equal to |f 1 -f 2 |. _ Amplitude fluctuation rate : a) < ~15 fluctuations/second → Beating (e.g. 5 fluctuations/second)5 fluctuations/second b) between ~15 and ~75-150 fluctuations/second → Roughness (e.g. 35 fluct/sec) Regardless of origin (narrow harmonic intervals, fast trills, amplitude modulation, etc. ), such amplitude fluctuation rates give rise to the sensation of roughness. ( gradual move of 2 sinusoids from a major second to a unison harmonic interval )35 fluct/sec gradual move of 2 sinusoids from a major second to a unison harmonic interval c) > ~ 150 fluctuations/second → combination tones, envelope pitch, etc..

6. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Helmholtz (1885) Plomp & Levelt (1965) Kameoka & Kyriyagawa (1969a,b) Hutchinson & Knopoff (1978) Parncutt (1989) Daniel & Weber (1997) Sethares (1998) Leman (2000) Pressnitzer, McAdams & Colleagues (1997, 1999a, 1999b, 2000) (auditory periphery mechanisms models) MacCallum & Einbond (2008) Roughness calculation models

7. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University b 1 = 3.5 b 2 = 5.75 x* = 0.24 s 1 = 0.0207 s 2 = 18.96 Roughness, frequency separation, and register

8. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University General assumption: Roughness is proportional to A 1 * A 2 Roughness and amplitude Previous experimental studies: von Béckésy (1960) Terhardt (1974)

9. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Amplitude modulation depth versus degree of amplitude fluctuation

10. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Proposed Roughness Calculation Model Roughness estimation model (sine pairs): R = X * Y * Z (Vassilakis, 2001, 2005) X = (Amin*Amax) 0.1 Dependence of R on the absolute amplitude of the sines (Terhardt, 1974; Vassilakis, 2001) Y = 0.5 [2Amin / (Amin+Amax )] 3.11 Dependence of R on the relative amplitudes of the sines (von Béckésy, 1960; Terhard, 1974; Vassilakis, 2001) Z = e -b1s(fmax - fmin) – e -b2s(fmax - fmin) [b1 = 3.5; b2 = 5.75; s = 0.24/(s1fmin + s2); s1 = 0.0207; s2 = 18.96] Dependence of R on relative (frequency difference of the sines) and absolute (frequency of the lower sine) frequencies of the added sines (Kameoka & Kuriyagawa, 1969a&b; Plomp & Levelt, 1965; Sethares, 1998)

11. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness, phase, and signal envelope asymmetry Pressnitzer and McAdams (1999a,b)

12. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Comparison of 3 roughness calculation models

13. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Comparison of 3 roughness calculation models (Vassilakis)

14. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Drawbacks of traditional FFT Spectral Analysis Frequency and time values returned are forced to fit onto the time- frequency grid defined by the analysis window Frequency/temporal “smearing” and uncertainty on precise energy values

15. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Frequency analysis and spectral peak picking STFT algorithm based on the Reassigned Bandwidth-Enhanced Additive Model (Fitz & Haken, 2002; Fitz et al., 2003) Spectral Analysis Dual STFT, fine-tuning spectral analysis results _ Frequency: time derivative of the argument (phase) of the complex analytic signal representing a given frequency bin _ Time: frequency derivative of the STFT phase, defining the local group delay (correction that pinpoints the precise excitation time) Theory: _developed by Kodera et al. (1976) _expressed mathematically by Auger & Flandrin (1995) _implemented to sound spectral analysis by Fulop & Fitz (2006a,b; 2007)

16. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University SRA online http://musicalgorithms.ewu.edu/algorithms/roughness.html http://www.acousticslab.org/roughness

17. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Musical tension / release & Roughness _ tonal center _ dynamics _ pitch _ duration _ orchestration _ performance techniques _ sensory dissonance (presence of auditory roughness) Contrasts in terms of (Bigand, 1993, 1997; Bigand et al., 1996; Bigand and Parncutt, 1999; Krumhansl, 1996, 1997, 2002) _ Auditory roughness contrasts constitute just one of the expressive tools contributing to the creation of musical tension/release patterns. _ The assumption within the Western musical tradition that rough sounds should be avoided limits the range of roughness variations explored. Performance practices outside the Western art musical tradition explore a much wider roughness range. (Vassilakis, 2005)

18. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University _ Interaction between drone and melody (India) _ Quechua Haraui songs (Peru) _ Xingu river Taquara flutes (Brazil) _ Mijwiz (two paired single-reed pipes - Middle East) _ Ganga songs (Bosnia) Non-Western musical traditions & roughness

19. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Limited possibilities for extensive exploration of dynamic, pitch, tonal, and timbral (other than roughness) contrasts. Mijwiz (two paired single-reed pipes - Middle East) (Vassilakis et al. 2006) For its expressive power, the mijwiz relies on extensive explorations of narrow harmonic intervals, corresponding to sophisticated manipulations of roughness degrees. Pieces on the mijwiz are good candidates for examining a.roughness and musical tension contrast relationships, as determined by listeners familiar with the style A piece on the mijwiz: roughness/tension patterns b. the relationship between cultural background and musical tension judgments. requires high pressure air-flow to sustain a steady tone is often performed using the circular breathing technique has narrow melodic range and limited harmonic menu has two reeds that are activated just by air pressure

20. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University A piece on the mijwiz: roughness/tension patterns Musical piece: Recording of a stylized improvisation on the mijwiz, performed by A. J. Racystylized improvisation on the mijwiz Roughness Profile Estimation Based on SRA, a custom application that automates frequency analysis and roughness calculation of complex signals, at user-specified time intervals.SRA Purpose: Compare the roughness profile of a musical piece on the mijwiz to patterns of tension and release identified by listeners

21. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University A piece on the mijwiz: roughness/tension patterns Participants could listen to the piece prior to the experiment as many times as necessary to familiarize themselves with it. Tension / Release Judgments Improviser & Western-trained musicians Procedure Experiment designed using MEDS (Kendall, 2002) While listening to the piece, they were asked to tap the ‘M’ key on the keyboard whenever they sensed the musical tension increasing and the 'Z' key whenever they sensed the musical tension decreasing. They were instructed to continue tapping the appropriate key throughout the increase or decrease in tension and to tap no key if they did not sense a change in tension.

22. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness Profile

23. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University r=0.422 Roughness & Tension Profiles

24. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University r=0.068 r=0.422 Roughness & Tension Profiles

25. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness/Tension Profiles - Results Different listeners may employ different listening strategies that are related to musical expectations outlined by the different underlying musical traditions. Improviser (Middle Eastern) Roughness contrasts created through frequent moves between narrow and unison harmonic intervals constitute musical expressive cues that are widely accepted and employed within the Near Eastern musical tradition to communicate changes in musical tension. Composers and performers incorporate these contrasts when setting up tension/release patterns and listeners look out for them, even if implicitly, when they organize perceptually and attach meaning to musical pieces.

26. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University For listeners raised within the Western musical tradition, auditory roughness contrasts play a limited role when it comes to identifying musical tension/release patterns, while tonal and temporal contrasts provide more significant musical cues and set up stronger tension/resolution expectations. Listeners (Western-trained) Roughness/Tension Profiles - Results Different listeners may employ different listening strategies that are related to musical expectations outlined by the different underlying musical traditions.

27. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University r=0.068 r=0.422 Roughness/Tension Profiles - Results

28. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Ganga Correct and good performances are governed by complex and systematic rules. Consists of alternating solo and choral sections, with the choral sections involving moves between minor 2 nd and major 2 nd harmonic intervals, occasionally interrupted by short passages in unison. Within its geographical territory, ganga is often valued for its distinct sonic effect rather than its semantic content. Singing style common in Bosnia-Herzegovina and the Dalmatian Zagora regions of the Balkans. The length and content of the lyrics vary greatly among performances. Singers are relatively free in their choice of words, which are often just vocalizations (Petroviç 1977).

29. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Ganga rules Singers must sing loudly and maintain uniform strength among each other and through time. The voices must be as similar in timbre as possible, nasal, and without vibrato so that they blend. Singers must not move or dance while singing. They must stand tightly together, in an arch, turned slightly towards each other so that their voices will “collide” at the right point.

30. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Ganga rules Singers must sing loudly and maintain uniform strength among each other and through time. The voices must be as similar in timbre as possible, nasal, and without vibrato so that they blend. Singers must not move or dance while singing. They must stand tightly together, in an arch, turned slightly towards each other so that their voices will “collide” at the right point.

31. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Ganga rules Singers must sing loudly and maintain uniform strength among each other and through time. The voices must be as similar in timbre as possible, nasal, and without vibrato so that they blend. Singers must not move or dance while singing. They must stand very tightly together, in an arch, turned slightly towards each other so that their voices will “collide” at the right point. The two to three voice parts in the choral sections must be sung in minor and major 2 nd harmonic intervals with possible periodic insertions of unisons, always resolving on a major second.

32. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Ganga songs are good candidates for examining a.the relationship between auditory roughness profiles and patterns of musical tension, as determined by listeners familiar with the style b. the relationship between cultural background and musical tension judgments. If the conditions outlined in the rules are not fulfilled, the ganga “will not be good.” Ganga represents a rare folk vocal genre where the sense of a song is related more to its sound than to its lyrics, with the conditions for a good song that makes sense being also conditions that guarantee the perceptual salience of roughness contrasts. Ganga rules (cont.)

33. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University A ganga song: roughness/tension patterns Musical piece: Odkad seke nismo zapjevale in “Bosnia: Echoes from an Endangered World – Music and Chant of the Bosnian Muslims” Ted Levin and Ankiça Petroviç, © 1993 Smithsonian Folkways RecordingsOdkad seke nismo zapjevale Roughness Profile Calculation Based on SRA, a custom application that automates frequency analysis and roughness calculation of complex signals, at user-specified time intervals.SRA Purpose: Comparison among the roughness profile of a ganga song and patterns of tension and release identified by i) a ganga expert and ii) musician listeners unfamiliar with the ganga style.

34. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Participants could listen to the piece prior to the experiment as many times as necessary to familiarize themselves with it. Tension / Release Judgments Bosnian-born ganga scholar & 10 Western-trained musicians Procedure Experiment designed using MEDS (Kendall, 2002) During the experiment, they were asked to tap the ‘M’ key on the keyboard whenever they sensed the musical tension increasing and the 'Z' key whenever they sensed the musical tension decreasing. They were instructed to continue tapping the appropriate key throughout the increase or decrease in tension and to tap no key if they did not sense a change in tension. A ganga song: roughness/tension patterns

35. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness Profile

36. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness & Tension Profiles

37. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness & Tension Profiles

38. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness & Tension Profiles r=0.44

39. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness & Tension Profiles

40. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness & Tension Profiles

41. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness/Tension Profiles - Results

42. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University r=0.44 r=0.36 r=0.82 Roughness/Tension Profiles - Results

43. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness/Tension Profiles - Results Different listeners may employ different listening strategies that are related to musical expectations outlined by the different underlying musical traditions. Ganga scholar (Bosnian) Roughness contrasts created through frequent moves among minor 2 nd, major 2 nd, and unison harmonic intervals constitute musical expressive cues that are widely accepted and employed within the ganga style to communicate changes in musical tension (“harmonious dissonance”). Ganga singers incorporate these contrasts when setting up tension/release patterns and initiated listeners look out for them, even if implicitly, when they organize perceptually and attach meaning to the songs.

44. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University For listeners trained / raised within the Western musical tradition, auditory roughness contrasts play a limited role when it comes to identifying musical tension/release patterns, while tonal and temporal contrasts provide more significant musical cues and set up stronger tension/resolution expectations. Listeners (Western-trained) Roughness/Tension Profiles - Results Different listeners may employ different listening strategies that are related to musical expectations outlined by the different underlying musical traditions.

45. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University r=0.44 r=0.36 r=0.82 Roughness/Tension Profiles - Results

46. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University r=0.068 r=0.422 Roughness/Tension Profiles - Results

47. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University Roughness/Tension Profiles - Results Both experiments presented indicate that, using familiar sound-parsing strategies on pieces created employing unfamiliar sound-organization tools, may leads to listener interpretations that differ from the performer’s expressive intent. Musical tension and release are culture-specific concepts, guided by the equally culture-specific musical cues used to organize and recognize them.

48. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University References Auger, F. and Flandrin, P. (1995). "Improving the readability of time frequency and time scale representations by the reassignment method," IEEE Transactions on Signal Processing 43: 1068-1089. von Békésy, G. (1960). Experiments in Hearing. New York: Acoustical Society of America Press (1989). Beyer, R.T. (1999). Sounds of Our Times. Two Hundred Years of Acoustics. New York: AIP Press; Springer. Bigand, E. (1993). The influence of implicit harmony, rhythm and musical training on the abstraction of "tension-relaxation schemas" in tonal musical phrases. Contemporary Music Review 9(1&2): 123-137. _ (1997). Perceiving musical stability: The effect of tonal structure, rhythm, and musical expertise. Journal of Experimental Psychology, Human Perception, and Performance 23(3): 808-822. Bigand, E. and Parncutt, R. (1999). Perceiving musical tension in long chord sequences. Psychol. Res. 62(4): 237-254.

49. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University References (cont.) Bigand, E., Parncutt, R., and Lerdahl, F. (1996). “Perception of musical tension in short chord sequences: The influence of harmonic function, sensory dissonance, horizontal motion, and musical training,” Perception & Psychophysics 58: 125-141. Daniel, P. and Weber, R. (1997). “Psychoacoustical roughness: Implementation of an optimized model,” Acustica 83: 113-123. Fitz, K. and Haken, L. (2002). “On the use of time-frequency reassignment in additive sound modeling,” Journal of the Audio Engineering Society 50(11): 879-893. Fitz, K., Haken, L., Lefvert, S., Champion, C., and O'Donnell, M. (2003). "Cell-utes and flutter-tongued cats: Sound morphing using Loris and the Reassigned Bandwidth-Enhanced Model," Computer Music Journal 27(4): 44-65. Fulop, S.A. and Fitz, K. (2006a). "Algorithms for computing the time- corrected instantaneous frequency (reassigned) spectrogram, with applications," J. Acoust. Soc. Am. 119(1): 360-371.

50. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University References (cont.) Fulop, S.A. and Fitz, K. (2006b). "A spectrogram for the twenty-first century," Acoustics Today 2(3): 26-33. Fulop, S.A. and Fitz, K. (2007). "Separation of components from impulses in reassigned spectrograms," J. Acoust. Soc. Am. 121(3): 1510-1518. Helmholtz, H. L. F. 1885 [1954]. On the Sensations of Tone as a Physiological Basis for the Theory of Music. 2nd English edition. New York: Dover Publications. [Die Lehre von den Tonempfindungen, 1877. 4th German edition, trans. A. J. Ellis.] Kameoka, A. and Kuriyagawa, M. (1969a). "Consonance theory, part I: Consonance of dyads," J. Acoust. Soc. Am. 45(6): 1451-1459. Kameoka, A. and Kuriyagawa, M. (1969b). "Consonance theory, part II: Consonance of complex tones and its calculation method," J. Acoust. Soc. Am. 45(6): 1460-1469. Kendall, R.A. (2002). Music Experiment Development System (MEDS) 2001B for Windows. Los Angeles: University of California Los Angeles, Department of Ethnomusicology, Program in Systematic Musicology.

51. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University References (cont.) Krumhansl, C.L. (1996). Aperceptual analysis of Mozart’s Piano Sonata K. 282: Segmentation, tension, and musical ideas. Music Perception 13(3): 401-432. _ (1997). An Exploratory Study of Musical Emotions and Psychophysiology. Canadian Journal of Experimental Psychology 51(4): 336-353. _ (2002). Music: A Link Between Cognition and Emotion. Current Directions in Psychological Science 11(2): 45-50. MacCallum, J. and Einbond, A. (2008). Real-Time Analysis of Sensory Dissonance. Berlin / Heidelberg: Springer. Parncutt, R. (1989). Harmony: A Psychoacoustical Approach. Berlin: Springer-Verlag Petroviç, A. (1977). Ganga: A Form of Traditional Rural Singing in Yugoslavia. Unpublished Dissertation. Belfast: University of Belfast; Department of Social Anthropology. Plomp, R. and Levelt, W. J. M. (1965). "Tonal consonance and critical bandwidth," J. Acoust. Soc. Am. 38(4): 548-560.

52. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University References (cont.) Pressnitzer, D. and McAdams, S. (1997). "Influence of Phase Effects on Roughness Modeling," ICMC: International Computer Music Conference, Thessaloniki, Greece, September 1997. Pressnitzer, D. and McAdams, S. (1999a). "Two phase effects on roughness perception, ". J. Acoust. Soc. Am. 105(5): 2773-2782. Pressnitzer, D. and McAdams, S. (1999b). Summation of roughness across frequency regions. In: Dau T, Hohmann V, and Kollmeier B (Eds) Temporal processing in the auditory system: Psychophysics, physiology and models of hearing, pages 105-108. World Scientific Publishing, Singapore. Pressnitzer, D., McAdams, S., Winsberg, S., and Fineberg, J. (2000). " Perception of musical tension for non-tonal orchestral timbres and its relation to psychoacoustic roughness," Perception and Psychophysics, 62(1): 66-80. Racy, A.J. (1994). A dialectical perspective on musical instruments: the East Mediterranean Mijwiz. Ethnomusicology 38(1): 37-58. Sethares, W. A. (1998). Tuning, Timbre, Spectrum, Scale. London: Springer- Verlag.

53. Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University References (cont.) Terhardt, E. (1974). "On the perception of periodic sound fluctuations (roughness)," Acustica 30(4): 201-213. Vassilakis, P.N. (2001). Perceptual and Physical Properties of Amplitude Fluctuation and their Musical Significance. Doctoral Dissertation. University of California, Los Angeles. Vassilakis, P.N. (2005). Auditory roughness as means of musical expression. Selected Reports in Ethnomusicology 12: 119-144. Vassilakis, P.N., Kendall, R.A., and Racy, A.J. (2006). The worlds of music: culture-dependent emotional reactions to an improvisation on the mijwiz. Proceedings of the 51st SEM: 197. University of Hawai’i at Manoa. Vassilakis, P.N. and Fitz, K. (2007). SRA: A Web-based Research Tool for Spectral and Roughness Analysis of Sound Signals. Supported by a Northwest Academic Computing Consortium grant to J. Middleton, Eastern Washington University. [http://musicalgorithms.ewu.edu/algorithms/roughness.html] or [http://www.acousticslab.org/roughness]http://musicalgorithms.ewu.edu/algorithms/roughness.htmlhttp://www.acousticslab.org/roughness