MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University SRA: An online research tool for spectral and roughness analysis of sound signals Pantelis N. Vassilakis - DePaul University Chicago, USA MERLOT 2007 New Orleans

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University 1.Auditory roughness Concept & Models 2.Spectral Analysis New versus old methods 3.SRA a. Outline b. Examples of use At A Glance

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Auditory Roughness Auditory Roughness: Harsh, raspy sound quality of narrow harmonic intervals. An acoustic/sensory dimension of dissonance. One of the perceptual manifestations of interference, expressed as a function of a complex signal’s spectral distribution; a dimension of timbre _ Adding two sine signals with frequencies f 1 and f 2 results in a complex signal whose amplitude fluctuates between a minimum and a maximum value at a rate equal to |f 1 -f 2 |. _ Amplitude fluctuation rate: a) < ~15 fluctuations/second → Beating b) between ~15 and ~ fluctuations/second → Roughness c) > ~ 150 fluctuations/second → combination tones, envelope pitch, etc..

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Helmholtz (1885) Plomp & Levelt (1965) Kameoka & Kyriyagawa (1969a,b) Hutchinson & Knopoff (1978) Daniel & Weber (1997) Sethares (1998) Leman (2000) Pressnitzer, McAdams & Colleagues (1997, 1999a, 1999b, 2000) (auditory periphery mechanisms models) Previous roughness calculation models

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University b 1 = 3.5 b 2 = 5.75 x* = 0.24 s 1 = s 2 = Roughness, frequency separation, and register

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University General assumption: Roughness is proportional to A 1 * A 2 Roughness and amplitude Previous experimental studies: von Béckésy (1960) Terhardt (1974)

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Amplitude modulation depth versus degree of amplitude fluctuation

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Proposed Roughness Calculation Model Roughness estimation model (sine pairs): R = X * Y * Z (Vassilakis, 2001, 2005) X = (Amin*Amax) 0.1 Dependence of R on the absolute amplitude of the sines (Terhardt, 1974; Vassilakis, 2001) Y = 0.5 [2Amin / (Amin+Amax )] 3.11 Dependence of R on the relative amplitudes of the sines (von Béckésy, 1960; Terhard, 1974; Vassilakis, 2001) Z = e -b1s(fmax - fmin) – e -b2s(fmax - fmin) [b1 = 3.5; b2 = 5.75; s = 0.24/(s1fmin + s2); s1 = ; s2 = 18.96] Dependence of R on relative (frequency difference of the sines) and absolute (frequency of the lower sine) frequencies of the added sines (Kameoka & Kuriyagawa, 1969a&b; Plomp & Levelt, 1965; Sethares, 1998)

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Roughness, phase, and signal envelope asymmetry Pressnitzer and McAdams (1999a)

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Comparison of 3 roughness calculation models

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Comparison of 3 roughness calculation models

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Drawbacks of traditional FFT Spectral Analysis Frequency and time values returned are forced to fit onto the time- frequency grid defined by the analysis window Frequency/temporal “smearing” and uncertainty on precise energy values

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Frequency analysis and spectral peak picking STFT algorithm based on the Reassigned Bandwidth-Enhanced Additive Model Spectral Analysis Dual STFT, fine-tuning spectral analysis results _ Frequency: time derivative of the argument (phase) of the complex analytic signal representing a given frequency bin _ Time: frequency derivative of the STFT phase, defining the local group delay (correction that pinpoints the precise excitation time) Theory: Developed by Kodera et al. (1976) Expressed mathematically by Auger & Flandrin (1995) Implemented to sound spectral analysis by Fulop & Fitz (2006a,b; 2007)

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University SRA online SRA online

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Dissonance & Orchestration

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Dissonance & Orchestration

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University Roughness Profile

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University r=0.422 Roughness & Tension Profiles

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University r=0.068 r=0.422 Roughness & Tension Profiles

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University References Auger, F. and Flandrin, P. (1995). "Improving the readability of time frequency and time scale representations by the reassignment method," IEEE Transactions on Signal Processing 43: von Békésy, G. (1960). Experiments in Hearing. New York: Acoustical Society of America Press (1989). Daniel, P. and Weber, R. (1997). "Psychoacoustical roughness: Implementation of an optimized model," Acustica 83: Fitz, K. and Haken, L. (2002). "On the use of time-frequency reassignment in additive sound modeling," Journal of the Audio Engineering Society 50(11): Fitz, K., Haken, L., Lefvert, S., Champion, C., and O'Donnell, M. (2003). "Cell-utes and flutter-tongued cats: Sound morphing using Loris and the Reassigned Bandwidth-Enhanced Model," Computer Music Journal 27(4): Fulop, S. A. and Fitz, K. (2006a). "Algorithms for computing the time corrected instantaneous frequency (reassigned) spectograms with applications," J. Acoust. Soc. Am. 119(1): Fulop, S. A. and Fitz, K. (2006b). "A spectrogram for the twenty-first century," Acoustics Today 2(3): Fulop, S.A. and Fitz, K. (2007). "Separation of components from impulses in reassigned spectrograms," J. Acoust. Soc. Am. 121(3): Helmholtz, H. L. F [1954]. On the Sensations of Tone as a Physiological Basis for the Theory of Music. 2nd English edition. New York: Dover Publications. [Die Lehre von den Tonempfindungen, th German edition, trans. A. J. Ellis.] Kameoka, A. and Kuriyagawa, M. (1969a). "Consonance theory, part I: Consonance of dyads," J. Acoust. Soc. Am. 45(6): Kameoka, A. and Kuriyagawa, M. (1969b). "Consonance theory, part II: Consonance of complex tones and its calculation method," J. Acoust. Soc. Am. 45(6): Kendall, R. A. (2002). Music Experiment Development System (MEDS) 2001B for Windows. Los Angeles: University of California Los Angeles, Department of Ethnomusicology, Program in Systematic Musicology. Plomp, R. and Levelt, W. J. M. (1965). "Tonal consonance and critical bandwidth," J. Acoust. Soc. Am. 38(4): Pressnitzer, D. and McAdams, S. (1997). "Influence of Phase Effects on Roughness Modeling," ICMC: International Computer Music Conference, Thessaloniki, Greece, September 1997 Pressnitzer, D. and McAdams, S. (1999a). "Two phase effects on roughness perception, ". J. Acoust. Soc. Am. 105(5): Pressnitzer, D. and McAdams, S. (1999b). Summation of roughness across frequency regions. In: Dau T, Hohmann V, and Kollmeier B (Eds) Temporal processing in the auditory system: Psychophysics, physiology and models of hearing, pages World Scientific Publishing, Singapore. Pressnitzer, D., McAdams, S., Winsberg, S., and Fineberg, J. (2000). " Perception of musical tension for non-tonal orchestral timbres and its relation to psychoacoustic roughness," Perception and Psychophysics, 62(1): Sethares, W. A. (1998). Tuning, Timbre, Spectrum, Scale. London: Springer-Verlag. Terhardt, E. (1974). "On the perception of periodic sound fluctuations (roughness)," Acustica 30(4): Vassilakis, P. N. (2001). Perceptual and Physical Properties of Amplitude Fluctuation and their Musical Significance. Doctoral Dissertation. Los Angeles: University of California, Los Angeles; Systematic Musicology. Vassilakis P. N. (2005). "Auditory roughness as a means of musical expression," Selected Reports in Ethnomusicology 12 (Perspectives in Systematic Musicology):

MERLOT 2007 – SRA: An online research tool for spectral and roughness analysis of sound signals - Pantelis N Vassilakis - DePaul University