1. Applied Psychoacoustics Lecture 3: Masking Jonas Braasch

2. From ATH to masked detection thresholds So far we have measured the absolute threshold of hearing (ATH) throughout the auditory frequency range for sinusoids. Now we would like to investigate how we detect sounds if other sounds are present as well. from tonmeister.ca after Zwicker & Fastl 1999

3. We have seen that a sine sound does not only excite the part of the basilar membrane that corresponds to its frequency but also other frequencies as well (traveling waves). The traveling wave moves from the base (high freqs.) to the apex (low freqs.) and declines after passing the resonance frequency. Therefore, we expect that a sound at a given frequency also affects the detection of a sound at another frequency. We will utilize this effect to the determine the shape of auditory filters. Preliminary thoughts

4. Method We now re-measure the threshold of hearing, but in this case we present to sinusoids to the listeners. One, the masker, is fixed in frequency (1 kHz) and level (60 dB SPL). The second one, the target, is varied in level as before. We measure at various frequencies the minimum sound pressure level at which the second tone is detected.

5. Absolute Threshold of Hearing Our absolute threshold of hearing (ATH) for a single tone now changes to … from tonmeister.ca after Zwicker & Fastl 1999

6. Masked Detection Threshold from tonmeister.ca after Zwicker & Fastl 1999 … to this one. We now speak of the masked detection threshold, with the sinusoids at 1 kHz, 60 dB SPL being the masker. Note that we find a steep slope just below 1 kHz, because in this range the basilar membrane is not much affected by the sound, while the slope is shallow for frequencies just above 1 kHz. steep curve for low freqs shallow slope for high freqs

7. Of course the masked detection threshold depends on the characteristics of the masker. In this graph, several thresholds curves are shown for various masker levels (20-100 dB SPL). 100 dB SPL is a very high value. I recommend NOT to go above 85 dB SPL if you want to repeat this measurement at home. from tonmeister.ca after Zwicker & Fastl 1999 Masked Detection Threshold

8. Auditory Filters Fletcher (1940) postulated that the auditory system behaves like a bank of pass-band filters with overlapping passband. Helmholtz (1865) already had similar ideas. Auditory filters can be measured in amplitude and phase as functions of frequency.

9. Measurement of auditory filters We will firstly restrict ourselves to the amplitude of auditory filters. We can use a similar paradigm to measure auditory filters as in our previous experiment: –present two sinusoids with same level and same frequency to the listener. The level was adjusted just above sensation level. –Next, we vary the frequency of the sinusoids in opposite direction. –The two sinusoids become inaudible at the point where they do not fall into one auditory filter anymore. In this case, the energy within each of the two auditory filters becomes to small to be detected.

10. Critical Bands: Zwicker (1961) Zwicker (1961) measured the critical bandwidth using two narrow-band maskers which masked a sine target at the center of the critical band. Zwicker recorded the detection threshold of the sine tone (varied in level) as a function of the frequency gap between both maskers. Unfortunately, the interference between the lower frequency noise masker and an the sine target lead to interference effects, and combination tones at different frequencies become audible, while the signal remains undetected. This leads to the abrupt decrease in the detection threshold at 0.3 kHz. (Fig.:Terhardt 1998)

11. Zwicker’s critical bands Linear range contradicts new findings

12. Zwicker’s critical band rate The graph shows the Critical band rate in Bark as a function of Frequency f. The equations were established to fit the data. Previous slide Errors between measured and predicted values (from equations below)

13. Zwicker’s critical bandwidth The graph shows the Critical band width in Bark as a function of Frequency f. The equation was established to fit the data. Errors between measured and predicted values (from equations below) Previous slide

14. Patterson ‘74: Measurement method Patterson (1974) used a broadband noise masker to avoid harmonicity to influence the results. from Patterson (1974) MASKING Hypothetical Shaded area: part of noise that is effectively masking the test tone

15. Patterson 74: Results

17. Off-Frequency Listening masker Auditory filter tone masker Auditory filter tone on-frequency listening off-frequency listening By placing the center frequency of of the Auditory Filter above the test- tone frequency, the signal-to-noise ratio between the tone and the masker can be increased. This way the test tone is easier targeted.

18. Avoiding off-Frequency Listening masker Auditory filter tone masker Auditory filter tone on-frequency listening off-frequency listening In this two masker case off-frequency listening does not pay off anymore. By shifting the auditory filter, the influence of one masker is reduced, while the influence of the 2. masker is increased. Overall the signal-to- noise ratio balance decreases. In this experiment, it is assumed that the auditory filter is symmetrical, which is a good-enough approximation. 2. masker ff ff

19. Noise Gap Masking To avoid off-frequency listening, Patterson (1976) measured the threshold of the sinusoidal signal as a function of the width of the spectral notch in the noise masker. The shaded areas shows the amount of noise passing through the auditory filter. (Fig.: Moore 2004)

20. Auditory filter shape Typical shape of an auditory filter as measured by Patterson (1976). The center frequency is 1 kHz. (Fig.: Moore 2004)

21. Auditory Filter non-linearity (Fig.: Moore 2004) This graph shows the non-linearity of the auditory filter. In the left graph the filter curves Are normalized to 0 dB. The filters were measured for several 2-kHz sine tones from 30 to 80 dBs. Note how the filter broadens toward low frequencies with increasing level. In the right graph the filters were not normalized. 30 dB 80 dB

22. Auditory Filter Bandwidths (Fig.: Moore 2004) Width of auditory filters measured with different techniques. The dashed curve shows the values of Zwicker (1961), the solid line the ERB N values which was measured using Patterson’s (1976) notched-noise method. Note the large deviations of Zwicker’s results at low frequencies, which are based on indirect measures.

23. ERB calculations Glasberg and Moore (1990) ERB N in Hz, f=frequency in kHz ERB N # in Hz, f=frequency in kHz

24. Psychophysical Tuning Curves The psychophysical tuning curves (PTC) were determined by measuring the masked detection thresholds for 6 sine tones which were presented 10 dB above sensation level (black circles). The masker was a sine tone as well which was varied in level (Data from Vogten, 1974). Fig.: Moore 2004

25. Each curve shows the tuning curve for a specific auditory nerve neuron of an anesthetized cat. The curves show the minimal sound pressure of a sinusoidal sound pressure to excite the neuron above a given threshold (also called frequency-threshold curves). The minimum of each curves show the CF of the neuron. Tuning curves of the auditory nerve from Palmer (1987) CFs Neurons tuned toward higher frequencies

26. Masking Patterns (Fig.: Moore 2004, data from Egan and Hake, 1950) Masking patterns (audiograms) for a narrow band of noise centered 410 Hz. The curves show the increase in threshold for a sinusoidal signal as a function of frequency. The number above each curve gives the SPL of the noise masker.

27. Excitation Patterns (Fig.: Moore 2004) Estimation of the excitation pattern from auditory filterbank data for a 1-kHz sinusoid. For each filter band the filter amplitude at the frequency of the test tone is determined (points a-e). Afterwards, these points are plotted at the center frequency of the corresponding filter, which represents the excitation pattern.

28. Excitation Patterns (Fig.: Moore 2004) The figure shows the excitation patterns for the 1-kHz sinusoid for various sound pressure levels from 20 to 90 dB in steps of 10 dB.

29. Temporal masking Temporal masking occurs because of the sluggishness of the basilar membrane, and time-dependent processing of the hair-cells and other neurons (refractory time). Simultaneous masking: masking induced by a simultaneous masker Forward masking: masking induced by a masker that precedes the test tone in time. Backward Masking: Masking that occurs for a target that precedes a masker Overshoot: Initial masked threshold increase during the onset phase of the masker.

30. Noise vs. tonal masker 410 Hz, 90 Hz bandwidth 400 Hz Egan and Hake (1950)

31. Applied Psychoacoustics Lecture 4: Loudness Jonas Braasch

32. Definition Loudness Loudness is the quality of a sound that is the primary psychological correlate of physical intensity. Loudness is also affected by parameters other than intensity, including: frequency bandwidth and duration.

33. Measuring loudness in phon At 1 kHz, the loudness in phon equals the sound pressure level in dB SPL. At all other frequencies the loudness the corresponding dB SPL value is determined by adjusting the level until the loudness is equally high to the reference value at 1kHz (so-called equal loudness curves or Fletcher-Munson curves). ref 1 kHz Target ? kHz time level Level adjusted until loudness matches ref

34. Equal loudness curve Detection threshold reftarget

35. Scales of Measurement nominalordinalintervalratio Non-numeric scaleScale with greater than, equal and less than attributes, but indeterminate intervals between adjacent scale values equal intervals between adjacent scale values, but no rationale zero point Scale has a rationale zero point e.g., color, gendere.g., rank order of horse race finalist Loudness in phon e.g., the difference between 1 and 2 is equal to the difference of 101 and 102 e.g., Temperature in Fahrenheit e.g., the ratio of 4 to 8 is equal to the ratio of 8 to 16. e.g., Temperature in Kelvin.

36. Typical goal of our measurement Determine the relationship between a physical scale (e.g., sound intensity) and the psychophysical correlate (e.g., loudness)

37. Measurement Methods method of adjustment or constant response –The subject is asked to adjust the stimulus to a fulfill certain task (e.g., adjust the stimulus to be twice as loud). method of constant stimuli –Report on given stimulus (e.g., to what extent it is louder or less loud then the previous stimulus).

38. Loudness in Sone Stevens proposed in 1936 to measure loudness on a ratio scale in a unit he called Sone. He defined 1 Sone to be 40 phons. The rest can be derived from measurements. Do you have an idea how he could have done it?

39. 1. Measure the Sone scale at 1 kHz e.g., by asking the subject to adjust the level of the sound stimulus (1-kHz sinusoidal tone) such that it is twice, 4 times … as loud as the Reference stimulus at 1 Sone (40 phons).

40. Relationship between Sones and Phons At 1-kHz this is also the psychometric function between Sone and dB SPL

41. 2. Measure the Sone scale at all other frequencies The easiest way is to follow the equal loudness contours. If they are labeled with the correct Sone value at 1 kHz, they are still valid.

42. 2 Sone 1 Sone 4 Sone 8 Sone 16 Sone 32 Sone 64 Sone Equal loudness curve

43. Zwicker Excitation Pattern

46. Zwicker loudness model N=  N’ m m=1 24 Overall loudness

47. Example 1: 1-KHz Sine tone

50. Example 200-Hz Sine Tone

52. Example: White Noise

53. White Noise

54. Pink Noise

56. Frequency weighting for dBA and dBC from: Salter, Acoustics, 1998

57. dB (A), roughly 35 phons dB (B) dB (C) Equal Loudness Contours

58. Level/Loudness Comparison 200 Hz sine 1 kHz sine Pink noise White noise dB SPL67 70 dB(C)67 6865 dB(A)56676667 Sone582219

59. Applied Psychoacoustics Lecture 4: Pitch & Timbre Perception Jonas Braasch

60. Euclid (330-275 BC) Some sounds are higher pitched, being composed of more frequent and more numerous motions

61. Contents Pitch perception –Pure Tones –Place and Rate Theory –Complex Tones Timbre

63. IntervalSemitonesFreq. Ratio Prime01:1 Minor second116:15 Major second29:8 Minor third36:5 Major third45:4 Perfect Fourth54:3 Augmented fourth Diminished fifth 645:32 64:45 Perfect Fifth73:2 Minor sixth88:5 Major sixth95:3 Minor seventh1016:9 Major seventh1115:8 Perfect Octave122:1

64. IntervalEqual temp.Just intonation Prime00 Minor second100112 Major second200204 Minor third300316 Major third400386 Perfect Fourth500498 Augmented fourth Diminished fifth 600590 610 Perfect Fifth700702 Minor sixth800814 Major sixth900884 Minor seventh1000996 Major seventh11001088 Perfect Octave1200 cents

65. Equal Temperament One semitone equals: 12 √2=1.0595=5.9463% One cent: 1200 √2=1.0006=0.059% Perfect fifth: 1.5=700 cent=50% Perfect Octave: 2=1200 cent=100%

66. Definition of Pitch Pitch is that attribute of auditory sensation in terms of which sounds may be ordered on a scale extending from low to high. Pitch depends mainly on the frequency content of the sound stimulus, but it also depends on the sound pressure and the waveform of the stimulus. ANSI standard 1994

67. Frequency JND’s Different symbols show different studies (Fig.:Terhardt 1998)

68. Frequency Difference Limens Wier et al., 1977

69. Frequency Difference Limens At low sound pressure levels (<10 dB SPL), the JND for pitch Increases. The hump at 800 Hz was not confirmed in follow-up studies At one 1-kHz the difference limens is about 3 cents (0.2%) At high and low frequencies, we are less sensitive to pitch (e.g., 0.5% at 200 Hz and 1% at 8 kHz. Melody recognition disappears for frequencies above 4-5 kHzMelody recognition disappears for frequencies above 4-5 kHz

70. DL compared to a semitone

71. Pure Tone Frequency Discrimination from Cheveigne, 2004

72. Effect of signal duration Duration (ms) d’ relative to 20 ms Large improvement in F0 discrimination with duration for unresolved harmonics (White and Plack, 1998):

73. Theories on Pitch Perception Place Theory –Pitch is determined by the location of the firing inner hair cell population on the basilar membrane Rate Theory –Pitch is determined by the rate code of the inner hair cells (phase locking)

74. Place Theory Excitation on Basilar membrane from: Hartmann, 1996

75. Place Theory Excitation on Basilar membrane Excitation on Basilar membrane for two sinusoids of same frequency f but 30 dB level difference from: Hartmann, 1996

76. Pitch shift for level variation according to Terhardt 1982 Pitch variation of a sinusoid as function of SPL

77. Autocorrelation Cross-Correlation Models  Y (  )= 1/(t 1 -t 0 ) Y(t)Y(t+  )  t=t 0 t1t1 Licklider (1951)

78. Rate model (Sinusoid analysis) (from: de Cheveigne, 2004)

79. Rate Pitch Model f=1/t log(f)=log(1/t)

80. 1-kHz sine tone FFT

81. 1-kHz harmonic complex with equally strong harmonics

82. 250-Hz harmonic complex with equally strong harmonics

83. 250-Hz harmonic complex with missing fundamental

84. 250-Hz harmonic complex with decreasing harmonic strength

85. Cochlear Implants Research on Cochlear Implant users suggest that our auditory system makes use of both the rate and place when determining the pitch. The analysis of the rate code is not possible for high frequencies

86. Cochlear Implants Illustration from "Functional Replacement of the Ear," by Gerald E. Leob, 1985

87. Cochlear Implants

88. Excitation (dB) ResolvedUnresolved Centre Frequency (Hz) Excitation pattern: Auditory filterbank: Frequency (Hz) Input spectrum: Level (dB) from Plack, Oxenham, 2002

89. Which harmonic determines pitch? from: Chris Darwin Time (t) -4 -3 -2 0 1 2 3 4 1 Period = 1/200 s = 5ms Frequency (Hz) Amplitude 200400600800 Harmonic spacing = 200 Hz Fundamental = 200 Hz Pressure

90. Pitch remains the same without fundamental (Licklider, 1956): from: Chris Darwin Time (t) -3 -2 0 1 2 3 1 Period = 1/200 s = 5ms Frequency (Hz) 200400600800 Harmonic spacing = 200 Hz Pressure Amplitude

91. Absolute Pitch "Passive" absolute pitch –Persons who are able to identify individual notes which they hear, –They can typically identify the key of a composition "Active" absolute pitch –Persons with active absolute pitch will be able to sing any given note when asked. –Usually, people with active absolute pitch will not only be able to identify a note, but recognize when that note is slightly sharp or flat. –1 in every 10,000 people in the US posses active absolute pitch possessors (1/20 in some other locations).

92. Motoric Absolute Pitch Persons who can reproduce an absolute reference tone to determine the pitch of other tones (e.g. professional singer knowing their range, persons who speak a tone language).

93. Literature William M. Hartmann (1996): Pitch, periodicity, and auditory organization, J. Acoust. Soc. Am. 100, 3491-3503. Alain de Cheveigne (2004) Pitch Perception models, in: Pitch (Plack, Oxenham, eds.), Springer, New York.