Lecture 9 Perception of combination tones Modes of hearing                                                                                   Lecture 9 Perception of combination tones Modes of hearing Consonance & dissonance Theories of pitch perception Instructor: David Kirkby (dkirkby@uci.edu)

Miscellaneous Pick up Problem Set #3 today. The average was 66%. The midterm is in class on Thursday (Oct 31). Remember to bring a ScanTron 8000 form and a #2 pencil. Physics of Music, Lecture 9, D. Kirkby

Review of Lecture 8 We examined the perception of pitch for a pure tone. We described the unusual ability (absolute or perfect pitch) to identify or reproduce a tone without using an external reference tone. We contrasted the frequency analysis capabilities of vision (absolute) and hearing (relative). This completes our survey of the perception of pure tones. Now we are ready to tackle the more difficult (and more interesting) questions surrounding the perception of sounds consisting of one or more pure tones. Physics of Music, Lecture 9, D. Kirkby

Pairs of Pure Tones What do a pair of pure tones sound like? Listen to these samples in which each note of the pair are played separately first, and then together: 15Hz First note = Middle C (256Hz) 1Hz 5Hz Physics of Music, Lecture 9, D. Kirkby

Frequency Regimes At large frequency differences, the sound of the combined tones is smooth and the pitches of the two contributing tones can be resolved. At small frequency differences (less than about 15 Hz) the combined tones result in a pulsing sound, with slower beats as the frequency difference gets smaller. Only a single pitch can be resolved. At intermediate frequency differences, the sound is rough and only a single pitch can be resolved. Are the beats and roughness due to human auditory processing, or physics, or both? Physics of Music, Lecture 9, D. Kirkby

Beats The phenomenon of pulsing loudness when two pure tones of almost the same frequency are played together is called beats. The effect is due simply to the Principle of Superposition: Pure tone 1 Pure tone 2 Superposition 1+2 Physics of Music, Lecture 9, D. Kirkby

Beat Frequency The frequency of the beats is equal to the difference of frequencies of the two tones, Df = |f1 - f2| and the perceived tone is the average (f1 + f2)/2. So the combination of 250 Hz and 252 Hz tones pulses twice per second and has a perceived pitch of 251 Hz. The presence of beats allows you to detect frequency differences much smaller than what you can detect when two tones are played consecutively (the JND which is 2-3 Hz near middle C). This is a very useful technique for getting musical instruments in tune with each other, but also means that the audience are more sensitive to mis-tunings when two instruments play the same note together. Physics of Music, Lecture 9, D. Kirkby

When the beat frequency is faster than about 15 Hz, you can no longer resolve individual beats (remember that you average the loudness of a pure tone over about 1/5 s) but they are still there: Perfect 4th (4:3) Octave (2:1) Physics of Music, Lecture 9, D. Kirkby

As the frequency difference increases above 15 Hz, the sound is initially rough and you cannot resolve the individual frequencies. Once the two frequencies are separated by about one critical band, the sound becomes smooth and you can resolve the individual frequencies. Listen for the transitions between these 3 regimes in this sample in which the lower note is held fixed at middle C and the upper note increases by semitones over an octave. beats smooth rough Physics of Music, Lecture 9, D. Kirkby

Masking within the Critical Band When two pure tones generate vibrations in the same region of the basilar membrane, a louder tone can mask a quieter tone. Listen to these sample in which the frequency of a quiet tone decreases in steps, crossing through the critical band centered on a loud tone. +10 +7 +3 +2 -2 -3 -10 -7 frequency difference of quiet tone (semitones) Loud tone = C5 (~512 Hz) +5 +14 -5 -14 Physics of Music, Lecture 9, D. Kirkby

Combination Tones When two pure tones are heard together, additional “combination” tones of different frequencies are sometimes also heard. In this case, you are hearing something that is not really there! Listen to this sample in which one tone is held fixed and the other increases smoothly in frequency. frequency time Physics of Music, Lecture 9, D. Kirkby

If you are only hearing the two generated tones, then you should only hear a single ascending high-pitched ramp. Listen for additional combination tones as descending ramps or lower-pitched ascending ramps. frequency time Physics of Music, Lecture 9, D. Kirkby

There are many possible combination tones, each of which follows a straight line given by n1 f1 + n2 f2 where n1,n2 are integers (possibly negative). But only three are usually heard, and only in a limited range of frequency differences. A useful trick to help identify the different combination tones is to add a quiet “probe” tone whose frequency is slightly different (eg, by 4 Hz) than the tone you want to pick out. This has the effect of causing beats to be heard whenever the target tone is audible. probe target Physics of Music, Lecture 9, D. Kirkby

Listen to the samples below in which a different probe tone is used to pick out the three strongest target combination tones: frequency f2-f1 3f1-2f2 2f1-f2 time Physics of Music, Lecture 9, D. Kirkby

Perfect linear response: y12 = y1 + y2 What causes these phantom tones to be heard? Is this due to physics or the way we process auditory signals in the brain? The answer is physics: these tones are due to the fact that the ear is not an ideal linear system and has some small non-linearities in its response. Perfect linear response: y12 = y1 + y2 Non-linear response: y12 = y1 + y2 + y1y2 Extra signal from non-linearity: (has frequencies f1+f2, |f1-f2|) Physics of Music, Lecture 9, D. Kirkby

Combinations of Many Tones When many pure tones are combined, there are essentially 3 different modes for hearing, as illustrated by the samples below. These 3 modes can be represented as the corners of a triangle. Pitch with timbre Chord (many pitches) Sound w/o pitch Physics of Music, Lecture 9, D. Kirkby

What is Timbre? Timbre is the term used for the tone quality or “color” of a complex sound. A guitar and a saxophone can play the same note at same intensity. In this case, they will have the same perceived pitch and loudness. But they are still easy to distinguish: this is because their sounds have different timbres. Timbre is fundamentally different from pitch and loudness since its value can not be quantified on a single scale. Physics of Music, Lecture 9, D. Kirkby

Timbre of Musical Sounds Listen to these examples of a complex musical sound being built up progressive from many individual pure tones: This is not the whole story since the timbre of a musical instrument is dynamic and depends on many factors. For example, timbre often changes dramatically with pitch as illustrated in this example: Timbre also changes over the course of a single note, as illustrated in these 3 examples: Physics of Music, Lecture 9, D. Kirkby

Synthetic vs Analytic Hearing Some combinations of pure tones can be heard either as multiple pitches (a chord) or else as a single pitch with timbre, depending on the context and the cues available. When you hear such a sound as individual notes (a chord), you are listening analytically. When you hear such a sound as a single note with a complex timbre, you are listening synthetically (or holistically). In this example, you are forced to hear certain tones analytically that you would normally hear synthetically. Physics of Music, Lecture 9, D. Kirkby

Harmonic vs Inharmonic Timbre When a complex sound is heard synthetically, it may or may not have a definite perceived pitch. The key for pitch perception of a complex sound is that the pure tones contributing to the sound be close enough to a harmonic series. Most musical instruments satisfy this requirement and produce a combination of complex tones (due to their many resonances) that gives an impression of a single note being played with a complex timber. Percussion instruments also have many resonances, but they are usually inharmonic and do not combine to give a definite sense of pitch. Physics of Music, Lecture 9, D. Kirkby

Compare these samples in which some music is performed first by an “instrument” with harmonic overtones: Next the same music is performed but the instrument’s overtones are stretched so that an “octave” becomes a frequency ratio of 2.1 : 1 instead of 2 : 1 Physics of Music, Lecture 9, D. Kirkby

Overview This diagram gives an overview of the 3 modes in which we perceive complex combinations of pure tones: Pitch with timbre Sound w/o pitch Chord (many pitches) analytic synthetic harmonic inharmonic Physics of Music, Lecture 9, D. Kirkby

Pitch of Complex Sounds When you hear a complex sound synthetically and find that it has a definite pitch and timbre, what determines the pitch you hear? In the case of a harmonic series, you hear a pitch corresponding to the frequency of the fundamental (1st harmonic). Listen to this sample of a complex sound consisting of 10 harmonics, when successively more harmonics are removed: The persistence of the fundamental in your pitch perception, even when it is no longer physically present, is known as virtual pitch. Physics of Music, Lecture 9, D. Kirkby

Circular Pitch Illusion It is possible to create artificial sounds whose timbre changes continuously in a way that defeats our pitch perception mechanisms. Here is an example of a sound whose pitch appears to rise continuously. This is an example of an auditory illusion, analogous to the famous Escher optical illusion of a staircase rising forever. Physics of Music, Lecture 9, D. Kirkby

Consonance and Dissonance We use the term consonant to describe two sounds that are pleasing and relaxing to hear together. Two sounds that cause tension in the listener when heard together are said to be dissonant. Are these purely aesthetic judgments, or is there some physical basis to consonance and dissonance? Most people agree on which sounds are most consonant, and we can predict how consonant or dissonant two sounds will be when played together using a physical model. So there is a physical basis to consonance and dissonance. Physics of Music, Lecture 9, D. Kirkby

Two pure tones are most dissonant when they differ in frequency by about 1/4 of a critical band (about 25 Hz or two semitones near middle C): this is too fast to hear audible beats, but slow enough to cause a rough sound. beats smooth rough Middle C (~256 Hz) critical band (~100 Hz) Physics of Music, Lecture 9, D. Kirkby

Pairs of complex sounds with timbre (i. e Pairs of complex sounds with timbre (i.e., two different notes played at the same time by real musical instruments) are consonant if each overtone of one complex sound is consonant with all pure tones of the other complex sound. Dissonance is due to pairs of overtones whose frequencies are close enough to sound rough, but not so close enough to produce audible beats. Two musical sounds can be dissonant even when their fundamentals are well separated in frequency: this occurs when one or more pairs of overtones are close in frequency. Physics of Music, Lecture 9, D. Kirkby

Example: the augmented fourth (diminished fifth) interval is frequently used for horror movies because of the tension it creates, but the fundamentals are well-separated in frequency. In this case, the dissonance is due to the roughness between the 2nd and 3rd harmonics of the two notes: Physics of Music, Lecture 9, D. Kirkby

In general, a (linear) plot of the dominant pure-tone frequencies that make up two complex sounds is an easy way to identify any dissonance: Dissonant overtones frequency More dissonant overtone pairs make a more dissonant total effect. Most musical sounds have their dominant contributions at harmonic frequencies. Which combinations of notes should sound the most consonant in this case? Physics of Music, Lecture 9, D. Kirkby

Smaller gap, but still consonant Notes whose fundamentals are an octave (2:1) apart cannot have any dissonant overtones! frequency Notes whose fundamentals are a fifth (3:2) apart are also very consonant: Smaller gap, but still consonant frequency Physics of Music, Lecture 9, D. Kirkby

Most people, when asked to make an aesthetic judgment, would order the harmonic frequency ratios in consonance as: Octave (2:1) Perfect fifth (3:2) Perfect fourth (4:3) Major sixth (5:3) Major third (5:4) Minor sixth (8:5) Minor third (6:5) becoming less consonant This ordering matches what we predict based on the frequency separation of overtones. Physics of Music, Lecture 9, D. Kirkby

Theories of Pitch Perception The signal sent to the brain from the basilar membrane provide two different types of cues about the membrane’s vibrations: Position of vibrations along membrane encodes wavelength. Timing between electrical pulses generated by hair cells encodes period (1/frequency). Physics of Music, Lecture 9, D. Kirkby

These two signals are usually complementary, although it is possible to create artificial sounds in which the apparent frequency and wavelength are not consistent. There are two extreme theories of pitch perception: Place theory: assumes that only wavelength cues are used to determine the perceived pitch. Periodicity theory: assumes that only period cues are used to determined the perceived pitch. Physics of Music, Lecture 9, D. Kirkby

Neither of these theories can explain the results of all tests Neither of these theories can explain the results of all tests. For example, the place theory cannot explain virtual pitch, synthetic listening, or why the just-noticeable-difference (JND) is so much smaller than the critical band. The periodicity theory cannot explain why pitch perception relies more on low-frequency overtones than high-frequency ones. The reality is somewhere in between these extremes: the brain combines both wavelength and period cues to arrive at a final perceived pitch for a complex tone. Physics of Music, Lecture 9, D. Kirkby

Auditory Processing in the Brain Are the signals from the left and right ears combined in the brain before deciding on the pitch, or does the brain assign a separate pitch for each ear? We can get some insight into how the brain combines the signals from both ears using samples in which both ears hear different sounds played through headphones. These two samples will not work in class but you can try them later on headphones: Binaural beats: you can hear beats between you left and right ears, suggesting that your brain superimposes these signals. Cerebral dominance: are you left- or right-eared? Physics of Music, Lecture 9, D. Kirkby

Vision Analogy Combinations of two or more pure (spectral) colors are always perceived synthetically: as a (non-spectral) color. Spectral colors lie on the border of this curve. Combination colors fill the curve’s interior. Physics of Music, Lecture 9, D. Kirkby

There is no analog of analytic hearing for vision, in which you can perceive the individual spectral frequencies contributing to a color. There is also no analog of the harmonic vs inharmonic distinction for colors. Vision perception is based on just 3 critical bands of fixed frequencies, compared with the ~24 critical bands of variable frequencies that cover the auditory range. Vision performs a much cruder sampling of frequencies than hearing, but more than compensates with very sophisticated sampling of different points in space. Physics of Music, Lecture 9, D. Kirkby

Summary Two pure tones playing together will be heard in one of 3 different ways (beats, rough, smooth) depending on their frequency difference. We saw an example of not hearing something that is really there: masking, due to critical bands of vibration and two examples of hearing something that is not really there: Combination tones (due to nonlinear ear response) Virtual pitch (due to pattern matching in the brain) Physics of Music, Lecture 9, D. Kirkby

A complex tone made up of many pure tones can by heard in one of 3 ways (chord, pitch with timbre, un-pitched sound) depending on whether you are hearing analytically or synthetically, and the spacing of the tone frequencies. Complex combinations of spectral colors are always seen synthetically (there are so visual chords). The consonance or dissonance of two complex musical sounds heard together can be understood in terms of the spacing of their overtones. Pitch perception combines wavelength and period cues from the basilar membrane. Physics of Music, Lecture 9, D. Kirkby