2. SPPA 4030 Speech Science Stephen M. Tasko Ph.D. CCC-SLP

3. Topic 1: The Speech Chain Learning Objectives Outline the general sequence of biological/physical events that occur from speech formulation to speech perception. Describe the different types of information content embedded within the speech signal. Know and describe the different branches of physics and biology used to inform basic mechanisms of speech production and perception.

4. The Speech Chain (Denes & Pinson, 1993)

5. What information is embedded in the speech signal? Phonetic information Affective information Personal information Transmittal information Diagnostic Information

6. Branches of science employed to understand speech communication Physics Acoustics Aerodynamics Kinematics Dynamics Biology Anatomy – Gross anatomy – Microscopic anatomy – Molecular biology – Neuroimaging Physiology – Electrophysiology

7. Physical Quantities Review An Independent Learning Activity Learning Objectives Distinguish between basic and derived units Distinguish between scalar and vector quantities Define a range of derived quantities with special emphasis on displacement, velocity, acceleration, force, pressure, intensity, resistance and their physical relationship

8. Assignment 1 See Assignments section of course website Due September 12, 2013

9. Topic 2: The Source-Filter Theory of Speech Production: An Introduction Learning Objectives Outline the key assumptions of the source filter theory of speech production Distinguish between the source signal, filter characteristics, and the output signal Use a range of examples to demonstrate understanding of the source filter theory Distinguish to role that different vocal tract structures play in speech sound generation and speech sound filtering

10. Producing Speech The vocal tract can be conceived as a set of interconnecting tubes and valves. Speech production is achieved through the systematic regulation of air pressures and flows within the vocal tract.

11. Source-Filter Theory of Speech Production The sounds we hear as speech is the product of a sound source that has undergone filtering by the vocal tract source and the filter may be considered to be independent of each other

12. Vocal tract is a tube that can vary its shape From Titze (1994)

13. Source Filter Theory Source (Phonation) Filter (Resonator) Speech (What We Hear) Input Spectrum Frequency Response Curve Output Spectrum

14. Same Source, Different Filter

15. Different Source, Same Filters White Noise

16. Different Source, Same Filters (Human) burp

17. Different Source, Same Filters (Human) snore

18. Different Source, Same Filters (Human) Lip buzz

19. Different Source, Same Filters (Human) ?

20. Different Source, Same Filters (Non-Human) sheep

21. Different Source, Same Filters (Non-Human) accordion

22. Different Source, Same Filters (Non-Human) If it quacks like a duck…

23. Source Filter Theory Applied: Alaryngeal Speech

24. Source-Filter Theory Applied: Esophageal Insufflation Test

25. Source-Filter Theory Applied: Tracheoesophageal (TE) Speech

26. Source Filter Theory Applied: The Talkbox

27. Source-Filter Theory Applied: The Talkbox http://www.youtube.com/watch?v=YS3gAVNlceg

28. Topic 3: A Brief Review of Physical Acoustics Learning Objectives Outline the physical processes underlying simple harmonic motion using the mass-spring model Describe the molecular basis of sound wave propagation Define the key characteristics of sinusoidal motion including – Amplitude: instantaneous, peak, peak-to-peak, root-mean-square (RMS), the decibel scale – Frequency/period including units of measure – Phase – Wavelength Briefly describe the relation between the sine wave and uniform circular motion Outline the relationship between the frequency and wavelength of a sound wave

29. Spring Mass Model Mass (inertia) – Newton’s first law of motion – Opposition to acceleration/deceleration Elasticity – Opposition to displacement – Rest position – Recoil force Friction http://phet.colorado.edu/en/simulation/mass-spring-lab

30. What is sound? It may be defined as the propagation of a pressure wave in space and time. Sound must propagate through a medium

31. Sound-conducting media Medium is composed of molecules Molecules have “wiggle room” Molecules exhibit random motion Molecules can exert pressure AB

32. Model of air molecule vibration (Time 1) Rest positions Air molecules sitting side by side

33. Model of air molecule vibration (Time 2)

34. Model of air molecule vibration (Time 3)

35. Model of air molecule vibration (Time 4)

36. Model of air molecule vibration (Time 5)

37. Model of air molecule vibration Time 1 2 3 4 5 Distance abcd

38. Wave action of molecular motion Time 1 2 3 4 5 Distance

39. Amplitude waveform Position Time

40. Amplitude waveform Amplitude Time

41. Where is pressure in this model? Time 1 2 3 4 5 Pressure measuring device at a specific location

42. Pressure waveform Time Sound Pressure Ambient Pressure + - 0

43. Measuring Sound Amplitude Frequency Phase Wavelength

44. Measuring Sound: Signal Amplitude Ways to measure it Instantaneous Peak Peak-to-peak Root mean square (RMS) Decibel –see later Time Sound Pressure + - 0

45. Measuring Sound: Signal Amplitude Root mean square (RMS)

46. What units do we use to measure signal amplitude? Pressure: Force/area Intensity = Power/area where power=work/time & work=Force*distance Intensity is proportionate to Pressure 2

47. Brief Review: The decibel scale decibel scale typically used to represent signal amplitude Many common measurement scales are absolute and linear However, the decibel scale is relative and logarithmic

48. Absolute vs. relative measurement Relative measures are a ratio of a measure to some reference Relative scales can be referenced to anything you want. decibel scale doesn’t measure amplitude (intensity or pressure) absolutely, but as a ratio of some reference value.

49. Typical reference values Intensity – 10 -12 watts/m 2 – Threshold for normal hearing at 1000 Hz Sound Pressure Level (SPL) – 20 micropascals

50. However… You can reference intensity/pressure to anything you want For example, Post therapy to pre therapy Sick people to healthy people Sound A to sound B

51. Linear vs. logarithmic Linear scale: 1,2,3… For example, the difference between 2 and 4 is the same as the difference between 8 and 10. We say these are additive

52. Linear vs. logarithmic Logarithmic scales are multiplicative Recall from high school math and hearing science 10 = 10 1 = 10 x 1 100 = 10 2 = 10 x 10 1000= 10 3 = 10 x 10 x 10 0.1 = 10 -1 = 1/10 x 1 Logarithmic scales use the exponents for the number scale log 10 10 = 1 log 10 100 = 2 log 10 1000=3 log 10 0.1 = -1

53. Logarithmic Scale base doesn’t have to be 10 In the natural sciences, the base is often 2.7… or e

54. Logarithmic Scale Why use such a complicated scale? – logarithmic scale squeezes a very wide range of magnitudes into a relatively compact scale – this is roughly how our hearing works in that a logarithmic scales matches our perception of loudness change

55. Combining the idea of logarithmic and relative… bel= log 10 (I m / I r ) I m –measured intensity I r – reference intensity A bel is pretty big, so we tend to use decibel where deci is 1/10. So 10 decibels makes one bel dB IL = 10log 10 (I m / I r )

56. Intensity vs. Pressure Intensity is difficult to measure. Pressure is easy to measure – a microphone is a pressure measuring device. Intensity is proportionate to Pressure 2

57. Extending the formula to pressure Using some logrithmic tricks, this translates our equation for the decibel to dB SPL = (2)(10)log 10 (P m / P r ) = 20log 10 (P m / P r )

58. Measuring Sound: Frequency/period Period (T): duration of a single cycle Frequency (F): rate that cycle repeats itself (1/T) Time Sound Pressure + - 0 Period (T)

59. Measuring Sound: Frequency/period Absolute measure – Cycles-per-second: Hertz (Hz) Relative measure – Octave (double or halving of frequency) – Semitones (12 semitones = 1 octave)

60. Phase: Uniform Circular Motion

61. Initiating a sound waves that differ only in phase A force is applied to molecule at frequency f and time t same force applied at frequency f at time t+a where a < the period of vibration

62. Spatial variation in pressure wave wavelength ( ) is the distance covering adjacent high and low pressure regions

63. Spatial variation in pressure wave Time 1 2 3 4 5

64. Spatial variation in pressure wave

65. Relation between frequency and wavelength =c/F where : wavelength F: is the frequency c: is sound speed in medium (35,000 cm/sec)

66. Learning Objectives Draw and describe time-domain and frequency-domain representation of sound Distinguish between simple and complex sound sounds with regard to physical characteristics and graphical representations Distinguish between periodic and aperiodic sounds with specific emphasis on terms such as fundamental frequency/period, harmonics, and overtones Distinguish between continuous and transient sounds Describe how waves sum, define Fourier's theorem and be able to describe the basics of Fourier analysis

67. Graphic representation of sound Time domain – Called a waveform – Amplitude plotted as a function of time Frequency domain – Called a spectrum – Amplitude spectrum amplitude vs. frequency – Phase spectrum phase vs. frequency – May be measured using a variety of “window” sizes

68. Same sound, different graphs Time domain Frequency domain From Hillenbrand

69. Classification of sounds Number of frequency components – Simple – Complex Relationship of frequency components – Periodic – Aperiodic Duration – Continuous – Transient

70. Simple periodic sound Simple: one frequency component Periodic: repeating pattern Completely characterized by – amplitude – period (frequency) – phase Other names: sinusoid, simple harmonic motion, pure tone

71. Simple periodic sound: Graphic appearance From Hillenbrand

72. Complex periodic sounds Complex: > one frequency component Periodic: repeating pattern Continuous Frequencies components have a special relation – Lowest frequency: fundamental frequency Symbol: f o Frequency component with longest period – Higher frequency components: harmonics integer (whole number) multiples of the f o

73. Complex periodic sounds: Graphic appearance Time domain: – repeating pattern of pressure change – within the cycle, things look complex Frequency domain: – spectral peaks at evenly spaced frequency intervals Auditory impression: sounds ‘musical’

74. Complex periodic sounds: Graphic appearance From Hillenbrand

75. Glottal Source Time Frequency Amplitude

76. Amplitude vs. Phase Spectrum Amplitude spectrum: different Phase spectrum: same From Hillenbrand

77. Amplitude vs. Phase Spectrum Amplitude spectrum: same Phase spectrum: different From Hillenbrand

78. (Complex) Aperiodic sounds Complex: > one frequency component Aperiodic: Does not repeat itself Frequency components are not systematically related May be – Continuous – Transient

79. Aperiodic sounds: Graphic appearance Time domain: – no repeating pattern of pressure change Frequency domain: – the spectrum is dense – No “picket fence” Auditory impression: sounds ‘noisy’

80. Aperiodic sounds: Graphic appearance From Hillenbrand

81. Analysis of complex waves Waves can be summed Complex waves are the sum of simple waves Fourier: French Mathematician: – Any complex waveform may be formed by summing sinusoids of various frequency, amplitude and phase Fourier Analysis – Provides a unique (only one) solution for a given sound signal – Is reflected in the amplitude and phase spectrum of the signal – Reveals the building blocks of complex waves, which are sinusoids

82. Learning Objectives Draw and differentiate the waveform and the waveform envelope Draw and differentiate the amplitude spectrum, the phase spectrum and the spectrum envelope Differential between short-term spectra and long-term average spectra.

83. The “envelope” of a sound wave Waveform envelope: – imaginary smooth line that follows the peak of the amplitude of a sound pressure waveform Spectrum envelope: – Imaginary smooth line drawn on top of the amplitude spectrum

84. Waveform envelope From Hillenbrand

85. Waveform envelope Time

86. Spectrum envelope Frequency Amplitude

87. Thought Question Can an aperiodic and complex periodic sound have identical spectrum envelopes?

88. Amplitude Spectrum: Window Size “short-term” vs. “long-term average” amplitude spectrum

89. “Instantaneous” Amplitude Spectra

90. (Long Term) Average Amplitude Spectrum

92. Learning Objectives Describe how the amplitude spectrum and the spectrogram are related. Identify the axis units of the spectrogram. Provide some advantages of the spectrogram over the amplitude spectrum. Distinguish between a wide band and narrow band spectrogram and outline the different information each provides. Distinguish between a harmonic and a formant on a spectrogram. Be able to draw stylized (highly simplified) spectrograms based on spectra and spectrum envelopes.

93. The Spectrogram

94. Rotate 90 degrees F A F A Building a spectrogram

95. Rotate it so that The amplitude is Coming out of the page F A This is really narrow because it is a slice in time F Time Building a spectrogram

96. Dark bands = amplitude Peaks Time Frequency Building a spectrogram

97. Two main types of spectrograms Narrow-band spectrograms – Akin to amplitude spectrums “lined up” – Frequency resolution is really sharp Wide-band spectrograms – Akin to spectrum envelopes “lined up” – Frequency resolution not so sharp

98. Highlights harmonic structure Highlights spectrum envelope Wide vs. Narrow Band Spectrograms

99. Learning Objectives Define an acoustic filter Draw and label a frequency response curve Draw and differentiate different types of acoustic filters Define terms such as cutoff frequency, center frequency, roll off rate, gain, and bandwidth Define and draw a basic filter system and relate that to the source-filter theory of speech production

100. What is an “Acoustic” Filter holds back (attenuates) certain sounds and lets other sounds through - selective.

101. Why might we be interested in filters? Human vocal tract acts like a frequency selective acoustic filter Human auditory system behaves as a frequency selective filter helps us understand how speech is produced and perceived.

102. Kinds of frequency selective filters Low-pass filters – Lets low frequencies “pass through” and attenuates high frequencies High-pass filters – Lets high frequencies “pass through” and attenuates low frequencies Band-pass filters – Lets a particular frequency range “pass through” and attenuates other frequencies

103. Low Pass Filters Frequency lowhigh Gain + -

104. High Pass Filters Frequency lowhigh Gain + -

105. Band Pass Filter Frequency lowhigh Gain + -

106. Frequency Response Curve (FRC) Frequency lowhigh Gain + - Center frequency lower cutoff frequency upper cutoff frequency passband 3 dB

107. Operation of a filter on a signal NOTE: Amplitude spectrum describes a sound Frequency response curve describes a filter

108. Learning Objectives Define resonance, free and forced vibration Describe how the pendulum and spring mass models can help explain resonance. Outline how mass and stiffness influences the resonant frequency of a mass spring system. Outline how acoustic resonators behave like acoustic filters. Calculate resonant frequencies of a uniform tube based on its physical dimensions. Describe how the wavelength of the sound determines the resonant frequency of tube.

109. Free vibration objects tend to vibrate at a characteristic or resonant frequency (RF)

110. Forced vibration A vibrating system can force a nearby system into vibration The efficiency with which this is accomplished is related to the similarity in the resonant frequency (RF) of the two systems

111. Forced vibration If the RF of the two systems are the same, the amplitude of forced vibration will be large If the RF of the two systems are quite different, the amplitude of forced vibration will be small or nonexistent

112. Resonance refers to The tendency for an object to vibrate at a particular frequency or frequencies. The ability of a vibrating system to force another system into vibration.

113. Back to the mass spring model Vibratory frequency of the mass spring determined by – Mass – Stiffness of the spring http://phet.colorado.edu/en/simulation/mass-spring-lab

114. Acoustic Resonance Ideas from mechanical resonance also applies to acoustic systems Acoustic chambers will transmit sound frequencies with more or less efficiency, depending upon the physical characteristics Therefore, they act as filters, passing through (and even amplifying) some frequencies and attentuating others.

115. Acoustic Resonance And since they act as filters, they have most of the same features of a filter, even though we might use different names. Center frequency is often termed the resonant frequency. Frequency response curve often termed the resonance curve.

116. Helmholtz Resonator

117. Actions of a Helmholtz Resonator

118. Stephen M. Tasko Other Acoustic Resonators: Tube Resonators Uniform tubes: Factors that influence resonance – Length. – Cross-sectional area along its length. – Whether it is closed at either or both ends.

119. Uniform tube, closed at one end

121. Uniform tube closed at one end First resonance or formant F 1 = c/4 l Where c=speed of sound (35,000 cm/sec) l = length of the tube males ~ 17.5 cm females ~ 14 cm Higher resonant/formant frequencies are odd multiples of F 1 For example, F 1 = (c/4 l )*1 F 2 = (c/4 l )*3 F 3 = (c/4 l )*5 Stephen M. Tasko

122. Comparing Helmholtz and tube resonators

123. Resonator Features Sharply tuned Broadly tuned

124. Resonator Features An example of the resonance characteristics of the human vocal tract Frequency Gain