1 Speech Science COMD 6305 Weds/Fri 1:00pm - 2:15pm William F. Katz, Ph.D.UTD
2 Chapter 1. The Nature of Sound INCLUDINGDefinition of Speech ScienceBasic physics conceptsOverview of soundResonance
3 What is ‘speech science’? “Speech Science is the experimental study of speech communication, involving speech production and speech perception as well as the analysis and processing of the acoustic speech signal.”Speech Science asks questions like:How is speech planned and executed by the vocal system?How do the acoustic properties of sounds relate to their articulation?How and why do speech sounds vary from one context to another?How do listeners recover the linguistic code from auditory sensations?How do infants learn to produce and perceive speech?How and why do speech sounds vary between speakers?How and why do speech sounds vary across speaking styles or emotions?(-M. Huckvale, UCL)
4 Physics PrinciplesEnergy Ability to do work* *(force to move a mass some distance in some amount of time) Common Units: Mass = kilograms (kg) Distance = meters (m) Time = seconds (sec)Energy: ability to do workForce required to move some mass some distance in some amount of timeMatter is made of particles which have massMass and spacing of articles determines density
5 Physics Principles - cont’ Opposing Forces- Affect objects in motionFrictionInertiaElasticityLimit the motion of objects3 types!a) friction: or resistanceOpposes movementRubbing hands or eraser across table
6 FrictionImpedes or opposes movement(not much friction here….)
7 In order to vibrate, an object must possess InertiaElasticity*Any object that can vibrate can produce sound
8 Inertia“Tendency for mass at rest to remain at mass” or “Object in motion to remain in motion” (Note: Amount of inertia is related to mass)b) inertia: tendency of mass at rest to remain at restObject in motion remain in motionAmount of inertia related to massIncreased mass needs increased force to change velocity
9 Elasticity: Restoring force Ability of a substance to recover its original shape and size after distortionAll solids are elastic; gases also behave as if elasticElasticity: ability to resist changing shapeForce applied to an object will deform it to some degreeStiff spring is very elastic, DOESN’T WANT TO MOVESo it easily returns to normalReturning to initial form by action of restoring forceStiffness: often used as measure of one of the resistive forces that opposes the setting into motion of an objectElasticity: resists compression
10 Hooke’s law – describes elasticity - “restoring force is proportional to the distance of the displacement and acts in the opposite direction”(if not stretched beyond its elastic limit)
11 Density (= mass per unit of volume) Lower Density Higher Density
12 Pressure Pressure = Pa Pa = N/m2, or dyne (older literature) N = kg x meters2(where N = Newton, a measure of force)See later slide for other measures of pressure!Pressure: force required to accelerate a kg of weight 1 meter per second, per second“per second, per second” = accelerationHow rate changes over time
13 Measurement of air pressure Dynes* per sq centimeter dyne/cm2Pounds per square inch psiMicrobar barPascal PaCentimeters of water cm H2OMillimeters of mercury mm Hg*The force required to accelerate a mass of one gram at a rate of one cm/sec2
14 Air pressure can also be measured in cm of H20 or mm of Mercury (Hg)
15 Two Units of Measure (pressure) dynes/cm2 “microbar” (CGS system)N/m2 “Pascal”* (MKS system)*(For speech, micropascal is more commonly used)Meters/Kilograms/SecsCentimeters/Grams/Secs
16 Pressure at different locations may vary P atmosP oralP trachP alveolar
17 Air Composed of gas molecules Particles travel in Brownian motion Has pressure (force per unit area)Pressure related to speed of Brownian motion“rapid changes in relatively static atmospheric pressure we call sound pressure or sound energy”Air composed of gas molecules (Nitrogen & oxygen, etc)Molecules always in motion, try to keep distance from each other -- Brownian motionPressure: force per unit areahigh pressure vs low pressure areaspressure related to speed of Brownian motionfaster BM and higher density = higher force and higher pressureAtmospheric air pressure: changes gradually w/ altitude and weatherAir pressure: changes smaller & more rapid
18 Air Movementdriving pressure: (difference in pressure) high pressure FLOWS to low pressurevolume velocity: rate of flow – e.g liter/seclaminar flow - in a parallel mannerturbulent flow – in a non-parallel manner (flows around an object - eddies)
20 Air Pressure, Volume, Density Volume: amount of space in three dimensionsDensity: amount of mass per unit of volumeBoyle’s law - as volume decreases, pressure increases(assuming constant temperature)
21 What is “sound?” Form of energy Waves produced by vibration of an objectTransmission through a medium (gas, liquid, solid)Has no mass or weightTravels in longitudinal wavetext pg. 7
22 Transverse vs. Longitudinal Wave Longitudinal – SOUND!In sound waves, also known as acoustic waves, the local oscillations always move in the same direction as the wave. Waves like this are called longitudinal waves. Unlike acoustic waves, radio waves or guitar-string vibrations are transverse waves; that is, the local oscillations are always perpendicular to the wave motion. Transverse waves can be set up in, say, a skipping rope or a washing line.Unlike acoustic waves, radio waves or guitar-string vibrations are transverse waves. That is, the vibrations of the medium are perpendicular to direction the wave is moving. Examples include an audience wave, waves of a ribbon or string, guitar-string vibrations and water waves.Cool!
23 Air Pressure Changes from Sound CompressionRarefaction
24 Propagation of Sound DEMO: Longitudinal Wave C = Condensation C R C R C R CC = CondensationR = RarefactionDEMO: Longitudinal Wave
25 Periodic waves Simple (sine; sinusoid) Complex (actually a composite of many overlapping simple waves)
26 Sinusoid wavesSimple periodic motion from perfectly oscillating bodiesFound in in nature (e.g., swinging pendulum, sidewinder snake trail, airflow when you whistle)Sinusoids sound ‘cold’ (e.g. flute)
27 Simple waves - key properties Frequency = cycles per sec (cps) = HzAmplitude – measured in decibels (dB), 1/10 of a bel(Note: dB is on a log scale, increases by powers of 10)
28 Frequency and intensity are independent! Frequency = cyclessecondOr “cps” or “Hertz (Hz)”Higher frequency = higher pitchIntensityMagnitude (amplitude) of vibrationAmount of change in pressureFrequency and intensity are independent!Frequency of vibration determines frequency of sound waveFreq – cycles/second (cps) -- HertzHigher frequency – higher the pitch
29 Amplitude (intensity) More force is applied and this increases pressure at peaksPeaks (condensations)= increases in pressureDRAW SEPARATELYTroughs (rarefactions)= decreases in pressure
31 Quickie Quiz! Q: What is the frequency of this wave ? HINT: It repeats 2.5x in 10 msec10 msec
32 Answer:250 Hz!(2.5 cycles in .01 sec = 250 cps)
33 Periodic waves A complex wave is periodic, but not sinusoidal Panels A and B are periodic waves, though Panel B is complex periodicB.
34 Complex periodic waves Results from imperfectly oscillating bodiesDemonstrate simple harmonic motionExamples - a vibrating string, the vocal folds
35 Complex periodic waves – cont’d Consists of a fundamental (F0) and harmonicsHarmonics (“overtones”) consist of energy at integer multiples of the fundamental (x2, x3, x4 etc…)
36 Harmonic seriesImagine you pluck a guitar string and could look at it with a really precise strobe lightHere is what its vibration will look like
37 From simple waves complex (showing power spectra) Also known as a “line spectrum”At bottom is a complex wave with an F0 of 100 HzNote energy at 200 Hz (second harmonic) and 300 Hz (third harmonic)
38 Fourier’s Theorem Fourier Synthesis Fourier Analysis Complex waveform Complex Sounds: Addition of Sinusoids: The top sinusoid in this slide depicts a 100-Hz sinwave, the next sinusoid down a 200-Hz sinwave, the third sinusoid from the top is a 300-Hz sinwave, and the red waveform on the bottom is the time-point for time-point sum of the amplitudes of the three sinusoids represented by the three blue sinwaves (100 Hz, 200 Hz, 300 Hz). Thus, the summed red waveform starts as a 100-Hz sinwave and it becomes more complex as the 200-Hz and then the 300-Hz components are added. This slide is an animation of Figure 4.2 in the Textbook.Complexwaveform
44 Continuous spectrumEnergy is present across a continuum of frequencies (noise)Frequency (Hz)- Amplitude +(not necessarilyequal energyat all frequencies!)
45 Decibels Unit used to express intensity or pressure of sound Ratio of 2 numbersExponential or logarithmic scaleExpressed in terms of a specified reference valueAmplitude of vibration or the extent of the particle displacement that determines the intensity of soundNamed after Alexander Graham Bell, the decibel or dB, is a unit ofmeasurement of intensity in the field of acoustics.Uses a logarithmic scaleIt is expressed in terms of a specified reference value. Therefore, it is a relative unitNot an absolute value – is a ratio of 2 numbers!Not linearIs exponential or logarithmicIncreasing sound pressure 10 fold creates only a 2dB increaseIncreasing sound pressure 100 times creates a 20dB increaseReference for the log ratio differs depending on the application
46 Linear vs. Logarithmic Scale Increments NOT EQUALBased on exponents of a given numberLog Scale1029001039,00010490,000105Incremental amount between unitsLinear Scale1,0002,0003,000Linear: a ruler, for example begins at zero and each amount is equal to the nextcan sum units by additionLogarithmic: log scale is based on exponents of a given numberFor dB, the base is 10 and the scale consists of incremental amounts that are not equal and are increasingly large numerical differences
47 Why use log scale for sound? Large dynamic range between pressure of smallest audible sound and largest tolerable soundDifficult to use with interval scale, so use a ratio scaleSound power = energy transferred over time (work done)so, unit is watts per meter squaredor can be described by how much pressure the sound exerts (Pa or N/m2)The human ear is sensitive to a large intensity range: that is, the human ear has a large dynamic rangeEx: 20 microPascals to 20,000,000 micro PascalsFor example: 1 intensity unit
48 Decibel Logarithm of the ratio of 2 intensities Ratio of power or pressure measured, divided by a reference power or pressure (usually 20 μPA)Logarithm labeled a “bel”“bel” is still too large, so 1/10th of bel is used: decibelThreshold of hearing = “softest sound of a particular frequency a pair of human ears can hear 50% of the time under ideal listening conditions”Human hearing is relatively insensitive to low bass (below 100 Hz), and also compresses at higher sound levels.Typically tested with a sound level meter
49 dB – cont’d The ‘speech banana’ Midrange frequencies can be perceived when they are less intense..whereas a very low or high frequency sound must have more intensity to be audibleWhy you need to boost the bass on your stereo system if you turn the volume down low -- otherwise the lows become inaudible.The horizontal axis of the chart contains information about frequencies a person can hear, from high to low. The vertical axis provides details about decibel levels the person's ears can detect, charted with the lowest levels at the top and the highest at the bottom. The speech banana is located around one third of the way down the chart.Someone who has difficulty hearing normal human speech can have difficulty learning to talk, understanding conversations, and learning material that is presented in spoken format. People with mild hearing loss may not realize that they cannot hear normally. Children with hearing loss often struggle in class because they cannot hear the teacher talking, but they don't realize their hearing is impaired because they can hear sounds in other ranges. Regular hearing tests are designed to identify hearing loss early so that interventions can be provided.When a hearing test is conducted, the results can be charted to show the range of sounds the subject of the test can hear. People with good hearing will have results that are located above the speech banana, meaning that they can hear sounds at both lower and higher frequencies than normal human speech, and lower decibel levels than normal human speech. If hearing test results fall below or within the speech banana, it means that the person may have difficulty hearing people talk.One approach to hearing loss is to fit the person with hearing aids. Hearing aids are adjusted and tested to generate a new audiogram showing what the person can hear while wearing the hearing aids. The goal is to provide the person with the ability to hear the normal range of human speech to facilitate communication. In other cases, hearing loss may be profound enough that hearing aids cannot provide people with a range of hearing that falls inside the speech banana. These individuals may need people to talk more loudly around them, or they may consider using modes of communication like sign language to facilitate conversation.The ‘speech banana’
50 More dB facts Orville Labs, Minneapolis -9.4 dBA Threshold of hearing = “softest sound of a particular frequency a pair of human ears can hear 50% of the time under ideal listening conditions”Many prefer A-weighting – considers human hearing in the best possible way (dBA)Typically tested with a sound level meter FACT: There can be a negative dB value!Orville Labs, Minneapolis-9.4 dBA
51 Physical vs. perceptual Fundamental frequency (F0) Amplitude/ Intensity Duration PERCEPTUAL“Pitch”“Loudness”“Length”
52 Phon Scale Psychoacoustic scale for intensity If a given sound is perceived to be as loud as a 60 dB sound at 1000 Hz, then it is said to have a “loudness of 60 phons”60 phons means "as loud as a 60 dB, 1000 Hz tone”Instrumentation referencePsychoacoustic scale for intensity is the phone scaleUses a 1000HZ pure tone as the reference frequency.If a given sound is perceived to be equal in loudness to a 60dB sound at 1000Hz, it is said to have a loudness of 60phonsEach curve represents the equivalent loudness level of a 1000hz tone.For ex: the 30 phone line is equally loud at all frequencies as a 1000 Hz tone at 30 dBPsychoacoustic scale for intensity
53 Bark scale (pitch) (1961) A psychoacoustical scale for pitch Basically log-linearRanges from 1 to 24 and corresponds to the first 24 critical bands of hearing.Similar to Mel Scale(1961)
54 Phase A measure of the position along the sinusoidal vibration These two waveforms are slightly out of phase (approx. 900 difference)Important for sound localization
56 Damping - example(Loss of vibration due to friction)
57 Review of source characteristics Simple waves are a good way to learn about basic properties of frequency, amplitude, and phase.Examples include whistling; not really found much in speechComplex waves are found in nature for oscillating bodies that show simple harmonic motion (e.g., the vocal folds)
58 Source-filter theory Vocal tract model demo See chapter 6 in book for more info
59 Let’s look at the filter In speech, the filter is the supralaryngeal vocal tract (SLVT)The shape of the oral/pharyngeal cavity determines vowel quality (via resonance)SLVT shape is chiefly determined by tongue movement, but lips, velum and (indirectly) jaw also play a role
60 ResonanceResonance = reinforcement or shaping of frequencies as a function of the boundary conditions through which sound is passedTo get a basic idea of resonance, try producing a vowel with and without a paper towel roll placed over your mouth! The ‘extra tube’ changes the resonance properties.
61 Resonance – cont’dThe SLVT can be modeled as a kind of bottle with different shapes… as sound passes through this chamber it achieves different sound qualitiesThe resonances of speech that relate to vowel quality are called formants. Thus, R1 = F1 (“first formant). R2 = F2, etc.F1 and F2 are critical determinants of vowel quality
63 Resonance - concepts Natural frequency (Resonant frequency RF) Mechanical (vibrating body, e.g. vocal folds)Acoustic (air-filled space)
64 ResonanceChild’s swing behaves like a pendulum and can help us understand resonanceRestorative & displacement forces, inertia & equilibriumTwo ways to inject energy into the oscillation:From Alison Behrman’s Speech and Voice ScienceThe oscillation of the swing behaves in the same way as the earlier example of the mass/ spring system.When the swing hangs straight down, it is at equilibrium.The displacement of the swing results in a buildup of a restorative force, with an overshoot of equilibrium due to inertia.If there is no one to push the child on the swing, and the child does not “pump” her legs, then friction will overcome the displacement and restorative forces.The arc (amplitude) of the swing will slowly decrease and eventually the swing will come to rest.There are two ways to put energy into the swing systemAn adult can come along and push the childThe child can pump her legs out and lean back as she moves from the equilibrium position forward to the top of the arc.Then as the restorative force overcomes the swing and it accelerates back toward equilibrium, the child can tuck her legs in and leanforward a bit.In both cases, an outside force (the adult’s push or the child’s pumping movements) results in a large increase in the amplitude of the vibration.Critical factor is the timing of the outside force.If energy is supplied to the swing with appropriate timing - the amplitude of its movement will increaseIf energy is supplied to a vibration at the same frequency of the vibration, it will increase the amplitude or energy of the vibration.If adult tries to push the swing while it is still in the backwards motion – the swing ay come to a stop.In that case energy was imparted to a swing at a frequency other than the resonant frequency of the swing.*the amplitude is independent of the frequency*Push the swing harder and it will not make the swing travel faster – only in a larger arc.The phenomenon of matching energy input to a system and the frequency of vibration of the system is called resonance.
65 ResonanceResonance: large increase in vibration when a force is applied at a natural frequency of the mediumSet an object into vibration (hit, drop, etc) and the rate at which it vibrates ~ natural frequency (or set of frequencies).
66 VibrationFree vibration – no additional force is applied after an object is set into motion.Forced vibration – when forces “drive” the motion to vibrate at its natural frequency. (e.g., guitar string).
67 Forced VibrationThe Tacoma-Narrows Bridge twisting in sympathetic resonance with the wind (left) and stretching to the point of breaking (right).In 1940, a bridge was constructed across Puget Sound in Washington State. It was a suspension bridge with 2800 feet been two suspension pillars. Very narrow, only 40 feet wide. Only a few months after completion, strong winds caused standing wave oscillations of the bridge. The amplitude of the oscillations became so large that the bridge actually collapsed. Contributing factors include narrowness & flexibility of the bridge & unusual wind patters that circled around the bridge at the resonant frequency of the bridge.
68 Resonant Freq of Glass Breaking glass with sinusoid sound Freaky deaky guy actually breaks glass with voice (Mythbusters)Typical case of faking it
69 Why is resonance important? In vocal tract, allows some frequencies to be magnified over others.In the auditory system (ear canal) allows for certain sounds to be amplified as compared to others.Resonance in the vocal tract (caused by articulator movements and vocal constriction) give rise to differences in speech acoustics, allowing some frequencies to be magnified over others.
76 Formants – ‘HAR’ ✓ H: Height relates inversely to F1. ✓ A: Advancement relates to F2.✓ R: Rounding is a function of lip protrusion and lowers all formants - through lengthening of the vocal tract by approximately 2 to 2.5 cm.
78 To be continued…We have described resonance and formant frequencies with respect to vowelsAlso important for consonantsBecause consonants require more spatial and temporal precision, these details will be covered later…