1. Psy1302 Psychology of Language Lecture 6 Speech Perception I

2. Understanding Speech Seemingly easy for us. Seemingly easy for us. –1 sec. of conversation contains roughly 8-10 phonemes 8-10 phonemes 3-4 syllables 3-4 syllables 2-3 words 2-3 words Difficult for engineers… Difficult for engineers…

3. Big Picture

5. Overview Preliminaries Preliminaries –Sound Waves, Spectrogram –Phonetics/Phonology Why is Speech Perception Difficult? Why is Speech Perception Difficult? –Lack of invariance –Coarticulation So How Do We Do It? So How Do We Do It? –Categorical Perception (our endowment) –Motor Theory vs. Auditory Theory (next Tues) –Fine-tuning of Categorical Perception (next Tues) –Top-down Influences (next Thurs)

6. Sounds A wave is a disturbance of a medium which transports energy through the medium without permanently transporting matter.

7. Listening Hearing Frequency: 20 Hz and 20000 Hz Hearing Frequency: 20 Hz and 20000 Hz Speech: 200-8000 Hz Speech: 200-8000 Hz Most sensitive to 1000-3500 Hz Most sensitive to 1000-3500 Hz Phones: 300-3400 Hz Phones: 300-3400 Hz

8. Complex Sounds Pressure Time

9. Complex Sounds Pressure Time Pressure Time Pressure Time + = Amplitude Frequency Amplitude Frequency Amplitude Frequency

10. Sounds and Spectrograms

11. Speaking Vocal Tract: Vocal Fold  Lips (Modeled as a tube) Vocal Fold Lips Average Man - Length = 17.4cm

12. Vocal Tract Model F1 F2 F3 Vocal Tract = 17.4 cm Speed of sound = 34800 cm/sec Speed = Distance/Time = Wavelength x Frequency Freq = Speed/Wavelength L = 17.4cm 500Hz 1500Hz 2500Hz λ = 4L λ = 4L/3 λ = 4L/5

13. Speaking c d a b c d ab on top of his deck Vocal folds Lips

14. Vocal Tract http://www.exploratorium.edu/exhibits/vocal_vowels/vocal_vowels.html ee oo oh ah eh

15. (heed) (hid) (hay) (head) (had) (hall) (hoe) (hood) (who) (hot) (hut) (her) Decrease of F2 (Place) Increase of F1 (Tongue Height) Vowels Vowels: unimpeded sound through vibrating vocal cords Vowels: unimpeded sound through vibrating vocal cords Lips http://www.youtube.com/watch?v=v9Wdf-RwLcs

16. Frequency (Hz) Time (s) 1000 Spectrogram of 3 vowels ee ooah

17. Phonetics and Phonology Phonetics: the study of speech sounds Phonetics: the study of speech sounds –(articulatory, acoustic, auditory phonetics) –how are speech sounds are produced, what are the physical properties of the speech sounds, how are they interpreted? Phonology: investigation the organization of speech sounds in languages, Phonology: investigation the organization of speech sounds in languages, –what are the phonotactic rules of the language, which sounds (phonemes) affect meaning?

18. Phonemes & Phones Phones: speech sounds Phones: speech sounds Phonemes: The unit of sound that makes a contribution to meaning Phonemes: The unit of sound that makes a contribution to meaning –Minimal Pairs [b] vs. [p] e.g., bat vs. pat [b] vs. [p] e.g., bat vs. pat [r] vs. [l] e.g., rump vs. lump [r] vs. [l] e.g., rump vs. lump –Crosslinguistic Differences The flied lice was yummy. The flied lice was yummy.

19. Allophones Sounds in complementary distribution… Sounds in complementary distribution… –…are allophones of a single phoneme –…do not occur in minimal pairs –…are noncontrastive –…are predictable (based on environment) E.g. the p in “pin” vs. “spin”. E.g. the p in “pin” vs. “spin”. (slides 20-35, 38-52 are taken or adapted from A.B. Wassink)

20. Real-life analogy of complementary distribution

21. Two people or one person? Do you ever see Superman and Clark Kent in the same environment? Do you ever see Superman and Clark Kent in the same environment?

22. Emergency Superman is always found in the environment of an emergency.

23. No Emergency Clark Kent is seen in the environment when there is no emergency.

24. We can conclude: Clark Kent and Superman are different identities of the same person. =

25. The analogy Clark Kent and Superman are like allophones. Clark Kent and Superman are like allophones. They are noncontrastive They are noncontrastive They appear in complementary distribution. They appear in complementary distribution.

26. Just as allophones are different forms of the same phoneme, Clark Kent and Superman are different realizations of the same person. Just as allophones are different forms of the same phoneme, Clark Kent and Superman are different realizations of the same person. /Superman/ [Clark Kent] [Superman] /p/ [p h ] [p]

27. Phonemes Two Kinds of Phonemes Two Kinds of Phonemes –Vowels –Consonants

28. Vowel articulation Chambers in mouth Chambers in mouth (above the glottis): (above the glottis): –Oral cavity –Pharynx (behind tongue) –Area between lips –(Nasal cavity) Length and shape of each chamber affect the ‘ resonance ’ (or the properties of the vibration) of vowel sound Length and shape of each chamber affect the ‘ resonance ’ (or the properties of the vibration) of vowel sound --- pharynx

29. Tongue body position Saggital view of tongue positions in vowels 1) Tip 2) body 3) root

30. Articulatory Description 4-part classification system for vowels: 1) Tongue height 2) Frontness vs. backness of tongue 3) Tenseness 4) Lip rounding [ also (5) Nasality (in many languages)]

31. (heed) (hid) (hay) (head) (had) (hot) (hoe) (hood) (who) (hall) (hut) (her) Decrease of F2 (Place) Increase of F1 (Tongue Height) Vowels Vowels: unimpeded sound through vibrating vocal cords Vowels: unimpeded sound through vibrating vocal cords Lips

32. 1) Vowel Height High vowels: tongue body is raised High vowels: tongue body is raised Mid vowels: tongue body is intermediate Mid vowels: tongue body is intermediate Low vowels: tongue body is lowered Low vowels: tongue body is lowered (heed) (hid) (hay) (head) (had) (hot) (hoe) (hood) (who) (hall) (hut) (her) Decrease of F2 (Place) Increase of F1 (Tongue Height) Lips

33. 2) Vowel Front/Backness Front vowels: tongue body is pushed forward Front vowels: tongue body is pushed forward Back vowels: tongue body is pulled back Back vowels: tongue body is pulled back Central vowels: tongue body is neutral Central vowels: tongue body is neutral (heed) (hid) (hay) (head) (had) (hot) (hoe) (hood) (who) (hall) (hut) (her) Decrease of F2 (Place) Increase of F1 (Tongue Height) Lips

34. 3) Vowel Tenseness Tense: more extreme position of the tongue or lips Tense: more extreme position of the tongue or lips Lax: less tense position of tongue or lips Lax: less tense position of tongue or lips (heed) (hid) (hay) (head) (had) (hot) (hoe) (hood) (who) (hall) (hut) (her) Decrease of F2 (Place) Increase of F1 (Tongue Height) Lips

35. 4) Vowel Roundness Rounded: produced with rounded lips Rounded: produced with rounded lips Unrounded: produced with unrounded lips Unrounded: produced with unrounded lips Many languages also have front rounded vowels (e.g., French) Many languages also have front rounded vowels (e.g., French) lit “bed”; lu “read”; loup “wolf” (heed) (hid) (hay) (head) (had) (hot) (hoe) (hood) (who) (hall) (hut) (her) Decrease of F2 (Place) Increase of F1 (Tongue Height) Lips

36. Scientific American 1999 Forming Formants

37. Consonants Consonants: Impeded sound Consonants: Impeded sound

38. Articulatory Description For consonants, three-part classification system: 1) Voicing 2) Place (of articulation) 3) Manner (of articulation) e.g., voiced labiodental fricative = [v]

39. 1) Voicing Voicing: what is happening at the LARYNX? Voicing: what is happening at the LARYNX? –Are the vocal folds spread apart (voiceless), or are they close together and vibrating (voiced)? Front Vocal folds Glottis Back

40. Voiced or Voiceless? voiced voiceless

41. 1.[p] pat[b] bat 2.[d] die [t] tie 3.[g] gill [k] kill 4.[f] fat[v] vat 5. [s] sip[z] zip Voiced or Voiceless?

42. [p] pat[b] bat [t] tie[d] die [k] kill[g] gill [f] fat[v] vat [s] sip[z] zip [ ө] thigh[δ] thy [š] dilution[ž] delusion [ č] etch[ĵ] edge Voiceless Voiced

43. liquid r * Voicing Value

44. 2) Place Place (of articulation): WHERE in the vocal tract is the constriction being made? Place (of articulation): WHERE in the vocal tract is the constriction being made?

45. Place of Articulation Place of contact creating the obstruction in making the consonant Place of contact creating the obstruction in making the consonant 1. Bilabial 2. Labial Dental 3. Dental/Interdental 4. Alveolar 5. Palatoalveolar 6. Palatal 7. Velar 8. Uvular 9. Glottal

46. Place Bilabial: w/ both lips Bilabial: w/ both lips –[b], [p], [m], [w] Labiodental: w/ lower lip and upper teeth Labiodental: w/ lower lip and upper teeth –[f], [v] (Inter-)dental: tip of tongue btw. the teeth (Inter-)dental: tip of tongue btw. the teeth –[ө] thin, [δ] then Alveolar: tongue tip on alveolar ridge Alveolar: tongue tip on alveolar ridge –[t], [d], [n], [l], [s], [z]

47. Place (Alveo-)Palatal: w/ tongue at or near hard palate (Alveo-)Palatal: w/ tongue at or near hard palate –Alveopalatal: [š] shill, [ž] azure, [č] church, [ĵ] jill –Palatal: [j] you Velar: w/ tongue at or near soft palate, or velum Velar: w/ tongue at or near soft palate, or velum –[k], [g], [ŋ] king Glottal: produced at the larynx Glottal: produced at the larynx –[/] uh-oh, [h]

48. liquid r

49. 3) Manner Manner (of articulation): HOW is the air being modified as it moves through the vocal tract? Manner (of articulation): HOW is the air being modified as it moves through the vocal tract?

50. Manner Stop: full obstruction in oral cavity (w/ velum raised/closed) Stop: full obstruction in oral cavity (w/ velum raised/closed) –[p], [b], [t], [d], [k], [g], [/] Fricative: partial obstruction w/ turbulence Fricative: partial obstruction w/ turbulence –[f], [v], [ө], [δ], [s], [z], [š], [ž] Affricate: stop followed by fricative Affricate: stop followed by fricative –[č], [ĵ] +

51. Manner Nasal: full obstruction in oral cavity w/ velum lowered/open Nasal: full obstruction in oral cavity w/ velum lowered/open –[m], [n], [ŋ] Liquid: constriction but no turbulence Liquid: constriction but no turbulence –[l] = lateral liquid –[r] = retroflex liquid Glide: slightly more constriction than a vowel Glide: slightly more constriction than a vowel –[w], [j] (and shows additional evidence of “consonantness”: patterns with consonants)

52. liquid r Manner

53. Consonants & Vowel Trivia Number of Consonants and Vowels Number of Consonants and Vowels –Varies dialectally (Mary, Marry, Merry) (Mary, Marry, Merry) –American Heritage 25 consonants and 18 vowels 25 consonants and 18 vowels

54. Consonants vs. Vowels Silbo Gomero, a Whistling Language Silbo Gomero, a Whistling Language Example 4 vowels and 4 consonants Example 4 vowels and 4 consonants 4000+ words 4000+ words –A: Hey, Servando! B: What? A: Look, go tell Julio to bring the castanets. B: OK. A: Hey, Julio! B: What? A: Lili says you should go get the kids and have them bring the castanets for the party. B: OK.OK.OK. http://www.cnn.com/2003/TECH/science/11/18/whistle.language.ap/ http://www.agulo.net/silbo/silbo.mp3

55. Why is it so difficult? (engineer’s lament) Segmentation Problem: Speech sounds seem separable and sequential: like beads on a string Speech sounds seem separable and sequential: like beads on a string Reality: Reality: –Speech sounds overlap –Each speech sound is affected by the elements around it –(See your spectrograms!)

56. Co-articulation speech sound is affected by the elements around it speech sound is affected by the elements around it /š/ (sh) in sheep, shoe /š/ (sh) in sheep, shoe /k/ (k) in keep, cook /k/ (k) in keep, cook /d/ (d) in dee, do /d/ (d) in dee, do

57. Speech Rate “I am going to leave.”

58. Why is it so difficult? (engineer’s lament) The “Lack of Invariance” problem – –local coarticulation effects – –speech rate & style – –prosody – –individual variability accent/dialect speaker’s vocal tract – length matters.

59. How do we do it? How do we perceive the sounds? How do we perceive the sounds? What acoustic cues do we use? What acoustic cues do we use?

60. http://www.haskins.yale.edu/featured/patplay.html Pattern Playback Machine

61. Spectrogram Revisted Steady State Steady State Transitional State Transitional State

62. Frequency (Hz) Time (msec) bada ga

63. Categorical Perception (Idealized Data)

64. Categorical Perception

65. Categorical Perception of Sounds Is the Categorical Perception phenomenon unique to speech sounds or can we find it in other sounds? Is the Categorical Perception phenomenon unique to speech sounds or can we find it in other sounds? Which part of the sound are we using to do Categorical Perception? Which part of the sound are we using to do Categorical Perception?

66. Categorical Perception of Sounds VS.

67. Mattingly et al. (1971) Which of the following manipulations results in categorical perception? Which of the following manipulations results in categorical perception? NO YESNO