2. Prosody Modeling (in Speech) by Julia Hirschberg Presented by Elaine Chew QMUL: ELE021/ELED021/ELEM021 26 March 2012

3. Sources Hirschberg, J. (2012). Lecture notes on Prosody Modeling (draws from Hirschberg, J. (2011). Interspeech Tutorial - More Than Words Can Say) – http://www.cs.columbia.edu/~julia/courses/CS4706/r epresenting.pptx Hirschberg, J. (2004). Pragmatics and Intonation. In L. R. Horn and G. L. Ward (eds.): The handbook of pragmatics, Blackwell Publishing Ltd., 515-537. Hirschberg, J. and M. E. Beckman (1994). ToBI Labeling Conventions. Cardinal examples: – http://www.cs.columbia.edu/~agus/tobi

4. Prosodic Variation and Interpretation Prominence: accents, stress – John only introduced MARY to Sue – John only introduced Mary to SUE Boundaries: disjuncture between words – Bill doesn’t drink | because he’s unhappy – Bill doesn’t drink because he’s unhappy

5. Example 1: Prominence

6. Example 2: Phrasing

7. ToBI Goal Capture enough variation to explain similarities and differences in prosodic meaning

8. ToBI Scheme ToBI annotation tiers: – Orthographic tier: Time-aligned words – Break-index tier: degrees of junction (0=no boundary; 4=full intonational phrase boundary) – Tonal tier: pitch accents, phrase accents, boundary tones – Miscellaneous tier: disfluencies, non-speech sounds, etc.

9. ToBI Break Indices Level 0: no boundary Level 1: word boundary Level 2: Strong juncture with no tonal boundaries Level 3: – minor or intermediate phrase – Consists of >=1 pitch accent(s), aligned with stressed syllable or lexical items (phrase accent) – Phr accents describe movement to phrase boundary: H-, !H-, L- Level 4: – major or intonational phrase (associated with tonal tier describing phrase accents and boundary tones for each level) – Consists of >= 1 Level 3 phrase(s) plus high/low boundary tone (H% or L%) at the right edge of phrase – Boundary tones describe pitch movement immediately before boundary

10. Standard Declarative Contour Ends with L- L% Example: H* L- L%

11. Standard Yes-No Question Contour Ends with H- H% Example: L* H- H%

12. Phrase Ending Types L-L%L-H%H-H%H-L%!H-L%

13. Break Indices Differences Associated with – Variation in f0 – Phrase final lengthening – Glottalization – Some amount of pause

14. Pitch accents: Intonational Prominence Achieved through – Different tone targets – Differences in f0 height – Being louder and longer Hierarchy – Last accented word tends to be most prominent – Most prominent accent in intermediate phrase is called phrase’s nuclear accent or nuclear stress

15. ToBI Accents H*: simple high (declarative) L*: simple low (yes-no question) L* + H: scooped, late rise (uncertainty, incredulity) L + H*: early rise to stress (contrastive focus) H + !H*: fall onto stress (implied familiarity) (* indicates stressed syllable) H* L* L+H* L*+H H+!H*

16. H* Most common accent in American English Simple peak in f0 contour Typically found in standard declarative utterances Commonly used to convey accented item should be treated as NEW information

17. L* Modeled as valleys in f0 Conveys accented item is salient but not part of what is being asserted Typically characterize prominent items in yes- no question contours Often employed to make initial prepositions or adverbs prominent, or to mark discourse readings of cue phrases

18. L + H* Can be used to produce a pronounced contrastive effect Example: The Smiths aren’t inviting anyone important – They invited L + H* Loraine (contradicts initial claim that Loraine is unimportant) – They invited L* + H Loraine (uncertainty about whether Loraine is an important person)

19. L + H*

20. H + !H* Fall onto stressed syllable Associated with implied sense of familiarity with the mentioned item Example (“reminding” case): – A: No German has ever won the Luce prize – B: H + !H* Joachim’s from Germany

21. H + !H*

22. More on !H Downstepped accent – !H*: Half the job is accomplished by just starting it – L + !H*: There’s a lovely one in Bloomingdale’s – L* + !H: Don’t hit it to Joey.

23. Example (Praat)

24. L*+H L* H* H-H%H-L%L-H%L-L% Examples from http://www.cs.columbia.edu/~agus/tobi

25. H* !H* H+!H* L+H* H-H%H-L%L-H%L-L% Examples from http://www.cs.columbia.edu/~agus/tobi

26. ToBI Family American English German Japanese Korean Mandarin Portuguese Greek Catalan

27. Exercise (3) Anna frightened the woman | with the gun Anna frightened | the woman with the gun Who held the gun in each case?

28. Exercise (4) Mary knows many languages you know Mary knows many languages | you know

29. Exercise (7) John laughed | at the party John laughed at | the party

30. Exercise (11) (12) We only suspected | they all knew that a burglary had been committed We only SUSPECTED | THEY all KNEW | that a BURGLARY had been committed