2. Prosody in Speech Interaction Expression of the Speaker and Appeal to the Listener Klaus J. Kohler IPDS, Kiel, Germany 8 th Phonetic Conference of China, Beijing International Symposium on Phonetic Frontiers 18 April, 2008

3. 1The pradigm of systemic linguistics Following de Saussure, a language is modelled as being held together by a tight system of categorically distinct units that enter into syntagmatic structures. These units and structures are defined formally. If communicative function is considered at all, it is brought in post hoc and limited to linguistic function, e.g. differentiation of words, of sentence mode, various types of focus, information structure.

4. The units are regarded as discretely different in their physical manifestation. –categorical perception Phenomena that cannot easily be accommodated in such a formal systemic framework, and depend on substantive identification in the flow of speech rather than on abstract paradigmatic differentiation are relegated to paralinguistics.

5. If linguists do not leave paralinguistics to the social sciences but tackle it themselves they tend to apply their systemic framework –categorical linguistic form –no network of communicative functions –no or insufficient contextualization in data acquisition, e.g. isolated sentences

6. 2The paradigm of Conversation Analysis naturally occurring data in talk-in-interaction phonetic and interactional analysis in parallel all details relevant, especially fine phonetic detail

7. behaviouristic approach –focus on singularity of spontaneous data –generalizing to other data not without problems –systematic experimental data excluded –categories of functional – phonetic relation emerge from the data °no intuitions °no testing of hypotheses

8. 3Combining aspects of both paradigms following the CA paradigm paralinguistic research must – rely on data from spontaneous dialogue interaction –have a functional orientation –take fine phonetic detail into account from auditory and instrumental observation A paradigm of speech communication

9. based on these observations –and relying on the native speaker’s intuition –hypotheses are formulated –and tested on data of large spontaneous corpora

10. based on the results –experimental data are collected in systematically controlled acquisition procedures °situational and linguistic contextualization °dialogue simulation –involving the speaker and the listener –for generalizations to categories of communicative functions

11. repetition of the cycle from data via hypotheses to categories and back again until satisfactory paralinguistic modelling has been achieved data-triggered deductive approach needs to start from initial postulates for speech communication Bühler’s Organon Model

12. factual world senderreceiver representation expressionappeal symbol signal symptom SIGN Appell

13. 4Developing a functional framework for speech communication linguistics is typically concerned with the function of representation, i.e. propositional meaning it makes a distinction between given and new –this is information structure in the factual world it ignores the argumentation structure into which the speaker projects information –this introduces the functions of concluding/ opening an argument 4.1Expression of the Sender

14. –early and medial synchronization of f0 peak contours “He used to be slim.”

15. the photo provides the given fact that “he used to be slim” –but the speaker decides on its argumentative weight –either a final conlusion “that’s it” –or an opening argument “I see it like this” –by focussing on low or high pitch in the accent

16. contrast and expressive evaluation can now come in –introducing a new perspective in an opening argument may be neutral: matter-of-fact observation “he’s slim in this old photo” contrastive: surprise observation °accepting the contrast “in this old photo he’s slim, but he no longer is” °opposing expressive evaluation of the contrast “my word, he has put on a bit of weight” –medial – late peak, peak height, phonation

17. Expression superimposed on Representation “He used to be slim.”

18. (a)Experimental Analysis of Argumentation Structure in German perceptual test paradigm, 11th ICPhS Tallinn 1987 –shift of complete f0 contours through otherwise constant utterances –f0 peak contour shift from early via medial to late synchronization with the articulation of an accented syllable –discrimination of pitch changes –identification of meaning changes –"Sie hat ja gelogen." "She's been lying."

19.  70Hz 140Hz 85Hz 90Hz 110Hz

20. –functional link of peak synchronization shown by contextualization for Sie hat ja gelogen. °early ¶Wer einmal lügt, dem glaubt man nicht. ¶"Once a liar, always a liar." °medial ¶Jetzt versteh ich das erst. ¶Now I understand. °late ¶Oh! –Identification as +- matching in context

21. Identification in context “now I understand” Sie hat ja gelogen. "She's been lying." early > medial

22. pragmatic functions of peak contour synchronization –early - finality °knowing °summarizing °coming to the end of an argument °resignation

23. –medial - openness °observing °realising °starting a new argument

24. –late - unexpectedness °observing, realising in contrast to one‘s expectation °surprise °disbelief

25. f0 peak synchronization strong cue for the three functional categories but there are other acoustic cues –shape of the f0 contour –durations of the pre-accent and accent syllables –energy levels in pre-accent and accent syllables these property values are usually coupled in the natural productions of the three categories so peak shift paradigm introduces an artifact decoupling must be analysed systematically to evaluate the contribution of each property

26. Coupling of F0 and energy time courses “sie hat ja gelogen” ‘medial' 'late’ F0 Energy F0 Energy

27. the characteristics of the 3 peak categories are –early – falling pitch into the accented vowel to low level, decreasing prominence –medial – rising pitch into the accented vowel to high level before fall, increasing-decreasing prominence –late – rising pitch from low level late in the accented vowel to high before fall, late increasing prominence

28. peak categorization went into development of KIM The Kiel Intonation Model –which became the basis for prosodic labelling of the Corpus of Read and Spontaneous Speech of German –which in turn made corpus analyses possible

29. (b) Experimental Analysis of Argumentation Structure in English Same perceptual and functional differentiation of pitch peak contours in English –"She's been lying." exactly parallel –formally tested by Kleber °MA dissertation, Kiel 2005 under my supervision ° SP2006 Dresden " Form and Function of Falling Pitch Contours in English"

30. AS = accented syllablePAS = postaccented syllable 111 Hz 114 Hz 140 Hz 96 Hz 76 Hz  ASPAS t F0 She's gone to Malaga.

31. basis for modelling peak categories in corpus analysis excerpts from the Film "The Queen" with Helen Mirren 'early''medial''late'

32. 'early' and 'late' in dialogue in this film –"The sheer joy of being partial." –"Yes. ('late') Of course one forgets that as sovereign, you are not entitled to vote." ('late') –"No." ('early' conceding)

33. "No." 'early' – accepting, conceding 'late' (2x) – emphatically rejecting

34. (c) Aspects of Argumentation Structure in Mandarin Chinese I first reported on the form and function of pitch peak contours at an international level in 1986 at AT&T Bell Labs, Murray Hill there was a young Chinese doctoral student in the audience, Chilin Shih –who had no knowledge of German –but who perceived the same psychophonetic category changes –and equated them with Mandarin tones

35. °early peak – the low tone 3 °medial peak – the falling tone 4 °late peak – a combination of tones 2 + 4 3 conclusions deducible from these observations –aspects of pitch perception have a psychophonetic basis independently of the language and its comprehension –Mandarin uses similar pitch categorizations at the lexical level that are systematized at the phrase level in German and English: low vs. high fall –so, how do Chinese speakers realise the types of argumentation structure associated with f0 peaks?

36. It is to be expected that the functions of finality and openness in argumentation are basic in speech communication in any language. –they should therefore also be manifested in adapted form in a tone language like Mandarin –hao with low tone 3 °"OK, I am forced to agree" °"OK, I am happy to agree" –xing with rising tone 2 °the same distinction in argumentation

37. Chinese hao "OK" (upper) und xing "OK" (lower), resigned (left), happy (right); Yi Xu, UCL

38. Chinese hao "OK" (upper) und xing "OK" (lower), resigned (left), happy (right); Aoju Chen, MPI Psycholinguistics, Nijmegen

39. In the 'resigned' as against the 'happy' context –hao °either no rise or a much lower one °intensity lower and descending more quickly °the syllable is shorter –xing in the two contexts is similarly differentiated °lower vs. higher pitch rise °lower vs. higher and faster vs. more slowly descending intensity °shorter vs. longer duration.

40. –both speakers differentiate the two contexts across the different word tones in the same way by lowering vs. raising pitch and prominence, superimposed on the lexical tones –this essentially parallels signalling of the functional categories in non-tonal languages The conclusions to be drawn from these few data are –the substance – form – function relationship of argumentation in Chinese needs detailed analysis –collection of a contextualized database –including all 4 tones of Mandarin Chinese

41. 4.2Appeal to the Receiver question, request, command are receiver-directed, –typical appeal functions –formally coded – lexically, morphologically, syntactically, prosodically –thus of great interest to the linguist under the perspective of propositional representation –resulting in simple associations, in German/English °of question-word questions with falling pitch °of polarity questions with rising pitch

42. 4.2.1Signals to the reveiver in questions Rising vs falling in word-order questions –original rising; female speaker ANS, g091a013 “würde Ihnen das passen?” “would that suit you?” °final rise: openness as to ‘yes-no’ °final fall: speaker expects ‘yes’ °final late fall: speaker contrasts ¶expected ‘yes’ ¶with potential ‘no‘, which would be inappropriate ¶irritation

43. rising vs falling in word-order questions original rising; ANS, g091a013 “würde Ihnen das passen?”

44. 4.2.2Surprise signals to the receiver ‘focus’ –with peak contours –signals sender and fact orientation ‘focus’ to express receiver orientation –high rising valley contours –category of surprise expression superimposed on appeal

45. “He’s in Rome?” 1.matter-of-fact echo question 2.disbelieving echo question 3.increased disbelief

46. Valleys may be synchronized early or late with the accented-vowel onset early and late valleys are functionally differentiated –‘Factual Surprise’, e.g. in an echo question as a request for repetition and confirmation in case of bad transmission –‘Affective Surprise’, expressing disbelief

47. 5The empirical basis of the paradigm I have presented a large number of audio examples to illustrate various categories of a network of functions in a paradigm of speech communication for paralinguistic research. They may strike some of you as anecdotal and therefore unscientific. These “anecdotal” examples are, however, based on two decades of empirical investigation into the production and perception of speech interaction, with particular focus on prosody.

48. This research has combined corpus analysis with experimental data acquisition in the cyclic way I proposed at the beginning of this talk and which is further referenced in the published paper. So, my proposals have an empirical foundation which has not only proved to be reliable between data sets and experiments but also valid as a basis for explaining aspects of speech communication.

49. Since I am a firm believer in Bertrand Russel’s important distinction between ‘knowledge by acquaintance’ and ‘knowledge by description’ I also prefer to introduce an audience to complex structures by the demonstration of examples that appeal to their intuitive knowledge and let description follow. The main message of my talk is that the many frontiers linguistics has set up need to be transcended.

50. I hope it's come across Thank you for your attention!