2. Methods for defining categories in intonational phonology: A check on Italian data Barbara Gili Fivela Università del Salento – Lecce, Italy CRIL – Centro di Ricerche Interdisciplinare sul Linguaggio barbara.gili@ateneo.unile.it Universitat Autònoma de Barcelona - May the 14 th 2007

3. Overview Introduction   Linguistics and paralinguistics Intonation, meaning and categories Methods for defining categories in intonation Italian data: production and perception Production, perception, and perception-production:   Different constraints in production and perception?   Categorical perception in intonation

4. Prosody and intonation Prosody is due to variation in:   fundamental frequency (F0)-pitch   duration-length   intensity-loudness   speech tempo / speech rate   rhythm Intonation [Ladd, 1996]   suprasegmental: F0, intensity, duration   conveying meaning to phrases/utterances   organized in terms of categorically distinct entities and relations Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

5. Linguistics and paralinguistics Linguistics: the scientific study of language [Crystal, ’91] Paralinguistics: independent from the linguistic message…although it is ‘coordinated in time with the linguistic channel’ and influences the interpretation of the utterance [Ladd, 1996: 34]   Interaction (solidarity, aggression)   speaker’s attitude   Emotions (fear, surprise) “intonation clearly ‘feels’ paralinguistic” [Ladd, 1996: 38]   same features used for paralinguistic change (e.g.,voice quality)   over long stretches of speech (e.g. loudness)   affective and interpersonal meaning (e.g. doubt, irony) Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

6. Linguistics and paralinguistics in phonology/phonetics Linguistics: categorical distinction Paralinguistics: gradual changes Segmental level:   /i/ vs. /u/ ex.it. mito vs. muto; sp. si vs. su   /i/ vs. /i/ produced while smiling Suprasegmental level   Truth value of the utterance ex. In Saint Petersburg, OFFICERS always escort ballerinas [Rooth, 1985]   Sentence modality ex. it. vai vs. vai? ex. cat. volen una nena vs. ¿volen una nena?   …..produced for conveying surprise Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

7. Intonational meaning British tradition   functional units, such as head, nucleus, tail [Palmer, 1922]   intonation “involves the occurrence of pitch patterns, each of which is used with a set of relatively constant meaning, either on single words or on groups of words” [Cruttenden, 1986:9] IPO approach   pitch movements, defined through perceptual equivalence, combined according to a grammar of intonation in configurations and contours [‘t Hart and Collier, 1990]   intonation features have no intrinsic meaning, its semantics may be related to syntax, in cases of ambiguity resolution [id.] Autosegmental theories   sequences of L and H tone targets, belonging to pitch accents and edge tones [Bruce, 1977; Pierrehumbert, 1980]   pitch accents and edge tones convey both linguistic and paralinguistic meaning [Pierrehumbert and Hirschberg,’90; Kohler,’91; Ladd,’96; Gussenhoven,’04 ] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

8. Intonation conveys linguistic and paralinguistic meaning The meaning of intonation is considered as a way to shed light on its form [Ladd,1996: 98]   linguistic entities   paralinguistic cues imply modification of the way phonological categories are realized Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

9. Intonation conveys linguistic and paralinguistic meaning - II Tune-based analysis and tone-based analysis   Meaning conveyed by the whole contour [Liberman and Sag, 1974]   Meaning derived from contour’s components [Pierrehumbert and Hirschberg, 1990] Changes in the form of intonation   Implying a change in category   Gradual, for signalling paralinguistic changes   Due to phonetic implementation Problems with paralinguistic variation   High-fall vs. low-fall treated as contrasting in some analysis and as paralinguistic variants in others [O’Connor and Arnold, 1973 vs. Crystal, 1969]   Gradient form-meaning relations may be grammaticalized as discrete [Gussenhoven, 2002] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

10. Biological codes Frequency code [Ohala, 1983; Gussenhoven, 2002]   Differences due to phonatory system   low = dominant-self confidence-assertive mode   Grammaticalization: statement vs. questions Effort code [Gussenhoven, 2002]   Differences due to effort in production   High =important-surprise-emphasis-focus   Grammaticalization: focus Production code [Gussenhoven, 2002]   Differences due to energy dissipation   Lowering = end of constituent – finality Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

11. Which information are expressed by categorical elements? Accentuation, focus, phrase signals for indicating sentence modality, function and meaning   statement, yn-question [D’Imperio and House, 1997]   yn- and wh-question, check, focus, continuation [Cruttenden, 1986, Pierrehumbert and Beckman, 1986, Casper, 1998]   check, query and accessibility [Grice and Savino, 2003]   introducing, committing to presence, and selecting from background [Gussenhoven, 1984]   new, salient, linked to mutual believes, to be interpreted with the following [Pierrehumbert, Hirschberg,’90]   finality-knowing, openess-realizing [Kohler, 1987; 1991]   direct, indirect speech acts [Liberman and Sag, 1974; 1975]   topic and comment [Cresti, 2000] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

12. …and by gradient variations? Emotions, attitude…   Perception of paralinguistic form-function relation is influenced by subject’s background [Chen, 2005] Degrees of meaning related to the linguistic unit Openess-realizing and unexpectedness-opposing [Kohler, 1991; 2006] Neutral statement and contraddiction [Ladd and Morton, 1997]   two different meanings, categorically perceived Variations in:   pitch range, i.e. scaling of targets on the frequency scale   alignment, i.e. synchronization with segmental chain Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

13. Where do categories usually come from? Production   categories defined on the basis of speech recordings: read and (semi)spontaneous speech Perception   categories defined on the basis of perception experiments: IPO approach Production and perception   Patterns observed in production   Perception of different categories checked by means of perception experiments   Semantic contrast is no longer sufficient to ‘proof’ a structural difference [Kohler, 1991; Ladd and Morton, 1997; Gussenhoven, 2006] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

14. In intonation   contrast: both discrete and gradient   categorical = discrete   linguistic = categorical Production and perception mismatch   Production data suggest syllable onset as reference point for alignment [Caspers and van Heuven, 1992; van Santen and Möbius, 2000]   Ladd [1999] ‘segmental anchoring’ hypothesis, but see data discussed in the literature [Prieto and Torreira, in print; Loevenbruck and Welby, in print; Gili Fivela, 2004]   Perception data point to vowel onset as crucial for tone comparison [House, 1990:113]   Perceptual and acoustic tonal targets [D’Imperio, 2000] ‘Intonational’ and ‘categorical’ linguistic = discrete Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

15. Problems with methods for testing (categorical) perception?   ‘Even in situations where subjects can make sharp distinctions between classes, they are still able to discriminate within a class’ [Ladd and Morton, 1997]   “The ideal experiment […] observes the subject’s behaviour in a situation as close to natural conversation as possible” [Kochanski, 2006] ‘Perception’ and ‘categorical’ Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

16. ‘Linguistic’ and ‘categorical’ Kohler [2006] observes that according to some theories of intonational meaning:   Linguistically relevant elements are discrete and categorical   Categories of intonation have to be distinguished from paralinguistic modifications   Only categories are linguist’s concern Analyses of communicative functions and meanings   few meanings in the linguistic domain (accentuation, focus, phrasing, sentence modality)   intonation is mainly concerned with paralanguage (expressive and attitudinal;interactive; speaker evaluation of events - finality, openess; style) -> “Categorical perception in the classical sense is therefore a special case and not essential for pitch categorization” (see also Newport [1982]) Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

17. Methods for defining categories Production Speech recordings, both monologues and dialogues   read speech, proposing specific functions   (semi)spontaneous speech obtained by means of various tasks e.g., map-task, card games   Analysis of F0 tracks, in relation to segmental events   Invariant tonal events, independent of phonetic modifications “The ideal general methodology would then be some kind of cyclicity between test material and spontaneous speech using feedback from preceding studies” [Bruce and Touati, 1990]. Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

18. Methods for defining categories Perception - I Perceptual equivalence [‘t Hart and Collier, 1990] Structural discreteness tested by means of speaker intuition of perceptual equality, i.e. ‘passable imitations’ of each other   ‘passable imitation’ [Odé, 2005; Gussenhoven, 2006] Categorical perception [Repp, 1984; Gussenhoven, 1999]   Identification task   continuum between two phonological categories   stimuli are assigned to either category   abrupt shift   Discrimination task   pairs of stimuli to be judged ‘same’ or ‘different’   expected grater distinction across perceptual boundary Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

19. Categorical perception CAT I % CAT II Continuum of stimuli Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

20. Methods for defining categories Perception - II Perceptual magnet effect [Kuhl, 1991; Schneider et al., 2006]   each category has a prototype   lower discrimination sensitivity for its neighbours   Identification task   Goodness rating rating as for very bad/very good exemplar individuation of the prototype   Discrimination task prototype and (not necessarily adjacent) neighbour Semantic difference and scaling [Gussenhoven, 1999]   Gradient and categorical judgments on the presence of a meaning or its opposite (Grabe [1997] for discussion)   Judgement on the extent to which a meaning is conveyed - especially for paralinguistic On rating scales, see Chen [2005]; Rietveld and Chen [2006] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

21. Methods for defining categories Perception - III Imitation [Pierrehumbert and Steele, 1989; Gussenhoven, 1999]   continuum between two patterns   subjects are asked to imitate each stimulus, paying attention to the intonation pattern   in case they produce the whole continuum, the difference is gradient; in case of binomial distribution it is categorical   ‘correcting’ not acceptable patterns [Gussenhoven, 2006]   imitation of their own imitation [Brown et al., 2006; Kochanski, 2006] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

22. Check on Italian data Pisa Italian Production   Read speech   out of the blue   within context utterances   (Semi)spontaneous   Map-Task Perception   Perceptual equivalence   Passable imitation   Categorical perception   Perceptual magnet effect   Semantic difference and scaling   Imitation Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

23. Production Inventory of Pisa Italian tonal events [Gili Fivela, 2004] Functions   wh-question: query-wh   yn-question: query-yn, checks and align   statements: instruct   focalization Structurally distinct units   three types of nuclear pitch accents: H*, H*+L, H+L*   edge tones: L-L%, H-L%, L-H%, H-H% Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

24. Examples e dove dev(o) andare ? H* H+L* L-L% hai detto leggimelo ? H+L* H-L% and where should I go ? did you say read it to me? wh-question yn-question / check

25. Examples Allora ripartiamo riparti dalla partenza H+L* L- H+L* L-L% H* H+L* L-L% statement then let’s start again you begin from the start statement – narrow focus [L+]H*+L L-L% it will be a centimeter sarà un centimetro

26. Production: categories and meaning Meaning of pitch accents difficult to delimit   pitch accents are be shared by different functions   more than one pitch accent type may be exploited for a specific function   depending on pragmatic variation   general meaning, coherently with literature Analysis of contrasting characteristics lead to the choice of transparent, either abstract or detailed, labels   emphasizing the structurally distinctive characteristics   differentiating shared and dissimilar structural properties   low target point as starting point of a rise to H* Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

27. Examples Mangia il melone (…) statement broad focus utterance final s/he eats the mellon … H+L* L-L% [L+]H*+L L-L% [L+]H* L- statement contrastive focus utterance final statement broad focus utterance initial / (narrow focus)

28. Pitch accents under investigation [Gili Fivela, 2002] La pronuncia di lavaglielo non (la) ricordo mai The pronunciation of lavaglielo I never remember (it) B C [L+]H*+L L- [L+]H* L-

29. Perception Pitch accents under investigation Measurements of F0, latencies between targets and segmental points showed that the accents differ as for   target alignment [L+] H* [L+] H*+L ms   target scaling Hz   syllable duration ms Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

30. Acoustic manipulation Difference of mean values Number of steps for gradually getting from one pattern to the other one:   8 alignment steps: 15 ms   2 scaling steps: 13Hz(L) - 17Hz(H) - 6 Hz(L)   5 repetitions PRAAT – PSOLA resynthesis Perceval (Aix-en-Provence) for perception test Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

31. Perception of peak accents Identification test, manipulating the alignment and scaling characteristics of stimuli – 10 subjects ‘No. Ho detto velava velocemente’ I said velava quickly Alignment Is it a peremptory and conclusive correction? Stimuli are ambiguous as for pitch height Pitch height has a significant influence on perception Scaling [Gili Fivela, 2005] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

32. Summing up: peak accents In identifying two peak accents,   “S-shaped” plots in relation to alignment   but there was always an ambiguous pitch height value   Extremes are categorically perceived   Pitch height has an influence on ‘when’ a different pattern is perceived Discrimination task would be needed   best with no ambiguous cues Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

33. Peak accents: attention to cues In the identification test   subjects appear to rely on different cues AlignPitchRange Ar Fa Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

34. Imitation task Imitation of same stimuli (same steps of manipulation)   contrastive base, 3 subj Stimulus – say number - beep – target imitation Measurements of (L)HL target height and latencies: vowel onset-to-H ILL IL NI Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

35. Examples Mangia il melone (…) statement broad focus utterance final s/he eats the mellon … H+L* L-L% [L+]H*+L L-L% [L+]H* L- statement contrastive focus utterance final statement broad focus utterance initial / (narrow focus)

36. Perception Pitch accents under investigation-II Absence/presence of a (close) low target preceding a rise to peak Measurements of F0, latencies between targets and segmental points showed that the accents differ as for   target alignment H+L* [L+] H*+L ms   target scaling Hz   syllable duration ms Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

37. Acoustic manipulation Difference of mean values Number of steps for gradually getting from the broad focus pattern to the contrastive one:   5 alignment steps: 22 ms   4 scaling steps: 15 Hz (+ 1 step 7.5 Hz)   3 repetitions PRAAT (PSOLA) Perceval Test on perceptual relevance of L+ target [Gili F., 2006] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

38. Steps of align manipulation Perception of falling accents Identification test, manipulating the alignment and scaling characteristics, from a broad stimulus – 10 subj ‘Mangia il melone’ s/he eats the mellon Alignment Does it correct a preceeding utterance? Stimuli are ambiguous as for pitch height Pitch height has a small influence on perception Scaling [Gili Fivela, 2006] Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

39. Falling accents: attention to cues In the identification test   subjects appear to rely on different cues AlignPitchRange FVMA Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

40. Is there a base effect ? Identification test, manipulating the alignment and scaling characteristics, from a contrastive stimulus 12 subjects Scaling Alignment No S-shaped plot Stronger influence of pitch height Extremes are categorically perceived Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

41. Summary and comments In identifying two falling accents,   No S-shaped plots in relation to alignment   but there was always an ambiguous pitch height value   not even categorically perceived (broad base)   Pitch height alone has an influence, at least when a contrastive base is considered   other correlates? Syllable duration? Possible reasons for these results: Ambiguity in function or meaning?   Same phonological categories   not gradient variation in production   coherent, at least partly, with Gussenhoven’s hypothesis Not appropriated task   more articulated context: question-answer sequence?   sentence modality: question (check) vs statement? Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

42. What about a different task ? ‘Question-answer’ Broad focus base Contrastive focus base Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion Identification test, from both broad and contrastive base – 11/12 subjects Sequence of question-answer   Broad context: broad focus answer expected Cosa succede what’s up?   Is the answer adeguated to the question? Same results !

43. What about a different task ? ‘Is it a question?’ Identification test, from broad base - 13 subjects Utterance in isolation   Need to set up a quite complex context based on mutual believes   Could you interpret it as a check of information? Would you give a yes/no answer? Subjects were actually giving answers that were exactly the opposit of expected ones ! Too difficult task Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

44. Imitation task Imitation of same stimuli (same steps of manipulation)   contrastive base – 3 subj Stimulus - beep - imitation – say a number - target imitation Measurements of (L)HL target height and latencies: syllable onset-to-H IL NI ILL Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

45. Discrimination: falling accents Pairs: AB and AA   B with either higher or later peak [Ladd and Morton, 1997]   From both broad and contrastive base Same or different? 9 subjects No discrimination !   Reaction times give no information discrimination scoresreaction times Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

46. IL discrimination scores reaction times ILNI IL

47. Summary Production data quite straightforwardly showed the existence of (patterns) pitch accents related to functions/meanings Perception data seem to be far more problematic:   Reason may be that they have being investigated teasing alignment and scaling apart   In any case, the H* vs H*+L contrast appears to be perceptually more different that the H*+L H+L* contrast   Probably only one phonological distinction Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

48. Summary Two peak accents: [L+]H* vs [L+]H*+L   “S-shaped” identification results   but not all speakers appear to pay attention to the same cues   Extremes are correctly identified   2 out of three speakers appears to perform the imitation task   They both imitate the continuum by creating two different classes as for the peak distance to vowel onset Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

49. Summary Two falling accents: [L+]H*+L vs H+L*   No S-shaped identification results   and not all speakers appear to pay attention to the same cues   Extremes are correctly identified   2 out of three speakers appears to perform the task   One speaker creates two different classes as for the peak alignment   One speaker creates a continuum of alignment differences Raction times, in general, give no information, but   looking at the ‘discriminator’ speaker: Trend in the direction of categorical peak and coherent information for reaction time Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

50. Improvements Methods could be improved   Gussenhoven proposal of ‘passable imitation’   Perceptual Magnet Effect rather than traditional CP, in particular for the discrimination task Experiments to be runned without teasing apart correlates Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

51. Examples Mangia il melone (…) statement broad focus utterance final s/he eats the mellon … H+L* L-L% [L+]H*+L L-L% [L+]H* L- statement contrastive focus utterance final statement broad focus utterance initial / (narrow focus)

52. Discussion Problem due to the form of intonation   especially in the second sets of experiments, pitch accents were formally really different   Proportional change of all the features needed   The resynthesis of continua is reasonable to apply for similar patterns Problem due to meaning/function of intonation   general meanings   functions expressed by the same pitch accent Both production and perception should be taken into consideration in deciding whether there is a contrast   Production could be more robust than perception   need of producing redundant features   perception within the context of strictly linguistic message Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

53. …and discussion Phonological perception should say something on the properties of phonic chain However intonation has also being described in terms of morphemes Morphemes   have meanings   may select different meaning of a base (message?)   among their meanings, one may be selected depending on the base (message?) Intonational units have a meaning/function even though they are not categorically perceived ? Introduction Meaning and Categories Methods Check on Italian Production Perception Discussion

54. If a language does not show a contrast that is adapt for perceptual testing, does it really mean that the language does not exploit a/that phonological contrast? ….in Pisa as well…… In Saint Petersburg, OFFICERS always escort ballerinas or In Saint Petersburg, officers always escort BALLERINAS

55. THANK YOU!