1 Determining query types by analysing intonation

2 Overview u Using prosodic features of utterances u Generating set of prosodic labels with which test utterances are annotated u Trying to determine which class the utterances belong to – action, problem, connect, who, info, other

3 Contents u Motivation u Corpus u Prosody u System architecture – pitch extraction – segmentation – prosodic labelling – label sequences (n-grams) u Results u Conclusions

4 Motivation u Linguists (Crystal, Searle) found relationship between – prosody and utterance type (question, command…) – prosody and attitude u Edinburgh maptask group (Taylor, Wright) found prosody help distinguish utterance types

5 British Telecom corpus u Callers dial 100, requesting – alarm calls, collect calls – codes, numbers – connection problems – … u 8000 calls: first utterance only u Annotation – call types: by BT – prosody: by me

6 Call types in BT corpus

7 Prosody u má mà (lexical tone) u yés yès (word-level intonation) u Now is the time for | all good men to | come to the | aid of the | party

8 Simplified architecture pitch extractor / octave error correction segmenter clustering centroid LM     utterance classifier draw layers thingy!

9 Pitch extraction “ Yes, Manchester please ”

10 Octave error correction

11 Simplified architecture pitch extractor / octave error correction segmenter clustering centroid LM     utterance classifier draw layers thingy!

12 Data points for one segment showing line of best fit duration penalty prevents very short segments also minimum and maximum segment lengths

13 Varying the duration penalty

14 Minimum segment length Yeah, could I book a wake- up call please

15 Simplified architecture pitch extractor / octave error correction segmenter clustering centroid LM     utterance classifier draw layers thingy!

16 Assigning labels u Each segment in training corpus has features – duration – gradient – mid-point frequency u Clustering algorithm (K-means) places segments in feature space u Prosodic labels assigned to segments, based on cluster membership

17

18 2-D data points arranged in 15 clusters

19 Label trajectories on to clustering now: discretization

20 More trajectory schemes no maximum with normalization 10 clusters 40 clusters

21 70 clusters

22 Simplified architecture pitch extractor / octave error correction segmenter clustering centroid LM     utterance classifier draw layers thingy!

23 Label sequences u N-gram collocation model used – 台中 vs 台 and 中 – label sequence e.g. [4;11;13;1] statistically more useful than individual labels u Association of label sequences with each class in training data computed u Then estimate test data classes using maximum likelihood model

24 Results u Correct classification around 1/3 – correct classification by chance around 1/4 u But changing parameters does affect results u Some optimum parameters – 20 clusters (prosodic labels) – only label sequences seen 4 times used – sequences of 4 labels best, performance degrades with 5-grams

25 Conclusions u Psycholinguistic experiment showed humans find same task difficult u Prosody cannot be used by itself to classify utterances u But, in combination with a lexical model, could be of use

26 Introducing Linguistics u What do linguists do? u Grammar, and other aspects of language u Relationships between languages u How is linguistics used in the real world?

27 What do linguists do? u They don’t necessarily “learn languages” – Linguist and 語言學 are confusing terms u They are often interested in the structure of languages. They might – specialize in one language, or a group of languages – compare different languages – study features shared by all languages

28 Many linguists study grammar u Syntax – the way words are arranged to make sentences – John had lunch / *John lunch had u Morphology – the way words are modified to fit the circumstances – John had lunch / *John have lunch u Linguists study – what people actually say – not what they “should” say!

29 The sort of things linguists look at in syntax u Syntax (the way words are arranged to make sentences) – John saw the girl with the telescope – 爸爸給小明買鹹蛋超人 – Me and Dad went to the toyshop – Dad bought an Ultraman for John and I

30 And in morphology… u Affixation: hardly used in Chinese – My son has 73 Ultramen – 我 (? 的 ) 兒子有 73 只鹹蛋超人 (* 們 ) u Compounding – rare in English: greenhouse, blackbird – productive in Chinese »Verb-object compounds: 開車, 幫忙 »Resultative compounds: 來得及, 跑不掉 »Stump compounds: 交大

31 Phonology: the sounds of a language u How good is ㄅㄆㄇㄈ at representing the sounds of Chinese? – 雄 is xiong in 韓愈拼音, vs ㄒㄩㄥ. – 嗯 and 恩 are the same in ㄅㄆㄇㄈ, n vs en in Pinyin u Has 台灣國語 lost the sounds ㄓㄔㄕ ? u Why do we sometimes hear 禮拜ㄕ ?

32 Historical linguistics u How languages are related – Language families »Indo-European, Sino-Tibetan… – Areal linguistics »Greek, Bulgarian – Mostly borrowed words; also shared grammatical features »Chinese, Korean, Japanese u How language changes over time – sounds: poor vs paw, suit. – vocab: 咖啡, 颱風. Calque: 摩天大樓, skyscraper, gratte-ciel – grammar: Did you eat yet? Adversative passive 被

33 Sociolinguistics u Diglossia: “high” and “low” prestige languages – The role of Mandarin and Taiwanese in a bilingual society – The changing role of English in Taiwan society: borrowing, or showing off? – case and size: code- switching, or lexicalized Chinese words? Ta-hsüeh-shih-ching

34 Applications for linguistics u Speech disorders u Forensic linguistics – Accent detection – Style verification (eg police style) u Language teaching u Computational applications – Machine translation – Speech recognition and synthesis – Language identification