Kees van Deemter, Dublin, Trinity College, May 2009 What utility can do for NLG: the case of vague language Kees van Deemter University of Aberdeen Scotland, United Kingdom

Kees van Deemter, Dublin, Trinity College, May 2009 Two big issues Does NLG have proper foundations? –What is its mathematical core? Why is language vague? –When does vague language have benefits over crisp language?

Kees van Deemter, Dublin, Trinity College, May NLG and utility 2.Why is language vague? 3.Why should NLG systems be vague? 4.Conclusions

Kees van Deemter, Dublin, Trinity College, May 2009 Why bother about foundations for NLG?

Kees van Deemter, Dublin, Trinity College, May 2009 Why bother about foundations for NLG? Computing is based on –theory of formal languages/computability –Hoare logic/program correctness Linguistics is based on –data & stats –the theory of formal languages –mathematical logic/set theory

Kees van Deemter, Dublin, Trinity College, May 2009 Why bother about foundations for NLG? Computing is based on –theory of formal languages/computability –Hoare logic/program correctness Linguistics is based on –data & stats –the theory of formal languages (syntax) –mathematical logic/set theory (semantics)

Kees van Deemter, Dublin, Trinity College, May 2009 Why bother about foundations for NLG? Computing is based on –theory of formal languages/computability –Hoare logic/program correctness Linguistics is based on –data & stats –the theory of formal languages (syntax) –mathematical logic/set theory (semantics) –how about pragmatics? Not just whats true, but whats appropriate given the circs.

Kees van Deemter, Dublin, Trinity College, May 2009 An NLG program translates input to linguistic output Essentially the problem of choosing the best Form for a given Content. E.g. –McDonald 1987 –Bateman 1997 on Systemic Grammar What does this choice depend on?

Kees van Deemter, Dublin, Trinity College, May 2009 What could this choice depend on? –Roughly: which utterance is most useful In other words, utility A perspective seldom pursued. Exceptions: –Kibble 2003 (IWCS-5, referring expressions) –Klabunde 2009 (ENLG-12) NLG programs become more expressive, so this perspective now becomes tempting

Kees van Deemter, Dublin, Trinity College, May 2009 Example: the hazards of road ice

Kees van Deemter, Dublin, Trinity College, May 2009

Example: Roadgritting (Turner et al. 2009, this conference) Compare 1.Roads above 500m are icy 2.Roads in the Highlands are icy Decision-theoretic perspective: false positives, 2 false negatives false positives, 10 false negatives Suppose each false positive has utility of -0.1 each false negative has utility of -2

Kees van Deemter, Dublin, Trinity College, May 2009 Example: Roadgritting ( Turner et al. 2009, this conference) Suppose false positive has utility of -0.1 false negative has utility of -2 Then 1: 100 false pos, 2 false neg = -14 2: 10 false pos, 10 false neg = -21 So summary 1 is preferred over summary 2.

Kees van Deemter, Dublin, Trinity College, May 2009 Communication in Game Theory (following Crawford & Sobel 1982) Set of contents C Set of forms F Set of actions A Speaker strategy S : C F Hearer strategy H : F A Two utility functions U S : A [0,1] U H : A [0,1] (not necessarily the same!)

Kees van Deemter, Dublin, Trinity College, May 2009 A special case: Vagueness Two related questions: 1.Can Game Theory help us explain why language is so often vague? 2.Can Game Theory tell NLG systems when to use vagueness?

Kees van Deemter, Dublin, Trinity College, May 2009 Vagueness: An expression is vague iff it has borderline cases. –Example of a vague adjective: poor is vague because some people are borderline poor. –Saying this differently: different thesholds for poor may be used

Kees van Deemter, Dublin, Trinity College, May 2009 Example: Is John poor? £ p/a £ p/a £ p/a poverty Norm A poverty Norm B John Norm A: John is poor is True Norm B: John is poor is False

Kees van Deemter, Dublin, Trinity College, May 2009 Other vague adjectives: large, small,... Vague nouns: girl, giant, island,... Vague determiners: many, few,... Vague adverbs: often, slowly,... Relevant for any NLG system with continuous input –weather forecasts (FOG, Sumtime-Mousam) –patient data (e.g. Babytalk)

Kees van Deemter, Dublin, Trinity College, May 2009 From Babytalk corpus BREATHING – Today he managed 1½ hours off CPAP in about 0.3 litres nasal prong oxygen, and was put back onto CPAP after a desaturation with bradycardia. However, over the day his oxygen requirements generally have come down from 30% to 25%. Oxygen saturation is very variable. Usually the desaturations are down to the 60s or 70s; some are accompanied by bradycardia and mostly they resolve spontaneously, though a few times his saturation has dipped to the 50s with bradycardia and gentle stimulation was given. He has needed oral suction 3 or 4 times today, oral secretions are thick. [BT-Nurse scenario 1]

Kees van Deemter, Dublin, Trinity College, May 2009 The question is: When (if ever) is vague communication more useful than crisp communication? The question is not: Can vague communication be of some use? Compare Rohit Parikh (2000) Ann calls Bob to bring the blue book, her only book on topology

Kees van Deemter, Dublin, Trinity College, May 2009

Example by R.Parikh (the book scenario) Bob only has to search all blue books Anns instruction reduces the number of books that Bob can expect to have to check. Each calls some books blue that the other does not. But they agree on most books.

Kees van Deemter, Dublin, Trinity College, May 2009 Anns books blue-Ann=250 blue-Bob=300 blue-Ann-Bob=225 Anns books =1000

Kees van Deemter, Dublin, Trinity College, May 2009 Whats the utility of the blue book? Compare expected search times 1.without this instruction 2.with this instruction 1.Without instruction: ½*1000 = With instruction: 9/10*(½*300) + 1/10*(300+( 1/2 *700)) = 200

Kees van Deemter, Dublin, Trinity College, May 2009 In Parikhs example, blue is crisp. Scenario can be generalised to situations where each allows boundary cases.

Kees van Deemter, Dublin, Trinity College, May 2009 Why is language vague? Barton Lipman (in A.Rubinstein, Economics and Language, CUP 2000; working paper Why is Language Vague (2006)) When/why does vague communication give higher payoff than crisp language?

Kees van Deemter, Dublin, Trinity College, May 2009 One type of answer: conflict S and H may have very different utility functions U S and U H (Crawford & Sobel 1982, Aragones and Neeman 2000): If U S and U H are very different, it can be advantageous to hide information –Our food is healthy! –Our burgers are big!

Kees van Deemter, Dublin, Trinity College, May 2009

One type of answer: conflict S and H may have very different utility functions U S and U H Crawford & Sobel 1982, Aragones and Neeman 2000: If U S and U H are very different, it can be advantageous to hide information –Our food is healthy! –Our burgers are big! Henceforth (following Lipman): U S = U H

Kees van Deemter, Dublin, Trinity College, May 2009 Lipmans questions When does vague communication give higher payoff than crisp language? Lipman: the airport scenario

Kees van Deemter, Dublin, Trinity College, May 2009

Lipmans scenario Example: Airport scenario: I describe Mr X to you, so you can pick up X from the airport. All I know is Xs height; heights are distributed across people uniformly on [0,1]. If you identify X right away, you get payoff 1; if you dont then you get payoff -1

Kees van Deemter, Dublin, Trinity College, May 2009 Lipman: the airport scenario What description would work best? Optimal communication: state Xs height as precisely as possible. If each of us knows Xs exact height then the probability of confusion is close to 0.

Kees van Deemter, Dublin, Trinity College, May 2009 Lipman: the airport scenario What description would work best? Optimal communication: state Xs height as precisely as possible. If each of us knows Xs exact height then the probability of confusion at the airport is close to 0. Lipman: This is not vague, because there are no boundary cases!

Kees van Deemter, Dublin, Trinity College, May 2009 Some possible answers to Lipman

Kees van Deemter, Dublin, Trinity College, May Necessary vagueness? Input may be vague. E.g.: –verbatim repetition (hearsay) –memory may cause details to fade (e.g., number of casualties in a disaster) –perception may have been inadequate (e.g., the height of a seated person)

Kees van Deemter, Dublin, Trinity College, May Necessary vagueness? Input may be vague. E.g.: –verbatim repetition (hearsay) This begs the question –memory may cause details to faded (e.g., number of casualties in a disaster) ? –perception may have been inadequate (e.g., the height of a seated person) Lipman: Why cant we convey exactly what our perception/memory is? E.g. 24 degrees +/- 2 degrees

Kees van Deemter, Dublin, Trinity College, May 2009 the hazards of measurement 11m 12m

Kees van Deemter, Dublin, Trinity College, May 2009 Example: One house of 11m height and one house of 12m height 1.the 12m house needs to be demolished 2.the tall house needs to be demolished Comparison is easier and more reliable than measurement prefer utterance 2 (Van Deemter 2006)

Kees van Deemter, Dublin, Trinity College, May 2009 Example: One house of 11m height and one house of 12m height 1.the 12m house needs to be demolished 2.the tall house needs to be demolished Comparison is easier and more reliable than measurement prefer utterance 2 But arguably, this utterance is not vague Its vagueness is merely local

Kees van Deemter, Dublin, Trinity College, May 2009 Apparent vagueness is frequent the tall house the tallest house Physical exercise is good for young and old regardless of age Bad for bacteria, good for gums gums improve as a result of bacterial death Fast-flowing rivers are deep the faster the deeper (positive correlation between variables)

Kees van Deemter, Dublin, Trinity College, May Production/interpretation Effort GTh can reason about the utility of an utterance Effort needs to be commensurate with utility. In many cases, more precision adds little benefit (cf. Prashant Parikh 2000, Van Rooij 2003, Jaeger 2008) E.g., the feasibility of an outing does not depend on whether its 20C or 30C. Mild takes fewer syllables than twenty three point seven five. –Vague words tend to be short (Krifka 2002)

Kees van Deemter, Dublin, Trinity College, May 2009 But: Why not round the figure? The temperature is 24C

Kees van Deemter, Dublin, Trinity College, May Evaluation payoff Example: You ask the doctor about your blood pressure. –Utterance 1: Your blood pressure is 150/90. –Utterance 2: Your blood pressure is high. U2 offers less detail than U1 But U2 also offers more: an evaluation of your condition. –A link with actions (cut down on salt, etc.) –Especially useful if metric is difficult

Kees van Deemter, Dublin, Trinity College, May 2009 –But why does English not have a (brief) expression that says Your blood pressure is 150/90 and too high? Compare You are obese means Your BMI is above 30 and this is dangerous.

Kees van Deemter, Dublin, Trinity College, May Lack of a good metric Maybe areas where there exists a generally accepted measurement are rare –Multidimensional measurements: Whats the size of a house? –Maths: How difficult is a proof? (As the reader may easily verify)

Kees van Deemter, Dublin, Trinity College, May Lack of a good metric Maybe areas where there exists a generally accepted measurement are rare –Multidimensional measurements: Whats the size of a house? –Maths: How difficult is a proof? (As the reader may easily verify) –How beautiful is a sunset?

Kees van Deemter, Dublin, Trinity College, May 2009

6.Future contingencies Indecent Displays Control Act (1981) forbids public display of indecent matter

Kees van Deemter, Dublin, Trinity College, May 2009 Indecent Displays Control Act (1981) forbids public display of indecent matter Indecent at the time the law has been parameterised (Waismann 1968, Hart 1994, Lipman 2006)

Kees van Deemter, Dublin, Trinity College, May Vagueness facilitates search

Kees van Deemter, Dublin, Trinity College, May Vagueness facilitates search Palace scenario: A diamond has been stolen from the Emperor. The thief must have been one of the 1000 eunuchs. The sole witness gets wounded and says, with his last breath, the thief is tall. What should the emperor do?

Kees van Deemter, Dublin, Trinity College, May 2009 What if tall is crisp? Separate eunuchs: tall / not tall tall 500 not tall 500 Expect to search ½*500=250 tall eunuchs What if witness regards more people as tall? Seach 500 tall ones not-tall ones No difference between the non-tall ones

Kees van Deemter, Dublin, Trinity College, May 2009 What if tall is recognised as having borderline cases? Separate: tall / not tall / borderline 1. tall borderline-tall not-tall 500 Assume: probability of being called tall is highest for (1) and lowest for (3).

Kees van Deemter, Dublin, Trinity College, May 2009 What if tall is recognised as having borderline cases? Separate: tall / not tall / borderline 1. tall borderline-tall not-tall 500 Assume: probability of being called tall is highest for (1) and lowest for (3). E.g., assuming the probabilities in the figure: expect to search 0.5*50+0.5( )=175 p = 0.5 p = 0

Kees van Deemter, Dublin, Trinity College, May 2009 What if tall has a continuity of degrees? This would allow the emperor to rank eunuchs by their height: (1) tallest (2) tallest but 1 (3) tallest but 2... (n) shortest Assume: probability of being called tall is highest for (1), second-highest for (2), etc. Emperor should search (1) first, then (2), etc.

Kees van Deemter, Dublin, Trinity College, May 2009 First lesson Continuous domains invite differences between subjects, regarding their interpretation thresholds Stubborn insistence on your own thresholds is foolish: flexibility pays Minimal flexibility: tall/not tall/borderline Maximal flexibility: continuum of degrees

Kees van Deemter, Dublin, Trinity College, May 2009 Second lesson The emperors three search strategies correspond with three logics of vagueness: Knowledge as ignorance: tall is really crisp; we just dont know where the threshold is. Partial Logic: true, false, truth value gap. Degree theories (e.g. Fuzzy Logic).

Kees van Deemter, Dublin, Trinity College, May 2009 Second lesson The emperors three search strategies correspond with three logics of vagueness: Knowledge as ignorance: tall is really crisp; we just dont know where the threshold is. Corresponds with 1 st strategy Partial Logic: true, false, truth value gap. Corresponds with the 2nd strategy Degree theories (e.g. Fuzzy Logic). Corresponds with 3 rd strategy

Kees van Deemter, Dublin, Trinity College, May 2009 Second lesson This suggests that Knowledge as Ignorance is probably wrong, and so is Partial Logic.

Kees van Deemter, Dublin, Trinity College, May 2009 Relevance for NLG We have focussed on the question Why is language vague? What does this mean for NLG? –Do our answers translate directly to the question when NLG systems should use vagueness?

Kees van Deemter, Dublin, Trinity College, May 2009 Vagueness triggers for NLG? 1.Necessary vagueness 2.Apparent vagueness 3.Cost reduction 4.Bias/evaluation 5.Lack of good metric 6.Future contingencies 7.Facilitation of search

Kees van Deemter, Dublin, Trinity College, May 2009 Vagueness triggers for NLG? 1.Necessary vagueness. Yes: When NLG systems take vague information as starting point. (e.g.: Colour(x)=red) 2.Apparent vagueness. Yes: vague descriptions (e.g. the figure on the left) 3.Cost reduction: Yes: the fact that English could have had a short expression for 24 degrees +/- 2 is irrelevant.

Kees van Deemter, Dublin, Trinity College, May 2009 Vagueness triggers for NLG? 4.Bias/evaluation. (Similar to 3.) the fact that English could have had an expression for these shoes cost 200 pounds and I disapprove of that is irrelevant. The generator can only choose between existing expressions: 200 pounds, and expensive

Kees van Deemter, Dublin, Trinity College, May 2009 Vagueness triggers for NLG? 5.Lack of good metric. Relevant to NLG in principle, given challenging applications. (E.g. estate adverts) 6.Future contingencies. Similar to (5). 7.Facilitation of search. Relevant(?) unless system defines its terms explicitly, removing differencs between subjects (e.g., Reiter et al. 2005, the word evening)

Kees van Deemter, Dublin, Trinity College, May 2009 Towards an algorithm In what follows, just a few factors have been taken into account Caveat: Any resemblance with empirically supported facts is accidental

Kees van Deemter, Dublin, Trinity College, May 2009 Towards an algorithm uncertainty precision evaluation cost total penalty payoff benefit 39.82C approx. 40C high fever

Kees van Deemter, Dublin, Trinity College, May 2009 Written material What Game Theory can do for NLG: the case of vague language. Proc. 12 th European workshop on Natural Language Generation (ENLG-2009)

Kees van Deemter, Dublin, Trinity College, May 2009 Not Exactly: In Praise of Vagueness Oxford University Press To appear Jan 2010 Part 1: vagueness in science and daily life Part 2: logic and vagueness Part 3: vagueness and AI

Kees van Deemter, Dublin, Trinity College, May 2009 Not Exactly: In Praise of Vagueness Oxford University Press To appear Jan 2010 Chapter 11: When to be vague?

Kees van Deemter, Dublin, Trinity College, May 2009 Payoff could become central to NLG New research by game theorists, economists, logicians, linguists (e.g., Lipman, Van Rooij, Jäger, de Jaegher, Aragones, Neeman)

Kees van Deemter, Dublin, Trinity College, May 2009 Lipman: Why is language vague? Tentative answers are beginning to emerge Do not confuse with the question When should an NLG system (or a human speaker) be vague? since this depends on what the language can express

Kees van Deemter, Dublin, Trinity College, May 2009 Lipman: Why is language vague? Tentative answers are beginning to emerge Do not confuse with the question When should an NLG system (or a human speaker) be vague? since this depends on what the language can express Compare De Saussure: langue/parole

Kees van Deemter, Dublin, Trinity College, May 2009 Thanks... to Ehud Reiter and Albert Gatt.