1. Interactive Communication Management in an Issue- based Dialogue System DiaBruck 2003 Staffan Larsson Göteborg University, Sweden sl@ling.gu.se

2. Overview Interactive Communication Management (ICM) ”Verification” in dialogue systems Classifying and formalising feedback Feedback moves for GoDiS Issue-based grounding Formalising sequencing moves for GoDiS Conclusions & Future work

3. ICM (Allwood) Interactive Communication Management –As opposed to Own Communication Management (OCM): self-corrections, hesitations, etc. Feedback moves –(short) utterances which signal grounding status of previous utterance (”mm”, ”right”, ”ok”, ”pardon?”, ”huh?” etc.) Sequencing moves –utterances which signal dialogue structure (”so”, ”now”, ”right”, ”anyway” etc.) Turntaking moves

4. ICM in current commercial systems Usually, limited to ”verification” Examples (San Segundo et. al. 2001) –I understood you want to depart from Madrid. Is that correct? [”explicit v.”] –You leave from Madrid. Where are you arriving at? [”implicit v.”] Involves repetition or reformulation Appears in H-H dialogue, but not very common

5. From verification to ICM in dialogue systems ”Verification” is just one type of ICM behaviour –Perhaps the one most cruicial in dialogue systems given poor speech recognition Could a wider range of the ICM behaviour occurring in H-H dialogue be useful in dialogue systems? We want a typology of ICM moves for H-H dialogue –Feedback and sequencing moves We want to formalise it and use it in a system –Still we will implement only a subset We want to relate it to grounding in a system

6. Classifying feedback Level of action Polarity Eliciting or non-eliciting Form (syntactic realisation) Content type (object- or metalevel)

7. Feedback levels Action levels in dialogue (Allwood, Clark, Ginzburg) –Contact: whether a channel of communication is established –Perception: whether DPs are perciveving each other’s utterances –Understanding: Whether DPs are understanding each other’s utterances Non-contextual (”semantic”) meaning Contextual (”pragmatic”) meaning –Acceptance: Whether DPs are accepting each other’s utterances The function of feedback is to signal the status of utterance processing on all levels

8. Feedback polarity Polarity (Allwood et.al. 1992) –Positive: indicates contact, perception, understanding, acceptance –Negative: indicates lack of contact, perception, understanding, acceptance –We add a ”neutral” or ”checking” polarity – there is one or more hypotheses, but the DP lacks confidence in them Examples –”I don’t understand”: negative –”Do you mean that the destination is Paris?”: checking –”To Paris.”: positive –”Pardon”: negative

9. Formalising ICM dialogue moves Level –con: contact –per: perception –sem: semantic understanding (no context) –und: pragmatic understanding (relevance in context) –acc: acceptance Polarity –pos: positive –neg: negative –chk: checking

10. Feedback move notation icm:Level*Polarity{:Args} Examples –icm:per*pos:String – ”I heard you say ’londres’” –icm:und*neg – ”Sorry, I don’t understand” –icm:und*chk:AltQ – ”Do you mean x or y?” –icm:und*pos:P – ”To Paris.” –icm:acc*neg:Q – ”Sorry, I can’t answer Q” –icm:acc*pos – ”Okay”

11. GoDiS: an issue-based dialogue system Explores and implements Issue-based dialogue management (Larsson 2002) –Based on Ginzburg’s notion of a dialogue gameboard involving Questions Under Discussion (QUD) –Uses (mostly pre-scripted) dialogue plans Extends theory to more flexible dialogue –Multiple tasks, information sharing between tasks –ICM: feedback and grounding, sequencing –Question accommodation, re-raising, clarification –Inquiry-oriented, action-oriented, negotiative dialogue

12. System feedback for user utterances in GoDIS contact –negative (”I didn’t hear anything from you.”, ”Hello?”) [icm:con*neg] perception –negative: fb-phrase (”Pardon?”, ”I didn’t hear what you said”) [icm:per*neg] –positive: repetition (”I heard ’to paris’”) [icm:per*pos:String] semantic understanding: –negative: fb-phrase (”I don’t understand”) [icm:sem*neg] –positive: reformulation (”Paris.”) [icm:sem*pos:Content]

13. System feedback, cont’d pragmatic understanding –negative: fb-phrase (”I don’t quite understand”) [icm:und*neg] –positive: reformulation (”To Paris.”) [icm:und*pos:Content] –checking: reformulation (”To Paris, is that correct?”, ”To Paris?”) [icm:und*chk:Content] acceptance/integration –negative:fb-phrase with reformulation (”Sorry, I cannot answer Q”, ”Sorry, Paris is not a valid destination city.”) [icm:acc*neg:Content] –positive: fb-word (”okay.”) [icm:acc*pos]

14. User feedback for system utterances in GoDiS contact: - perception –negative: fb-phrase (”Pardon?”, ”I didn’t hear what you said”) [icm:per*neg] understanding: - acceptance/integration –negative: fb-phrase (”I don’t know”, ”Never mind”) [icm:acc*neg] –positive: fb-word (”okay.”) [icm:acc*pos]

15. Feedback selection Feedback polarity selected depending on: 1.Contact: Whether any speech is observed 2.Perception: Speech recognition score > threshold 3.Semantics: Whether system can find a non-contextual interpretation 4.Pragmatic understanding: Whether system can find a relevant contextual interpretation –none: extremely high score (certain of interpretation) –Positive: high score (pretty sure of interpretation) –Checking: medium score (not so sure) –Negative: low score (very unsure) 5.Acceptance: Whether system can accept what has been said General principles –Produce positive feedback only on highest possible level –When producing negative feedback on level N, produce positive feedback on level N-1

16. Feedback on multiple levels S> Lets see. What can I do for you? U> (mumble mumble) S> Sorry, I didnt hear what you said. What can I do for you? U> siridus S> I heard you say siridus. Sorry, I dont understand. What can I do for you? U> channel one S> channel one. I dont quite understand. Do you want to add a program or change channel? Negative perception feeback Positive perception Negative understanding Positive understanding (word meaning) Negative understanding (relevance) (Task clarification)

17. U> turn on the t v S> turn on the TV… Sorry, I cannot control the TV set. Do you want to add a program or change channel?” U> pardon S> Sorry, I cannot control the TV set. Do you want to add a program or change channel? U> change channel S> Okay. Rejection (negative acceptance feedback) Positive understanding User negative perception System repetition Positive acceptance

18. Reactions to positive and checking feedback After check feedback, –(e.g., ”channel one, is that correct?”) –positive response is needed, or system will assume it was wrong After positive feedback, –(e.g., ”channel one.”) –if user gives no response or positive response, system continues to assume it was right; –If user gives negative response, system assumes it was wrong correcting the system

19. Correcting the system S> Lets see. What can I do for you? U> add program S> Okay. Add a program… S> Lets see. What channel do you want? U> channel six S> OK, channel five. U> no S> not channel five. So, What channel do you want? U> channel six S> Okay. channel six.” Positive feedback Negative answer to fb.

20. Issue-based grounding Checking feedback on und-level explicitly raises understanding-issue –icm:und*chk:C raises ?und(C) ”Is C the meaning of the previous utterance?” –Positive response leads to adding C to shared commitments –Negative or no response leads to no action Positive understanding feedback does not raise the understanding-issue explicitly –In fact, tentatively assumes it has been positively resolved –but makes it available for interpretation of short answers in this case, ”yes” or ”no” –Negative reponse leads to retracting C –Positive or no response leads to no action So far, only implemented for understanding level

21. ISSUES, QUD and accommdation We distinguish QUD and ISSUES: –QUD (local): Questions available for resolution of short answers –ISSUES (global): Issues/questions that have been raised but not yet resolved QUD-to-ISSUES accommodation –If an answer is given which does not address any question on ISSUES, –But does address a question Q on QUD –Then, accommodate Q to ISSUES (this amounts to raising Q)

22. Feedback, ISSUES and QUD Assume previous move had content C Checking feedback –Push ?und(C) on ISSUES and QUD Positive feedback –Add C to shared commitments –Push ?und(C) on QUD (but not on ISSUES) –”yes” or ”no”  do QUD-to-ISSUES accommodation Integrating responses to understanding-issues –Done by a single rule, regardless of how the understanding-issue was raised –Given that ?und(C) is on ISSUES, –”yes”  add C to shared commitments if not already there –”no”  retract C from shared commitments if it’s there (actually, backtrack to saved previous shared commitments)

23. Negative response to positive feedback S> Lets see. What channel do you want? U> channel six S> OK, channel five. channel (5) added to shared commitments; ?und(channel (5)) on QUD U> no QUD-to-ISSUES accommodation  ?und(channel (5)) on ISSUES integrate ”no”  retract channel(5) S> not channel five. So, What channel do you want?

24. Negative response to cheking feedback S> Lets see. What can I do for you? U> add program S> Okay. Add a program… S> Lets see. What channel do you want? U> channel six S> channel five? ?und(channel (5)) on ISSUES and QUD U> no integrate ”no” ( do nothing) S> So, What channel do you want? U> channel six S> Okay. channel six.”

25. Sequencing ICM Discourse particles –”so” may be used to reraise a previously discussed question –”right”, ”anyway”, ”let’s see…” Verbose sequencing –”Let’s talk about …”, ”Let’s return to the issue of …” How do we classify these? –We correlate them with updates to the common ground

26. Sequencing moves generated by GoDiS Formal notation schema: –icm:Type{:Content} Loading a dialogue plan –icm:loadplan – ”let’s see.” Question accommodation –icm:accommodate – ”alright.” Reraising an issue –icm:reraise – ”so, ” –icm:reraise:Q – ”returning to the issue of …”

27. Sequencing ICM: reraising issue, loading a plan S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flight S> (…) What city do you want to go to? U> paris S> (…) What city do you want to go from? U> do I need a visa ? S> (…) Lets see. What country are you from? U> sweden S> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do you want to go from? Signals loading a plan Reraising an issue explicitly

28. Sequencing ICM: Accommodation S> Lets see. What can I do for you? U> channel five today S> Alright. add a program… Lets see. Okay. channel five… today… What time do you want to start recording? U> from six thirty to seven thirty S> Okay. from six thirty… to seven thirty… The program has been added. ” Signals accommodation

29. Future work Replace current ad-hoc forms based on corpus studies and existing work –”discourse markers” (Schiffrin) –”cue phrases” (Grosz & Sidner and others) ”Real” generation rather than templates –Integrate ICM with other utterances, rather than just appending them Explore other ways of generating ICM –Intonation, facial expression, etc. Extend the range of ICM generated (and interpreted) by the system Extend issue-based grounding to all levels

30. Conclusions By extending the range of ICM used by systems, their communication becomes more natural and comprehensive We have provided an initial classification of feedback and sequencing ICM useful in a dialogue system, and implemented it Issue-based grounding provides mechanisms allowing the user to react to system feedback Sequencing moves can be correlated with updates to common ground, and used to signal these updates to the user

31. Questions?

32. Relation to Traum’s computational theory of grounding Focus on positive feedback and corrections (self and other) –Deals with the question, when does a contribution end? Related to turntaking. –Focus on self- and other-corrections (not included here); involves turntaking and OCM, but also feedback –Does not include sequencing ICM –Based on the TRAINS corpus of H-H dialogue -> (arguably) focus on positive feedback Focus on understanding-level –”grounding” here refers only to the understanding level –Acceptance and rejection seen as ”core speech acts”

33. Object- or metalevel content Utterances with metalevel content explicitly refer to contact, perception, understanding or acceptance Object-level utterances instead refer to the task at hand Example –S: What city are you going to? –U: Paris –S(1a): Did you say you’re going to Paris? [meta] –S(1b): Are you going to Paris? [object] –S(2a): Do you mean Paris, France or Paris, Texas? –S(2b): Do you want to go to Paris, France or Paris, Texas? This dimension does not apply to all feedback, e.g. ”Paris.”, ”Pardon?” (Is 2b feedback or simply an alternative question?)

34. Realisation of feedback moves Syntactic form: –declarative: ”I didn’t hear what you said.”; ”The destination city is Paris.” –interrogative: ”What did you say?”; ”Do you want to go to Paris?” –imperative: ”Please repeat your latest utterance!” –elliptical interrogative: ”Paris?”, ”To Paris or from Paris?” declarative: ”To Paris.” In general, the exact formulation of ICM phrases may depend on various contextual factors –including activity, noise level, time constraints etc.

35. Eliciting / nonelciting feedback (Allwood et. al. 1992) Eliciting feedback is intended to evoke a response from the user Noneliciting feedback is not so intended –But may nevertheless recieve a response Rough correspondence / operationalisation –Checking feedback is eliciting; explicitly raises grounding issue –Positive feedback is noneliciting; may implicitly raise grounding issue What about negative feedback? –”pardon?”,”huh?”: eliciting? –”I didn’t hear you”: noneliciting?

36. Simplifying assumptions regarding feedback We only represent action level and polarity Eliciting/noneliciting dimension implicit –Negative feedback is eliciting in some sense; since something went wrong, it must be fixed –Checking feedback is also eliciting, since it poses a question that must be adressed –Positive feedback is not eliciting (we assume) Syntactic form not included in classification; decided by generation module Metalevel / object level perhaps not so interesting unless full compositional semantics are used –”Do you mean that you want to Paris?” vs. ”Do you want to go to Paris?”

37. Implicit feedback? Clark: ”relevant followup” to U counts as positive feedback –What is relevant? simple cases for followups to questions: –answer to question –”subquestion” –feedback concering question Complex cases: all other utterances –In general, complex inference and knowledge may be needed (implicatures) –Currently, irrelevant followup counts as negative feedback (a cautious assumption) What about no followup at all? –in reaction to ask-move or interrogative feedback, counts as negative –in reaction to answer or positive feedback, counts as positive

38. Rejection? S: ”Where do you want to go?” U1: ”Nowhere” U2: ”I don’t know” Should these count as rejections? –U1: negative answer? presupposition failiure? rejection? –U2: rejection? but not as definite as ”No comment!”

39. Grounding ”To ground a thing … is to establish it as part of common ground well enough for current purposes.” (Clark) making sure that the participants are percieving, understanding, and accepting each other’s utterances