Generating Feedback and Sequencing Moves in a Dialogue System AAAI Spring Symposium 2003 Staffan Larsson Göteborg University, Sweden

Overview Interactive Communication Management (ICM) ”Verification” in dialogue systems Classifying and formalising ICM ICM for a dialogue system Examples Conclusions & Future work

ICM (Allwood) Interactive Communication Management –As opposed to Own Communication Management (OCM): self-corrections, hesitations, etc. Feedback moves –(short) utterances which signal grounding status of previous utterance (”mm”, ”right”, ”ok”, ”pardon?”, ”huh?” etc.) Sequencing moves –utterances which signal dialogue structure (”so”, ”now”, ”right”, ”anyway” etc.) –Dialogue structure part of / modeled by common ground Turntaking moves

Grounding and ICM in current commercial systems Limited to ”verification” Examples (San Segundo et. al. 2001) –I understood you want to depart from Madrid. Is that correct? [”explicit v.”] –You leave from Madrid. Where are you arriving at? [”implicit v.”] Involves repetition or reformulation Appears in H-H dialogue, but not very common

From verification to ICM in dialogue systems ”Verification” is just one type of ICM behaviour –Perhaps the one most cruicial in dialogue systems given poor speech recognition Could a wider range of the ICM behaviour occurring in H-H dialogue be useful in dialogue systems? We want a typology of ICM moves for H-H dialogue –Feedback and sequencing moves We want to formalise it and use it in a system –Still we will implement only a subset, but more than verification

Classifying feedback Level of action Polarity Eliciting / noneliciting Form (syntactic realisation) Content type: object- or metalevel

Feedback levels Action levels in dialogue (Allwood, Clark, Ginzburg) –Contact: whether a channel of communication is established –Perception: whether DPs are perciveving each other’s utterances –Understanding: Whether DPs are understanding each other’s utterances Non-contextual (”semantic”) meaning Contextual (”pragmatic”) meaning –Acceptance: Whether DPs are accepting each other’s utterances The function of feedback is to signal the status of utterance processing on all levels

Feedback polarity Polarity (Allwood et.al. 1992) –Positive: indicates contact, perception, understanding, acceptance –Negative: indicates lack of contact, perception, understanding, acceptance –We add a ”neutral” or ”checking” polarity – there is one or more hypotheses, but the DP lacks confidence in them Examples –”I don’t understand”: negative –”Do you mean that the destination is Paris?”: checking –”To Paris.”: positive –”Pardon”: negative

Formalising ICM dialogue moves Action levels –con: contact –per: perception –sem: semantic understanding (no context) –und: pragmatic understanding (relevance in context) –acc: acceptance Polarity –pos –neg –chk (”int” in paper)

Feedback move notation icm:Level*Polarity{:Args} –icm:per*pos:String – ”I heard you say ’londres’” –icm:und*neg – ”Sorry, I don’t understand” –icm:und*chk:AltQ – ”Do you mean x or y?” –icm:und*pos:P – ”To Paris.” –icm:acc*neg:Q – ”Sorry, I can’t answer Q” –icm:acc*pos – ”Okay”

System feedback for user utterances in GoDIS contact –negative (”I didn’t hear anything from you.”, ”Hello?”) [icm:con*neg] perception –negative: fb-phrase (”Pardon?”, ”I didn’t hear what you said”) [icm:per*neg] –positive: repetition (”I heard ’to paris’”) [icm:per*pos:String] semantic understanding: –negative: fb-phrase (”I don’t understand”) [icm:sem*neg] –positive: reformulation (”Paris.”) [icm:sem*pos:Content] –checking: reformulation (”Paris, is that correct?”, ”Paris?”) [icm:sem*chk:Content]

System feedback, cont’d pragmatic understanding –negative: fb-phrase (”I don’t quite understand”) [icm:und*neg] –positive: reformulation (”To Paris.”) [icm:und*pos:Content] –checking: reformulation (”To Paris, is that correct?”, ”To Paris?”) [icm:und*chk:Content] acceptance/integration –negative:fb-phrase with reformulation (”Sorry, I cannot answer Q”, ”Sorry, Paris is not a valid destination city.”) [icm:acc*neg:Content] –positive: fb-word (”okay.”) [icm:acc*pos]

User feedback for system utterances in GoDiS contact: - perception –negative: fb-phrase (”Pardon?”, ”I didn’t hear what you said”) [icm:per*neg] understanding: - acceptance/integration –negative: fb-phrase (”I don’t know”, ”Never mind”) [icm:acc*neg] –positive: fb-word (”okay.”) [icm:acc*pos]

Feedback selection Feedback polarity selected depending on (for each level respectively): –Contact: Whether any speech is observed –Perception: Speech recognition score –Semantics: Whether system can find a non- contextual interpretation –Pragmatic understanding: Whether system can find a relevant contextual interpretation –Acceptance: Whether system can accept what’s been said

Feedback on multiple levels S> Lets see. What can I do for you? U> (mumble mumble) S> Sorry, I didnt hear what you said. What can I do for you? U> siridus S> I heard you say siridus. Sorry, I dont understand. What can I do for you? U> channel one S> channel one. I dont quite understand. Do you want to add a program or change channel? Negative perception feeback Positive perception Negative understanding Positive understanding (word meaning) Negative understanding (relevance) (Task clarification)

U> turn on the t v S> turn on the TV… Sorry, I cannot control the TV set. Do you want to add a program or change channel?” U> pardon S> Sorry, I cannot control the TV set. Do you want to add a program or change channel? U> change channel S> Okay. Rejection (negative acceptance feedback) Positive understanding User negative perception System repetition Positive acceptance

Sequencing ICM Discourse particles –”so” may be used to reraise a previously discussed question –”right”, ”anyway”, ”let’s see…” Verbose sequencing –”Let’s talk about …”, ”Let’s return to the issue of …” How do we classify these? –We will correlate them with updates to the common ground

Sequencing moves generated by GoDiS Formal notation schema: –icm:Type{:Content} Loading a plan –icm:loadplan – ”let’s see.” Question accommodation –icm:accommodate – ”alright.” Reraising an issue –icm:reraise – ”so, ” –icm:reraise:Q – ”returning to the issue of …”

Sequencing ICM: reraising issue, loading a plan S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flight S> (…) What city do you want to go to? U> paris S> (…) What city do you want to go from? U> do I need a visa ? S> (…) Lets see. What country are you from? U> sweden S> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do you want to go from? Signals loading a plan Reraising an issue explicitly

Sequencing ICM: Accommodation S> Lets see. What can I do for you? U> channel five today S> Alright. add a program… Lets see. Okay. channel five… today… What time do you want to start recording? U> from six thirty to seven thirty S> Okay. from six thirty… to seven thirty… The program has been added. ” Signals accommodation

Conclusions Current dialogue systems use only a fraction of the range of ICM used by humans By extending the range of ICM used by systems, their communication becomes more natural and comprehensive We have provided an initial classification of feedback and sequencing ICM useful in a dialogue system, and implemented it (ICM can be correlated with updates to common ground, and used to signal these updates to the user)

Future work Replace current ad-hoc forms based on corpus studies and existing work –”discourse markers” (Schiffrin) –”cue phrases” (Grosz & Sidner and others) ”Real” generation rather than templates –Integrate ICM with other utterances, rather than just appending them Explore other ways of generating ICM –Intonation, facial expression, etc. Extend the range of ICM generated (and interpreted) by the system

Questions?

Object- or metalevel content Utterances with metalevel content explicitly refer to contact, perception, understanding or acceptance Object-level utterances instead refer to the task at hand Example –S: What city are you going to? –U: Paris –S(1a): Did you say you’re going to Paris? [meta] –S(1b): Are you going to Paris? [object] –S(2a): Do you mean Paris, France or Paris, Texas? –S(2b): Do you want to go to Paris, France or Paris, Texas? This dimension does not apply to all feedback, e.g. ”Paris.”, ”Pardon?” (Is 2b feedback or simply an alternative question?)

Realisation of feedback moves Syntactic form: –declarative: ”I didn’t hear what you said.”; ”The destination city is Paris.” –interrogative: ”What did you say?”; ”Do you want to go to Paris?” –imperative: ”Please repeat your latest utterance!” –elliptical interrogative: ”Paris?”, ”To Paris or from Paris?” declarative: ”To Paris.” In general, the exact formulation of ICM phrases may depend on various contextual factors –including activity, noise level, time constraints etc.

Correcting the system S> Lets see. What can I do for you? U> add program S> Okay. Add a program… S> Lets see. What channel do you want? U> channel six S> OK, channel five. U> no S> not channel five. So, What channel do you want? U> channel six S> Okay. channel six.” (dialog4_edit) Interrogative feedback Negative answer to fb. Signals reraising an issue (Sequencing ICM)

Reactions to positive and checking feedback After positive feedback, system leaves a pause for the user to agree or disagree –if user gives no response, system assumes it was right After check feedback, positive response is needed –or system will assume it was wrong

Implicit feedback? Clark: ”relevant followup” to U –What is relevant? simple cases for followups to questions: –answer to question –”subquestion” –feedback concering question Complex cases: all other utterances –In general, complex inference and knowledge may be needed (implicatures) –Currently, irrelevant followup counts as negative feedback What about no followup at all? –in reaction to ask-move or interrogative feedback, counts as negative –in reaction to answer or positive feedback, counts as positive

Rejection? S: ”Where do you want to go?” U1: ”Nowhere” U2: ”I don’t know” Should these count as rejections? –U1: negative answer? presupposition failiure? rejection? –U2: rejection? but not as definite as ”No comment!”

Relation to Traum’s computational theory of grounding Focus on understanding-level –”grounding” here refers only to the understanding level –Acceptance and rejection seen as ”core speech acts” Focus on positive feedback and corrections (self and other) –Based on the TRAINS corpus of H-H dialogue –Deals with the question, when does a contribution end? –Corrections not included here; involves turntaking and OCM Does not include sequencing ICM

GoDiS: an issue-based dialogue system Explores and implements Issue-based dialogue management (Larsson 2002) –Based on Ginzburg’s notion of a dialogue gameboard involving Questions Under Discussion (QUD) –Uses (mostly pre-scripted) dialogue plans Extends theory to more flexible dialogue –Multiple tasks, information sharing between tasks –Feedback and grounding –Question accommodation, re-raising, clarification –…

Eliciting / nonelciting feedback (Allwood et. al. 1992) Eliciting feedback is intended to evoke a response from the user Noneliciting feedback is not so intended –But may nevertheless recieve a response

Simplifying assumptions regarding feedback We only represent action level and polarity In polarity, we replace ”neutral” by ”checking” –We exclude feedback which is neutral but not check- questions Eliciting/noneliciting dimension implicit –Negative feedback is eliciting; since something went wrong, it must be fixed –Checking feedback is also eliciting, since it poses a question that must be adressed –Positive feedback is not eliciting (we assume) Syntactic form not included

Grounding ”To ground a thing … is to establish it as part of common ground well enough for current purposes.” (Clark) making sure that the participants are percieving, understanding, and accepting each other’s utterances