2. 1 Issue-based Dialogue Management Staffan Larsson 2003

3. 2 overview of contents 1.Introduction 2.Basic issue-based dialogue management 3.Grounding and feedback 4.Adressing Unraised Issues 5.Action-oriented Dialogue 6.Multilinguality 7.Conclusions

4. 3 1. Introduction, goals explore and implement issue-based dialogue management –starting from Ginzburg’s theory of dialogue semantics based on notion of QUD (Questions Under Discussion) –adapt to dialogue system (GoDiS) and implement –extend theory coverage, taking in relevant theories general theory of dialogue –minimize effort for adapting dialogue system to new domains incrementally extending system to handle increasingly complex types of dialogue –clarifies relation between dialogue genres –promotes reuse of update rules Larsson (2002): Issue-based Dialogue Management (PhD Thesis)

5. 4 GoDiS: an issue-based dialogue system Built using TrindiKit –Toolkit for implementing and experimenting with dialogue systems based on the information state approach Explores and implements issue-based dialogue management Extends theory to more flexible dialogue –Multiple tasks, information sharing between tasks –Feedback and grounding –Accommodation, re-raising, clarification –Menu based action oriented dialogue –Multi-linguality & mutiple domains

6. 5 information state –an abstract data structure (record, DRS, set, stack etc.) –accessed by modules using conditions and operations dialogue moves are associated with IS updates using IS update rules –rule name and class –preconditon list: conditions on TIS –effect list: operations on TIS dialogue move engine –module or group of modules responsible for –updating the IS based on observed moves –selecting moves to be performed Information State Approach

7. 6 input inter- pret TIS DATABASE LEXICON DOMAIN data- base control updateselect gene- rate output lexicon domain knowledge DME

8. 7 TrindiKit GoDiS GoDiS-I GoDiS-A Travel Agency Auto- route Xerox manual VCR manager IBDM home device manager IS approach genre- specific applicatio n-specific

9. 8 2. Issue-based dialogue management enquiry-oriented dialogue (database search) basis: –Ginzburg’s Dialogue Gameboard (DGB) and –related DGB update protocols dialogue moves: ask, answer, greet, quit raising and addressing issues –incl. short answers. e.g.”yes”, ”no”, ”paris”, ”in april” dialogue plans sample domain: travel agency extension: –reraising issues –handling multiple issues

10. 9 Semantics simple First Order Logic without quantifiers, but with questions questions –Y/N-questions: ?P, P is a proposition –wh-questions: ?x.p(x) (p is a predicate) ? works much like like –alt-questions: {?P1, …, ?Pn} Content of short answers –individual markers: paris, april, … –yes, no

11. 10 Semantics, cont’d Q-A relations (adapted from Ginzburg) –resolves(A,Q): A resolves Q dest-city(paris) resolves ?x.dest-city(x) –relevant(A,Q): A is relevant to Q (about Q) not(dest-city(paris)) is relevant to ?x.dest-city(x), but does not resolve it

12. 11 basic GoDiS information state record type PRIVATE : PLAN : stack( Action ) AGENDA : OpenQueue( Action ) SHARED : BEL : set( Prop ) COM : set( Prop ) QUD : stack( Question ) LU: SPEAKER: Speaker MOVES: OQueue( Move )

13. 12 sample dialogue plan <findout(?x.transport(x)) findout(?x.dest-city(x)) findout(?x.depart-city(x)) findout(?x.dept-month(x)) findout(?x.dept-day(x)) raise( {?class(economy), ?class(business)} consultDB(?x.price(x)) >

14. 13 Answer integration integrateAnswer update rule Before an answer can be integrated by the system, it must be matched to a question on QUD pre: eff: in($/ SHARED/LU/MOVES, answer(A)) fst($/ SHARED/QUD, Q) $ DOMAIN :relevant(A, Q) ! DOMAIN: combine(Q, A, P) add(/ SHARED/COM, P)

15. 14 basic dialogue with updates U: ”price information please”; raises price issue –if user asks Q, push respond(Q) on AGENDA –if respond(Q) on AGENDA and PLAN empty, find plan for Q and load to PLAN –if findout(Q) first on PLAN, ask Q S: ”where do you want to go?” U: ”Paris” –if LM=answer(A) and A relevant to Q, add P=Q[A] to SHARED.COM –if P in SHARED.COM and Q topmost on QUD and P resolves Q, pop QUD –if P in SHARED.COM and P fulfils goal of findout(Q) and findout(Q) on PLAN, pop PLAN

16. 15 basics cont’d … S: ”Do you want economy class or business class?” U: ”economy class” –if consultDB(Q) on PLAN, consult database for answer to Q; store result in PRIVATE.BEL –if Q on QUD and P in PRIVATE.BEL s.t. P resolves Q, answer(P) S: ”The price is £123”

17. 16 Information sharing across plans GoDiS does not keep track of when propositions were added, or which plan was being executed so information sharing is determined by question sharing across plans plan for VISA question: findout(?x.dest-city(x)) findout(?x.citizenship(x)) –shares a questions with plan for ?x.price(x) so if visa-issue raised after price-issue, no need to ask for destination again

18. 17 dealing with multiple open issues if user asks Q, push Q on QUD and load plan for dealing with Q if users asks Q’ while system is dealing with Q, throw out plan for Q but Q remains on QUD; load plan for Q’ when Q’ resolved, Q topmost on QUD will trigger reloading plan for dealing with Q –general rule: if SHARED.COM contains info resolving Q, don’t ask Q –so any resolved questions in plan will be thrown out

19. 18 Sample dialogue: multiple tasks & info sharing S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flight S> (…) What city do you want to go to? U> paris S> (…) What city do you want to go from? U> do I need a visa ? S> (…) Lets see. What country are you from? U> sweden S> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do you want to go from?

20. 19 3. Grounding and feedback Grounding –making sure that the participants are percieving, understanding, and accepting each other’s utterances –dealing with problematic situtations where e.g. an utterance is not percieved Feedback moves –(short) utterances which signal grounding status of previous utterance Sequencing moves –utterances which signal switching task (”returning to…”), reraising questions (”so…”) etc.

21. 20 Grounding moves We want a typology of grounding moves –Both feedback and sequencing Some parameters: –Action levels in dialogue (Allwood, Clark) contact perception Understanding –Word meanings –Contextual meaning acceptance –Polarity Positive Negative Interrogative (asking for confirmation)

22. 21 Some grounding moves in GoDiS Formal representation –icm:Level/Type{*Polarity}{:Content} Feedback moves –icm:und*neg – ”I don’t understand” –icm:und*pos:P – ”To Paris.” –icm:acc*neg:Q – ”Sorry, I can’t …” –icm:acc*pos – ”Okay” Feedback type selected depending on –Quality of recognised speech –Whether system can find a (relevant) interpretation –Whether system can accept what’s been said Sequencing moves –icm:reraise:Q – ”Returning to the issue of Q” –icm:loadplan – ”Let’s see…”

23. 22 Grounding on multiple levels S> Lets see. What can I do for you? U> (mumble mumble) S> Sorry, I didnt hear what you said. What can I do for you? U> siridus S> I heard you say siridus. Sorry, I dont understand. What can I do for you? U> channel one S> channel one. I dont quite understand. Do you want to add a program or change channel? (dialog3a_edit) Negative perception feeback Positive perception Negative understanding Positive understanding (word meaning) Negative understanding (relevance) Task clarification

24. 23 U> turn on the t v S> turn on the TV… Sorry, I cannot control the TV set. Do you want to add a program or change channel?” U> pardon S> Sorry, I cannot control the TV set. Do you want to add a program or change channel? U> change channel S> Okay. (dialog3b_edit) Rejection (negative acceptance feedback) Positive understanding User negative perception System repetition Positive acceptance

25. 24 Correcting the system S> Lets see. What can I do for you? U> add program S> Okay. Add a program… S> Lets see. What channel do you want? U> channel six S> channel five? U> no S> not channel five. So, What channel do you want? U> channel six S> Okay. channel six.” (dialog4_edit) Interrogative feedback Negative answer to fb. Signals reraising (Sequencing ICM)

26. 25 4: Addressing Unraised Issues (QUD and ISSUES) General strategy: question accommodation issue accommodation QUD accommodation dependent issue accommodation issue reacommodation

27. 26 (A problem with QUD If QUD= and q1 is resolved, q2 is available for resolution of short answers –takes no account of how many turns since q2 was raised –but short answers a long distance away from the question are not as easily processed as an adjacent answer

28. 27 ISSUES and QUD We extend Ginzburg’s DGB by adding ISSUES of type Stack(Question) ISSUES contains all raised but unresolved questions –ISSUES determines relevance of user answers QUD used for resolving short answers –questions drop off QUD after N turns –a short answer to a question that’s on ISSUES but not QUD requires adjusting QUD by copying a question on ISSUES )

29. 28 Typical human-human dialogue S(alesman), C(ustomer) S: hi C: flights to paris S: when do you want to travel? C: april, as cheap as possible...

30. 29 Accommodation Lewis (1979): If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t Has been applied to referents and propositions, as parts of the conversational scoreboard / information state

31. 30 Question accommodation If questions are part of the information state, they too can be accommodated If the latest move was an answer, and there is an action in the plan to ask a matching question, then –put that question on ISSUES –(and QUD if it is a short answer) Requires that the number of possible matching questions is not too large –(or can be narrowed down by asking clarification question)

32. 31 issue accommodation PLANISSUES If –LM=answer(A) –no Q in ISSUES s.t. about(A,Q) then –find findout(Q) in PLAN s.t. about(A,Q) –push Q on ISSUES used when prevously unraised question (available in plan) is answered using a short or full answer

33. 32 QUD accommodation ISSUESQUD If –LM=answer(A) –no Q in QUD s.t. about(A,Q) then –find Q in ISSUES s.t. about(A,Q) –push Q on QUD –raise Q in ISSUES (make Q topmost) used when –previously raised question has dropped off QUD, but is answered using a short answer –previously unraised question is answered using short answer [needs PLANISSUES accommodation]

34. 33 dependent issue accommodation DOMAIN  ISSUES (+PLAN) If –LM=answer(A) –no Q in ISSUES s.t. about(A,Q) –no findout(Q) in PLAN s.t. about(A,Q) then –find Plan for some Q’ in DOMAIN s.t. findout(Q) or raise(Q) in Plan and about(A, Q) –push Q’ on ISSUES –set PLAN to Plan used when previously unraised question, unavailable in PLAN, is answered using full or short answer (AKA ”task accomodation”)

35. 34 Sample dialogue: accommodation S: Welcome to the travel agency. U: From London to Paris in April –not relevant to any question that has been raised, or to any current task –look in domain knowledge for a plan (for dealing with some question Q) with matching questions –load this plan, push Q on ISSUES –find in the plan the question(s) matching the user’s answer –integrate answer (requres matching question on ISSUES) S: Alright, you want to know about price. (…) –proceed to next plan item S: How do you want to travel? –ISSUES=

36. 35 Task clarification (VCR application) S> What can I do for you? U> channel five S> channel five. I dont quite understand. Do you want to add a program or change channel? U> change channel S> Okay. change channel. S> Lets see. channel five? U> yes S> The channel was changed

37. 36 Issue reraising SHARED.COM  ISSUES If –LM=ask(Q) –there is a P in SHARED.COM s.t. relevant(P,Q) then –push Q on ISSUES –remove P from SHARED.COM used when previously resolved question is asked again reraising should be inicated; ”so,…”; reformulation may be needed

38. 37 Issue reaccommodation SHARED.COM  ISSUES If –LM=answer(A) –no Q in ISSUES s.t. about(A,Q) –P in SHARED.COM s.t. there is a Q s.t. about(A,Q) and about(P,Q) then –push Q on ISSUES –remove P from SHARED.COM Used when previously resolved question is answered again

39. 38 Revision by reaccommodation and reraising S> What can I do for you? U> add program channel five S> Okay. Do you want to add a program? U> yes S> Lets see. channel five? U> yes S> What date? U> channel six S> Okay. channel six… What date?

40. 39 5: Issues and goals in Action- Oriented Dialogue (AOD) each goal action associated with a dialogue plan add field –SHARED.ACTIONS : StackSet(Action) New moves: request(Action), report(Action, Status) ACTIONS has a similar role to ISSUES adapt accommodation strategies to AOD

41. 40 AOD information state PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stackset( Question ) ACTIONS : stackset( Action ) LU:… ISSUES: stackset(Question)

42. 41 Requests vs. answers requests adresses a general question –”what shall I do next?”, ”what can I do for you?” or similar (”prompt”) –semantics in AOD: ?x.action(x) dialogue –”What can I do for you?” raises?x.action(x) –”Search the phonebook” interpreted as request(search_phonebook) Rule: –If ?x.action(x) is topmost on ISSUES and L- M is request(a) –then pop ISSUES

43. 42 Questions vs. answers a question can be regarded as a special type of request so in a specific sense, questions are also answers… questions adresses a general question –”what issue shall I address next?”, ”what can I do for you?” or similar (”prompt”) –semantics: ?x.issue(x) –this question is not presupposed semantics of ”How can I help you?” in IOD is ask(?x.issue(x))

44. 43 Action Oriented Dialogue subtypes ”Command dialogue” –user instructs system to perform actions device control; menu-based travel agency booking ”Instructional dialogue” –system instructs user to perform actions ”interactive manual” –system can explaing how to perform some action joint planning –user and system jointly agree to a plan –also involves negotiation

45. 44 input nuance inter- pret Information State LEXICON DOMAIN DEVICES lexicon vcrphone control updateselect gene- rate output nuance device phone domain vcrphone device vcr lexicon home domain home rel1 rel2 dim10 VCR device alert

46. 45 Converting menus to plans MenuPlan multi-choice list P 1, P 2,..., P n findout({?P 1,?P 2,...,?P n }) tick-box +/-Pfindout(?P) dialogue window p=___ findout(?x.p(x)) pop-up message M inform(M)

47. 46 Menu-based AOD S: Welcome to the telephone and VCR manager! U: Channel one today S: Right, I assume you want to add a program. Let’s see. Okay. Channel one. Today. What time do you want to start recording? S: What time is it? U: It’s one forty five S: Set the clock to two forty five U: Okay, set the clock. Two forty five, is that correct? S: Yes. U: Okay. The clock was set. Returning to timer recording. So, what time do you want to start recording? U: eight thirty to nine thirty S: Okay. Starting at eight thirty. Ending at nine thirty. The program has been added.

48. 47 Task and question accommodation in AOD S> Lets see. What can I do for you? U> channel five today S> Alright. add a program… Lets see. Okay. channel five… today… What time do you want to start recording? U> from six thirty to seven thirty S> Okay. from six thirty… to seven thirty… The program has been added. ” Signals accommodation (sequencing ICM)

49. 48 7. Multilinguality and multiple domains Language and domain resources can be switched on the fly Does not affect the rest of the system Thanks to modularity structure of system

50. 49 Multilinguality S> Lets see. What can I do for you? U> add program S> Okay. Do you want to add a program? U> yes S> Lets see. What channel do you want? U> svenska S> Okej. S> Gå tillbaks till lägg till ett program. Vilken kanal vill du spela in? U> kanal ett S> kanal ett, är det korrekt? S> Så, Vilken kanal vill du spela in? U> english S> Okay...

51. 50 Multiple domains S> Welcome to the VCR manager! S> Lets see. What can I do for you? U> change domain S> Okay. change domain… Lets see. What domain do you want? U> telephone S> Okay. This is your automatic telephone operator… Lets see. Please specify a function. U> call S> Okay. make a phone call. S> Lets see. Please specify the destination of the call. U> luis S> Okay. Luis… Calling

52. 51 8. Conclusions: GoDiS & Issue- Based Dialogue Management general solutions to –dealing with multiple tasks –sharing information between tasks –grounding and feedback –user initiative (accommodation) –menu-based dialogue rapid prototyping of applications –dialogue plans switching language and domain online

53. 52 Current and future work Extend to more complex dialogue types –Negotiation (theory exists, not yet implemented) –Tutorial dialogue –… Explore use of QUD and ISSUES to assign proper focus intonation –Stina Ericsson, forthcoming PhD thesis Build VCR control application, available on the phone –Users can phone up and get TV programs recorded on a computer –Collect corpus to use as basis for further developments Integrate with type-theoretical situation semantics (cf. Robin’s talk)

54. 53 Relation to plan-based approaches IBDM is compatible with, and can (should) be extended to include plan-based approaches –Task accommodation requires plan recognition; currently very simple (but task accommodation is not included as such in plan- based approaches) IBDM covers some dialogue-specific phenomena not usually covered in plan-based approaches –Short answers –Grounding and feedback –Accommotion However, some plan-based accounts cover some of the phenomena also covered by IBDM –We believe the issue-based account is simpler and thus more appropriate for simple kinds of dialogue –Plan-based approaches often unnecessarily complex for simple kinds of dialogue –We want to explore the middle ground beteen simple and complex accounts of dialogue