1. Rough schedule Multimodal, multi-party dialogue [30 min] D’Homme, SIRIDUS [10 min] –dialogues with networked devices in a smart house SRI demo (DM), (IBL robot (JB)) –negotiative dialogue –menu-based natural command dialogue (SL) –future work on TrindiKit (SL) Discussion: Dialogue genres [20 min] –reconfigurability (DM) –mixing infoseeking+command dialogue, switching domains dynamically (GoDiS demo) –more on building systems (incrementally) (SL) GoDiS and VoiceXML (SL): [10 min] General discussion (everyone): [20 min] –NL generation from infostates generation from DRSs (JB) generation of focus (SL) –Logics & proofs of dialogue, e.g.completeness of update rules –Real, practical applications: what are the problems –grammar based vs. statistical approaches (DM)

2. Future work on TrindiKit ESSLLI 2001

3. GUI (Graphical User Interface) Currently being developed Purpose –simplifying building of DS –pedagogical purposes –debugging Gives overview of –information state at various levels of dialogue –update rules –interactions between modules, infostate, and resources Eventually, build systems using GUI

4. Extended formalism improved infostate path language additional constructs for writing update algorithms –inspiration from SOAR, OZ, … extended typechecking & debugging facilities

5. Adding more readymade modules & interfaces include more powerful NL interpretation and generation modules –e.g. HPSG Build hook modules for various useful programs –planners, reasoning engines etc. –Any OAA module can be accessed from TrindiKit, so not limited to Prolog programs Resource interfaces for –SQL databases, –uPnP-specified devices, –etc.

6. Adding more speech modules New products come out all the time Actually using them in a system can be tricky and take lots of time ”Open Source”; users contribute –modularity promotes reuse –send us a link to your TrindiKit® software! –requires general solutions and explaining how module works

7. Future work on GoDiS ESSLLI 2001

8. Integrate with VoiceXML? Extended dialogue capabilities: Current –collaborative negotiation –multiple simultaneous plans –information sharing between plans Extended dialogue capabilities: Future –talking to autonomous robots –noncollaborative negotiation –complex planning tasks Implement more domains –no limits…

9. Typology of dialogues & dialogue systems

10. to what extent are dialogue types related by subsumption? To what extent are they orthoginal? –infoseeking < info exchange –instuctional, command < planning –negotiation meaning / dialogue strategy / domain collaborative / noncollaborative argumentative / non-argumentative

11. Typology of dialogues & dialogue systems Can dialogue types be defined in terms of theories / system features needed to handle them? –infostate contents –moves, rules Approach: build systems incrementally –succesively increase scope of dialogue phenomena handled by system –reuse of system components over dialogue types

12. Negotiative dialogue some definitions and ideas

13. Problem with current GoDiS assumes database always returns exactly one post (price info); not generally true but we want to be able to –talk about several flights, –allowing the user to ask questions about them, –deciding on one of them, and then –getting price information Requires negotiation

14. Negotiation vs. acceptance Clark’s ladder: –1. A attends to B’s utterance –2. A percieves B’s utterance –3. A understands B’s utterance (grounding) –4. A accepts or rejects B’s utterance Sidner and others sees negotiative dialogue as proposals and acceptance/rejections of proposals –this means that all dialogue is negotiative –all assertions (and questions, instructions etc.) are proposals But some dialogues are negotiative in another sense, by explicitly containing discussions about different solutions to a problem, and finally deciding on one –Negotiation is not Clark’s level 4

15. Two senses of “negotiation” Negotiation in Sidner’s sense –A: I want to go to Paris [propose] –B(1): OK [accept] –B(2): Sorry, there are no flights to Paris [reject] Negotiation in our sense –U: flights to paris on september 13 please [answer] –S: there is one flight at 07:45 and one at 12:00 [propose] –U: what airline is the 12:00 one [ask] –S: the 12:00 flight is an SAS flight [answer] –U: I’ll take the 12:00 flight please [accept]

16. Optimistic approach to acceptance DPs assume their utterances are accepted (and integrated into SHARED ) –If A asks a question with content Q, A will put Q topmost on SHARED.QUD If addresse indicates rejection, backtrack –using the PRIVATE.TMP field No need to indicate acceptance explicitly; it is assumed The alternative is a pessimistic approach –If A asks a question with content Q, A will wait for an acceptance (implicit or explicit) before putting Q on top of QUD

17. Negotiativity Negotiation is a type of problem-solving (cf. Di Eugenio et. al., Coconut) Negotiation: DPs discuss several alternative solutions before choosing one of them Negotiation does not imply conflicting goals –perhaps not 100% correspondence to everyday use of the word “negotiation”, but useful to keep collaborativity as a separate dimension from negotiation Both AOD and IOD can be negotiative –in a flight information service, the user does not become obliged to fly anywhere; so it’s IOD –but several different flights may be discussed

18. Negotiation tasks Some factors influencing negotiation –distribution of information between DPs –whether DPs must commit jointly (e.g. Coconut) or one DP can make the comittment (e.g. flight booking) We’re initially trying to model negotiation in flight booking –sample dialouge U: flights to paris on september 13 please S: there is one flight at 07:45 and one at 12:00 U: what airline is the 12:00 one S: the 12:00 flight is an SAS flight U: I’ll take the 12:00 flight please –Sys provides alternatives, User makes the choice –Sys knows timetable, User knows when he wants to travel etc.

19. Degrees of negotiativity non-negotiative dialogue: only one alternative is discussed semi-negotiative dialogue: a new alternative can be introduced by altering parameters of the previous alternative, but previous alternatives are not retained negotiative dialogue: several alternatives can be introduced, and old alternatives are retained and can be returned to

20. Semi-negotiative dialogue Does not require keeping track of several alternatives Answers must be revisable; this can be done using reraising of answered questions Correction of optimistic assumption of acceptance not necessarliy distinguished from revision Example: Swedish SJ system (Philips): ”Do you want an earlier or later train?”

21. Issues Under Negotiation i negotiative dialogue IUN is a question e.g. what flight to take In an activity, some questions are marked as negotiable issues; other questions are assumed to be non- negotiable Needs a new IS field: SHARED.IUN of type assocset(question,set(answer))

22. Alternatives in negotiation Alternatives are alternate answers to an IUN a proposal is the introduction of a new possible answer to IUN An IUN is resolved when an answer to it is given, i.e. when an alternative is accepted Alternatives and information about them is optimistically assumed to be accepted Alternatives are needed whenever database search can return more than one result

23. General and specific information General information concerns all alternatives, and is collected in an initial information- seeking dialogue (e.g. flights to paris) –e.g.  x.dest(x,Paris) Specific information concerns specific alternatives (e.g. flight f345 leaves at 10:45) Specific info usually results from a database search whose input is general info; does this motivate separate fields in IS?

24. Example IUN is x.sel_flight(x) (“which is the chosen flight”?) A: flight to paris, december 13 –answer(  x.dest(x,paris)) etc.; B: OK, there’s one flight leaving at 07:45 and one at 12:00 –propose(f1), propose(f2), –answer(dep_time(f1,07:45)), answer(dep_time(f2,12:00)) A: I’ll take the 07:45 one –answer(sel_flight(X), dep_time(X, 07:45)), –after contextual interpretation: answer(sel_flight(f1))

25. PRIVATE = PLAN = AGENDA = { findout(? x.sel_flight(x)) } SHARED = findout((? x. ccn(x)) book_ticket COM = dep_time(f1,0745), dep_time(f2,1200)  dest(paris),... QUD = <> LM = {propose(f1), propose(f2), answer(dep_time(f1,07:40),...} BEL = { flight(f1), dep_time(f1,0745),... } TMP = (same structure as SHARED) IUN = B: OK, there’s one flight leaving at 07:45 and one at 12:00

26. VoiceXML form-filling paradigm –forms, consiting of slots+values –menus, special case of forms speech recogntion grammars defined with scope over –slot –form –document Why is VoiceXML interesting for us? –becoming industry standard –supports plan constructs similar to GoDiS –multiple active grammars allow behaviour reminiscent of question and task accommodation

27. Dialogue capabilities VoiceXML: if input matches several fields, the first is chosen –GoDiS can ask clarification question VoiceXML: user initiated subdialogues cannot share information with main dialog, or other subdialogues –the default in GoDiS is that all information is shared between subdialogues unclear how to implement several simultaneously active plans

28. Architectural issues information state is global and keeps plan separate from accumulated propositions –VoiceXML based on forms, which can be seen as local information states VoiceXML mixes dialogue knowledge, domain knowledge, and language knowledge in a single specification –GoDiS keeps them separate, enabling easier reconfiguration and plug-and-play –when implement a new menu-driven system, there is no need to reimplement general dialogue strategies

29. future issues Open question whether VoiceXML can extend to more complex dialogue, e.g. negotiation –but the information state approach is ideal for complex dialogues –GoDiS is currently being extended to handle collaborative negotiation Can GoDiS be combined with VoiceXML? –have GoDiS use extended VoiceXML specifications, –or use standard VoiceXML specs in a smarter way

30. generating focus from information states

31. Accommodation Generality –no need to distinguish requested and non-requested (but relevant) information –single rule for integrating answers Theoretically motivated concept, with independent justification Easy to implement, given information state approach Gives useful side-effects (e.g. task accommodation leads to loading a plan) –supports the idea of accommodation as opposed to “direct interpretation”

32. Extensions Questions can also be reaccommodated –if the user answers a question which has already been answered: –remove proposition from shared beliefs, and put question back on QUD Extend to domain where there are many plans containing a question matching a given answer –e.g. menu-based dialogue Focus intonation and QUD

33. Focus and information states Focal Question Presupposition (FQP) (based on Ginzburg and others): –An utterance with narrow focus on a constituent presupposes a function/question obtained by abstracting over the focussed constituent Example: –“Jill likes BILL” [like(jill, bill)] presupposes “Who does Jill like?” [?x.like(jill, x)]

34. Focal Question Accommodation FQuAcc: interpretation version of FQP “When an utterance occurs which focally presupposes Q, and Q is not topmost on QUD, make Q topmost on QUD” pre: eff: in( SHARED.LM, Move) foc-presupp(Move, Q) ~fst( SHARED.QUD, Q) push( SHARED.QUD, Q)

35. Interpreting utterances with focus Example: A: Are you FLYING to london? ?(to-city(london)&transport(fly)) presupposes Q’ = ?(x.to-city(london)&transport(x)), i.e. “how are you getting to london” B1: Yes B2: No, I’m taking a TRAIN ?B3: No [Q’ still on QUD!]

36. Generating utterances with focus Generation version of FQP: –When generating Q, and there is a Q’ on QUD such that Q’ is an abstraction of Q over constituent C (Q=Q’(C)), put narrow focus on C

37. Generating clarifications with focus (cautious grounding) Example 1: –U: I want a flight please –S: What city do you want to go to? (Q’) –U: London –S: So you’re flying to LONDON? (Q) [Q=Q’(london); Q’ still on QUD, so put focus on “London”] Example 2: –U: I want to go to London –S: How do you want to travel? (Q’) –U: A flight please –S: So you’re FLYING to London? (Q)

38. Generating questions with focus II If –a question is to be asked, and –a parallell question has previously been answerd (there is a parallel information P in SHARED.COM), –put focus on the part that differentiates Q1 and P Example: –S: What CITY do you want to go to? –U: Paris.... –S: What city do you want to go FROM? –?x.to-city(x) parallell to ?x.from-city(x)

39. Generating questions with focus II (cont’d) Extension to focal question presupposition: –A question Q with narrow focus on C presupposes a parallell question Q’’ which differs from Q by having C replaced by some B Does accommodation apply? What about propositions? What, exactly, does “parallell” mean? (cf. Pulman 1998 for a formal account of parallellism for propositions)