Staffan Larsson & Annie Zaenen / October 2000 / page 1 / Document Transformations and Information States Document Transformations and Information States Staffan Larsson, Annie Zaenen SIGdial workshop, ACL 2000, Hong Kong

Staffan Larsson & Annie Zaenen / October 2000 / page 2 / Goals of this talk sketch a model of the relation between monologue and dialogue in instructional discourse, based on the TRINDI information state approach investigate possibilities of using domain task plans to generate discourse with various degrees of interactivity adapt an existing TrindiKit dialogue system, designed for information-seeking dialogue, to handle instructional dialogue and monologue NB. We are not dealing with educational (a.k.a. instructional) dialogue

Staffan Larsson & Annie Zaenen / October 2000 / page 3 / Overview of talk motivationmotivation monologue and dialoguemonologue and dialogue the TRINDI information state approachthe TRINDI information state approach IMDiS (Instructional Monologue and Dialogue System)IMDiS (Instructional Monologue and Dialogue System) from domain plan to monologue and dialogue plansfrom domain plan to monologue and dialogue plans monologue and dialogue behaviour in IMDiSmonologue and dialogue behaviour in IMDiS conclusions & prospectsconclusions & prospects

Staffan Larsson & Annie Zaenen / October 2000 / page 4 / Practical motivation for document transformation Practical advantages of being able to adapt the “same” document to different uses and mediaPractical advantages of being able to adapt the “same” document to different uses and media Costly to develop presentations from scratch for each new mediaCostly to develop presentations from scratch for each new media

Staffan Larsson & Annie Zaenen / October 2000 / page 5 / Theoretical issues Written discourse and human-human dialogue are the extremes of the scale of interactivityWritten discourse and human-human dialogue are the extremes of the scale of interactivity We explore some intermediate levels of interactivity using the TRINDI information state approachWe explore some intermediate levels of interactivity using the TRINDI information state approach How can we model the differences between these levels, independently of domain-specific knowledge?How can we model the differences between these levels, independently of domain-specific knowledge? Concepts:Concepts: –interactivity in monologue and dialogue –interactivity vs. initiative –spoken vs. written discourse –maxim of quantity in monologue and dialogue

Staffan Larsson & Annie Zaenen / October 2000 / page 6 / Degrees of interactivity Dialogue is interactive, since participants can influence each other’s movesDialogue is interactive, since participants can influence each other’s moves Monologue is typically non-interactiveMonologue is typically non-interactive Written monologue is minimally interactive; reader can influence order & speed of readingWritten monologue is minimally interactive; reader can influence order & speed of reading highnone written monologue human-human dialogue spoken monologue inter- activity minimal

Staffan Larsson & Annie Zaenen / October 2000 / page 7 / Interactivity vs. initiative Interactivity does not imply shared initiative; it is possible to have a dialogue with both task and dialogue initiative staying on one sideInteractivity does not imply shared initiative; it is possible to have a dialogue with both task and dialogue initiative staying on one side In instructional dialogue, task initiative is typically held by instructor; dialogue initiative may shiftIn instructional dialogue, task initiative is typically held by instructor; dialogue initiative may shift

Staffan Larsson & Annie Zaenen / October 2000 / page 8 / Spoken vs. written instructional discourse separate dimension from interactivity (and initiative)separate dimension from interactivity (and initiative) written monologue has advantage of providing minimal interactivitywritten monologue has advantage of providing minimal interactivity spoken monologue (e.g. tape recording) has advantage of leaving hands and eyes free to do other thingsspoken monologue (e.g. tape recording) has advantage of leaving hands and eyes free to do other things –prerecorded repetitions and reformulations can be used as a substitute for interactivity

Staffan Larsson & Annie Zaenen / October 2000 / page 9 / Quantity of information in monologue manuals are cooperative, and are based on implicit assumptions about the readermanuals are cooperative, and are based on implicit assumptions about the reader setting: instructor has know-how information which instructee does notsetting: instructor has know-how information which instructee does not all information, and preferably only that information, which is needed by the instructee to complete the task should be transferred (Grice’s maxim of quantity)all information, and preferably only that information, which is needed by the instructee to complete the task should be transferred (Grice’s maxim of quantity) –however, if education is an issue, additional information (explanations) should be supplied in manuals, instructor must keep provide more information than may be necessary for any given instructee (cf. Young ’97, how to generate text from dynamically generated plans which are very detailed)in manuals, instructor must keep provide more information than may be necessary for any given instructee (cf. Young ’97, how to generate text from dynamically generated plans which are very detailed)

Staffan Larsson & Annie Zaenen / October 2000 / page 10 / Quantity of information in dialogue interactivity makes it possible for instructee to control the amount of information given by the instructorinteractivity makes it possible for instructee to control the amount of information given by the instructor instructee must be able to inform instructor about her state, thus influencing the instructors movesinstructee must be able to inform instructor about her state, thus influencing the instructors moves instructor needs to keep track of the state of the instructee to determine what information to provideinstructor needs to keep track of the state of the instructee to determine what information to provide

Staffan Larsson & Annie Zaenen / October 2000 / page 11 / From written manual to spoken dialogue – 3 levels of interactivity 1. written/spoken monologue1. written/spoken monologue –no overt interaction (but minimal interaction in written m.) 2. recapture minimal interactivity in spoken presentation2. recapture minimal interactivity in spoken presentation –system can ask yes/no questions and deals with simple user feedback (“done”, “OK”, “what did you say?”) 3. increase interactivity3. increase interactivity –user can control level of detail by indicating whether she already knows certain subprocedures –requires more detailed domain task plan

Staffan Larsson & Annie Zaenen / October 2000 / page 12 / Example from a Xerox manual Reinstalling the print headReinstalling the print head Caution: make sure that the green carriage lock lever is STILL moved all the way forward before you reinstall the print headCaution: make sure that the green carriage lock lever is STILL moved all the way forward before you reinstall the print head 1. Line up the hole in the print head with the green post on the printer carriage1. Line up the hole in the print head with the green post on the printer carriage Lower the print head down gently into positionLower the print head down gently into position 2. Gently push the green carriage lock lever up until it snaps into place2. Gently push the green carriage lock lever up until it snaps into place This secures the print headThis secures the print head 3. Close the tops cover and re-attach the scanner3. Close the tops cover and re-attach the scanner 4. Press and release the yellow LED button4. Press and release the yellow LED button The printer will prepare the cartridge for printingThe printer will prepare the cartridge for printing Note: if the carriage does not move from the center position after you press the carriage change button, remove and reinstall the print headNote: if the carriage does not move from the center position after you press the carriage change button, remove and reinstall the print head

Staffan Larsson & Annie Zaenen / October 2000 / page 13 / The TRINDI information state approach Information states represent information available to dialogue participants, at any given stage of the dialogueInformation states represent information available to dialogue participants, at any given stage of the dialogue Dialogue moves trigger information state updates, formalised as information state update rulesDialogue moves trigger information state updates, formalised as information state update rules TrindiKit: software package for implementing dialogue systems; based on the information state approach to dialogue managementTrindiKit: software package for implementing dialogue systems; based on the information state approach to dialogue management –Explicit information state data-structure makes systems more transparentmakes systems more transparent closer to dialogue processing theorycloser to dialogue processing theory easier comparison of theorieseasier comparison of theories –modularity for simple and efficient reconfiguration and reusability

Staffan Larsson & Annie Zaenen / October 2000 / page 14 / Building a system TRINDIKIT dialogue theory (IS, rules, moves etc) domain knowledge (resources) domain-specific system domain-independent system software engineering (basic types, control flow)

Staffan Larsson & Annie Zaenen / October 2000 / page 15 / input inter- pret Information State resource control updateselect gene- rate output resource DME

Staffan Larsson & Annie Zaenen / October 2000 / page 16 / GoDiS - An experimental dialogue system built using the TrindiKit Information-seeking dialogueInformation-seeking dialogue Information state based Ginzburg’s notion of Questions Under Discussion (QUD)Information state based Ginzburg’s notion of Questions Under Discussion (QUD) Dialogue plans to drive dialogue; plans are dynamically interpreted by information state update rulesDialogue plans to drive dialogue; plans are dynamically interpreted by information state update rules Simpler than general reasoning and planningSimpler than general reasoning and planning More versatile than frame-filling and finite automataMore versatile than frame-filling and finite automata Moves: ask, answer, greet, thank, requestRepeat, repeatMoves: ask, answer, greet, thank, requestRepeat, repeat

Staffan Larsson & Annie Zaenen / October 2000 / page 17 / GoDiS information state type PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) BEL : set( Prop ) QUD : stack( Question ) LM: set( Move )

Staffan Larsson & Annie Zaenen / October 2000 / page 18 / from GoDiS to IMDiS information state SHARED.ACTIONS : stack of actions that the system has instructed the user to perform but that have not yet been confirmed as done

Staffan Larsson & Annie Zaenen / October 2000 / page 19 / IMDiS information state type PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) BEL : set( Prop ) QUD : stack( Question ) ACTIONS : stack( Action ) LM: set( Move )

Staffan Larsson & Annie Zaenen / October 2000 / page 20 / IMDiS in monologue mode 2 moves: Instruct, Inform2 moves: Instruct, Inform All actions use the update rule autoConfirmAll actions use the update rule autoConfirm rule autoConfirmrule autoConfirm pre: eff: val( SHARED.ACTIONS, A ) pop( SHARED.ACTIONS ) add( SHARED.BEL, done(A))

Staffan Larsson & Annie Zaenen / October 2000 / page 21 / IMDiS in dialogue mode 9 moves (6 GoDiS moves, the 2 above and Confirm)9 moves (6 GoDiS moves, the 2 above and Confirm) Confirmations are integrated by assuming that the current top-most action of the shared.actions has been performedConfirmations are integrated by assuming that the current top-most action of the shared.actions has been performed rule integrateUsrConfirm (slightly simplified)rule integrateUsrConfirm (slightly simplified) pre: eff: val( SHARED.ACTIONS, A ) pop( SHARED.ACTIONS ) add( SHARED.BEL, done(A)) val( SHARED.LM, confirm(usr) )

Staffan Larsson & Annie Zaenen / October 2000 / page 22 / Example from a Xerox manual (repeated) Reinstalling the print headReinstalling the print head Caution: make sure that the green carriage lock lever is STILL moved all the way forward before you reinstall the print headCaution: make sure that the green carriage lock lever is STILL moved all the way forward before you reinstall the print head 1. Line up the hole in the print head with the green post on the printer carriage1. Line up the hole in the print head with the green post on the printer carriage Lower the print head down gently into positionLower the print head down gently into position 2. Gently push the green carriage lock lever up until it snaps into place2. Gently push the green carriage lock lever up until it snaps into place This secures the print headThis secures the print head 3. Close the tops cover and re-attach the scanner3. Close the tops cover and re-attach the scanner 4. Press and release the yellow LED button4. Press and release the yellow LED button The printer will prepare the cartridge for printingThe printer will prepare the cartridge for printing Note: if the carriage does not move from the center position after you press the carriage change button, remove and reinstall the print headNote: if the carriage does not move from the center position after you press the carriage change button, remove and reinstall the print head

Staffan Larsson & Annie Zaenen / October 2000 / page 23 / Simple task plan EFFECT: reinstalled (print_head) NAME: reinstall(print_head) PRECOND: moved_forward(carriage_lock) close(top_cover) reattach(scanner) press_and_release(button) remove(print_head) reinstall(print_head) Final state Action line_up(hole, post) moved_from_center(carriage) N Y lower(print_head) push(lever) condition

Staffan Larsson & Annie Zaenen / October 2000 / page 24 / From domain task plan to monologue and dialogue plans

Staffan Larsson & Annie Zaenen / October 2000 / page 25 / Sample IMDiS information state PRIVATE =PLAN = AGENDA = { findout(?moved(carriage)) } SHARED = if_then( not moved( carriage ), [ remove(print_head), reinstall(print_head) ] ) COM = moved_forward(carriage_lock) done(secure(print_head)) done(close(top_cover)) done(reattach(scanner)) QUD = LM = { instruct(sys, press_and_release(yellow_button) } BEL = { } TMP = (same structure as SHARED) ACTIONS=

Staffan Larsson & Annie Zaenen / October 2000 / page 26 / Monologue and dialogue behavior in IMDiS interactivity level 1 – (spoken) monologue interactivity level 1 – (spoken) monologue –the text you have seen above but presented orally so the user can direct her attention to the task

Staffan Larsson & Annie Zaenen / October 2000 / page 27 / level 2 - dialogue behavior with minimal interactivity yes/no questionsyes/no questions confirm (”ok”)confirm (”ok”) requestRepeat (”what?”)requestRepeat (”what?”) grounding and avoidance of irrelevant informationgrounding and avoidance of irrelevant information example:example: S: Press and release the yellow buttonS: Press and release the yellow button U: okU: ok S: Has the carriage moved from the center position?S: Has the carriage moved from the center position? U: yesU: yes S: the print head is now installedS: the print head is now installed U: what?U: what? S: the print head is now installedS: the print head is now installed

Staffan Larsson & Annie Zaenen / October 2000 / page 28 / Complex task plan EFFECT: reinstalled (print_head) NAME: reinstall(print_head) PRECOND: moved_forward(carriage_lock) close(top_cover) reattach(scanner) press_and_release(button) remove(print_head) reinstall(print_head) Final state Complex action Action secure(print_head) moved_from_center(print_head) Condition N Y

Staffan Larsson & Annie Zaenen / October 2000 / page 29 / Subtask plan for complex action EFFECT: secured (print_head) NAME: secure(print_head) PRECOND: - line_up (hole, post) lower(print_head) push(lever)

Staffan Larsson & Annie Zaenen / October 2000 / page 30 / level 3: increased interactivity skipping subtask instructionsskipping subtask instructions –S: Put the print head in place –U: Ok, done, what now? –S: Close the top cover requesting subtask instructions; popping out of subdialoguerequesting subtask instructions; popping out of subdialogue –S: Put the print head in place –U: how? –S: Line up the hole in the print head with the green post on the printer carriage –U: done, I now remember the rest, the print head is secured

Staffan Larsson & Annie Zaenen / October 2000 / page 31 / Domain task plans – where do they come from? What we did: read the text and construct a corresponding domain plan that seems plausible; this is not a principled approachWhat we did: read the text and construct a corresponding domain plan that seems plausible; this is not a principled approach Alt.1: Annotating existing manuals with procedural relations (Scott et. al. 1995); use this to reconstruct domain planAlt.1: Annotating existing manuals with procedural relations (Scott et. al. 1995); use this to reconstruct domain plan Alt.2: Author uses authoring tool to specify domain plan (“knowledge model”); this has been done in the context of multilingual manual authoring (Scott 1996, DRAFTER project, GIST project)Alt.2: Author uses authoring tool to specify domain plan (“knowledge model”); this has been done in the context of multilingual manual authoring (Scott 1996, DRAFTER project, GIST project)

Staffan Larsson & Annie Zaenen / October 2000 / page 32 / ConclusionsConclusions We provide a simple schema modeling the relation between task plan and discourse plans for intermediate interactivity levelWe provide a simple schema modeling the relation between task plan and discourse plans for intermediate interactivity level –The schema also gives a model of how different levels of interactivity (from monologue to dialogue) are related –The degree of interactivity is limited by the level of detail with which the manual encodes the underlying task Document transformation requires separation of task plans and assumptions about interactionsDocument transformation requires separation of task plans and assumptions about interactions –Task plans can be obtained either by marking up existing manuals, or by constructing them using authoring tools Extended existing information-seeking dialogue system to handle instructional dialogue (reusability!); based on information state approachExtended existing information-seeking dialogue system to handle instructional dialogue (reusability!); based on information state approach

Staffan Larsson & Annie Zaenen / October 2000 / page 33 / ProspectProspect Use existing domain task plans produced by annotating text or using authoring toolsUse existing domain task plans produced by annotating text or using authoring tools TrindiKit dialogue system for information seeking dialogue, adapted to instructional dialogue; uses dialogue plans (NB: current system very simple)TrindiKit dialogue system for information seeking dialogue, adapted to instructional dialogue; uses dialogue plans (NB: current system very simple) Our model sketches a mapping from the domain plans to dialogue plansOur model sketches a mapping from the domain plans to dialogue plans So, building on existing components, this indicates a possible way to get instructional dialogue systems without a lot of extra workSo, building on existing components, this indicates a possible way to get instructional dialogue systems without a lot of extra work

Staffan Larsson & Annie Zaenen / October 2000 / page 34 / Possible future work Explore and implement further levels of interactivityExplore and implement further levels of interactivity Scale up experiment, e.g. a complete manualScale up experiment, e.g. a complete manual Explore use of domain plans generated using existing toolsExplore use of domain plans generated using existing tools Extend IMDiS (and GoDiS) to handle referent resolution subdialogues, asynchronous turntaking etc.Extend IMDiS (and GoDiS) to handle referent resolution subdialogues, asynchronous turntaking etc. Do real NL generation, possibly using existing componentsDo real NL generation, possibly using existing components

Staffan Larsson & Annie Zaenen / October 2000 / page 35 / extra slides... Relation to previous workRelation to previous work –Carlson, Kuppevelt –M. Young –Smith & Hipp –Interactive tutoring (Lester, Rickel & Johnson) –N.B. We have not read all papers by everyone...

Staffan Larsson & Annie Zaenen / October 2000 / page 36 / Relation to previous work: Kuppevelt, Carlson monologue as a special case of dialogue where all moves are answers to implicit questions resulting from the previous move(s)monologue as a special case of dialogue where all moves are answers to implicit questions resulting from the previous move(s) exampleexample –The printer will prepare the cartridge for printing –Q: What if the carriage does not move... –A: If the carriage doesn’t move,... but this is not a processing theory, and it is unclear how it could be implemented; the number of possible questions that could arise at any given moment is too bigbut this is not a processing theory, and it is unclear how it could be implemented; the number of possible questions that could arise at any given moment is too big –

Staffan Larsson & Annie Zaenen / October 2000 / page 37 / Relation to previous work: Young Young (1997) deals with how to give the right amount of information in a (monologue) text, given a dynamically (automatically) generated task plan which may be very complexYoung (1997) deals with how to give the right amount of information in a (monologue) text, given a dynamically (automatically) generated task plan which may be very complex Young deals only with monologueYoung deals only with monologue We deal with both monologue and dialogueWe deal with both monologue and dialogue Our plans are not automatically generated, and not as complexOur plans are not automatically generated, and not as complex

Staffan Larsson & Annie Zaenen / October 2000 / page 38 / Relation to previous work: Smith & Hipp Handles similar type of dialogue (problem solving) using “Missing Axiom Theory”, using general inference over FOL representationHandles similar type of dialogue (problem solving) using “Missing Axiom Theory”, using general inference over FOL representation Handles some phenomena which we don’t (e.g. referent disambiguation)Handles some phenomena which we don’t (e.g. referent disambiguation) S&H do not generate monologue, or explore several levels of interactivity; however, they allow mixed initiative to a larger extent than we doS&H do not generate monologue, or explore several levels of interactivity; however, they allow mixed initiative to a larger extent than we do S&H assume complex representation of the domain task plan (?)S&H assume complex representation of the domain task plan (?) Our approach is less complex and therefore easier to implement and more computationally tractableOur approach is less complex and therefore easier to implement and more computationally tractable We provide an explicit model of the relation between monologue and simple dialogueWe provide an explicit model of the relation between monologue and simple dialogue

Staffan Larsson & Annie Zaenen / October 2000 / page 39 / Dialogue models and monologue Unfortunately, dialogue models don’t look at monologuesUnfortunately, dialogue models don’t look at monologues Fortunately, dialogue models based on information states can be extended to account for monologuesFortunately, dialogue models based on information states can be extended to account for monologues Ways to adapt dialogue models to monologueWays to adapt dialogue models to monologue –General theory of action and communication –Monologue as a degenerate case of dialogue

Staffan Larsson & Annie Zaenen / October 2000 / page 40 / Relation to previous work: interactive tutorials using text

Staffan Larsson & Annie Zaenen / October 2000 / page 41 / Similarities between monologue and dialogue Hierarchically structuredHierarchically structured Can be represented as a sequence of questions and answers: Carlson, Van KuppeveltCan be represented as a sequence of questions and answers: Carlson, Van Kuppevelt

Staffan Larsson & Annie Zaenen / October 2000 / page 42 / Importance of Information States The main difference between monologue and dialogue lies in what each participant knows about what the others knowThe main difference between monologue and dialogue lies in what each participant knows about what the others know Instructional manuals as a genre make assumptions about information states (as do other text genres)Instructional manuals as a genre make assumptions about information states (as do other text genres) Between monologue and full-blown ‘natural’ dialogue one can devise various modes of interaction: everything is a question of style and genreBetween monologue and full-blown ‘natural’ dialogue one can devise various modes of interaction: everything is a question of style and genre

Staffan Larsson & Annie Zaenen / October 2000 / page 43 / theoretical aside: plan representation traditionally, plans are represented as decomposition treestraditionally, plans are represented as decomposition trees we add conditionals to the plan representation format, adding expressivitywe add conditionals to the plan representation format, adding expressivity plans are interpreted dynamically by information state update rulesplans are interpreted dynamically by information state update rules our domain task plans are not constructed dynamically by a planner; rather, we assume that they are constructed by a human (the author of the manual), possibly using some markup languageour domain task plans are not constructed dynamically by a planner; rather, we assume that they are constructed by a human (the author of the manual), possibly using some markup language discourse plans are generated from task plans using a simple schema which encapsulates the different levels of interactivitydiscourse plans are generated from task plans using a simple schema which encapsulates the different levels of interactivity

Staffan Larsson & Annie Zaenen / October 2000 / page 44 / Sample GoDiS information state (travel agency domain) PRIVATE =PLAN = AGENDA = { findout(?return) } SHARED = findout(? x.month(x)) findout(? x.class(x)) respond(? x.price(x)) COM = dest(paris) transport(plane) task(get_price_info) QUD = LM = { ask(sys, x.origin(x)) } BEL = { } TMP = (same structure as SHARED)

Staffan Larsson & Annie Zaenen / October 2000 / page 45 / What we have not done NL generation – instead, we used canned text derived from manualsNL generation – instead, we used canned text derived from manuals referent disambiguation issuesreferent disambiguation issues more complex user utterances, e.g. alternative questions (though latest version of GoDiS handles this partially)more complex user utterances, e.g. alternative questions (though latest version of GoDiS handles this partially)

Staffan Larsson & Annie Zaenen / October 2000 / page 46 / Possible extensions Larger scale experimentsLarger scale experiments More descriptive work, including on more fine- grained linguistic aspectsMore descriptive work, including on more fine- grained linguistic aspects Implementation of more complex dialogue behaviours, e.g. referent resolution subdialogues, explanatory subdialoguesImplementation of more complex dialogue behaviours, e.g. referent resolution subdialogues, explanatory subdialogues Explore higher levels of interactivityExplore higher levels of interactivity Build tools to make the creation of multi-purpose documents possibleBuild tools to make the creation of multi-purpose documents possible Mark-up schemes (XML)Mark-up schemes (XML)