2. Siridus Specification, Interaction and Reconfiguration in Dialogue Understanding Systems an information state approach to flexible spoken dialogue systems

3. Overview Siridus project Information State Approach Main research areas Demonstrators

4. Siridus Project Partners –University of Gothenburg (technical coordinator) –University of the Saarland (administrative coordinator) –University of Seville –Telefonica I+D –Linguamatics –SRI (original coordinator) Duration –Jan 2000 - Dec 2002

5. Motivation Provide flexible, user-centred dialogue systems Improve reconfigurability of dialogue systems for – new tasks – new domains Provide architectures allowing the dialogue state to be accessible for speech recognition, generation of prosody etc.

6. IS transitions IS for System IS for System dialogue as transitions between IS states what information do we need in the IS? how should it be structured? can this information be reused e.g. synthesis/recognition? examples –underspecified commands (DELFOS) –questions under discussion (GODIS) Interpret U Generate U’ IS for System

7. TrindiKit GoDiS GoDiS-IGoDiS-A Travel Agency VCR manager IBDM / KOS ISU approach Delfos basic system Delfos framework DELFOS-NCL Home manager ISU approach TeleDELFOS Home manager

8. Research Areas Dialogue types Dialogue phenomena Enhancing Contextual Appropriateness of System Output Enhancing speech recognition through IS Robust interpretation

9. Dialogue Types Natural command dialogues –user initiated commands e.g. transfer my calls to Mr. Jones turn off all the lights upstairs and lock the front door Negotiative dialogues –Propose, negotiate & agree parameters for a task e.g. arrival-time, destination-time alternative solutions to a problem e.g. different flight options Tutorial dialogues –guide student through task –non collaborative –hinting (to encourage active learning)

10. Dialogue phenomena Conditional responses –Enhancing collaboration by: Indicating source of failure: –U: Can I fly on the second? –S: Not if you want to fly economy class. Indicating contingency of success: –S: Yes, if you fly with SAS. Clarifications –Lack of specificity S: Which account do you want to transfer to? U: My bank account S: Do you mean your deposit account or current account? –Unspecified task e.g. U: Channel five S: Do you want to add a program or change channel?

11. Enhancing Contextual Appropriateness of System Output In order to make dialogue system utterances sound natural and contextually appropriate, it is important –to control intonation of spoken output –to control other aspects of realization, such as generation of short utterances We handle these aspects using Information Structure –what speaker means to address vs. what she wants to say about it –what is the same vs. what discriminates between similar bits of information in context Information Structure is determined from the IS

12. Need for Varied Intonation U: Which devices are switched on? S: The KITCHEN LIGHT is switched on. U: What is the status of the kitchen light? S: The kitchen light is switched ON. Die Küchenleuchte ist EINGESCHALTET. Die KÜCHENLEUCHTE ist eingeschaltet.

13. Enhancing speech recognition using IS In flexible dialogue, may have dynamically generated prompts Can’t rely on pre-defined grammars to recognise the response for each prompt Potential sources of information: –previous move/dialogue history (choice from n-best) –syntactic/semantic coherence of fragments (direct parsing of word lattices) –reference resolution –state of the world Combine/contrast sources of evidence to decide –most likely utterance in word lattice / n-best –when to clarify

14. Robust Interpretation Phrase spotters go to a particular depth of analysis even if a full parse is possible When syntactic information is available should use it When not, should do at least as well as keyword/phrase spotting Two novel techniques: –distribute the semantic representation using a semantic chart –ontological knowledge to propose combinations via semantics in competition with syntax: semantic-based composition turn on the light and the kitchen Reconciling traditional syntax and compositional semantics with robust approaches

15. Main Demonstrators TrindiKit 3 Automated Telephone Operator

16. Trindikit 3 Toolkit for dialogue researchers Developed from TrindiKit 1 built in EU Trindi Project IS update rules allow you to write your own dialogue manager –IS +move(s)  IS Update rules suited for domain and language independent dialogue management Global information state –all modules can access all information –composed from library of datatypes

17. Trindikit advantages Theoretical –theory-independent allows implementation and comparison of competing theories promotes exploration of middle ground between simplistic and very complex theories of dialogue –intuitive formalisation and implementation of dialogue theories; the implementation is close to the theory Practical –promotes reuse and reconfigurability on multiple levels –general solutions to general phenomena enables rapid prototyping of applications –allows dealing with more complex dialogue phenomena not handled by current commercial systems

18. Automated telephone operator Natural language telephone-based access to company telephone directory/PABX Functionality –Dial by name –Multi-party conference –Call transfers –Database search (e-mail address, office, phone number) KQML based message passing Delfos dialogue management User trials at Telefónica I+D

19. Telephone Operator GUI Corporate Directory / PABX

20. Further Demonstrations Specific modules –Intonation –Conditional responses –Robust interpretation Applications to illustrate reconfigurability –Travel agency –VCR –Command and control of multiple devices (D’Homme project)

21. Demonstrators: VCR S> “What can I do for you? " U> five thirty S> "five thirty. I dont quite understand. Do you want to add a program or set the clock?“ U> add a program today S> "Okay. add a program…. Lets see. today… Do you want from five thirty or until five thirty?“ U> from five thirty S> "Okay. from five thirty… What channel do you want? " U> set the clock S> "Okay. Do you want to set the clock?“ U> yes S> "Lets see. What time? " U> four fifteen S> "Okay. four fifteen… The clock was set…Returning to add a program. What channel do you want? “ U> channel five S> "Okay. channel five." U> pardon S> "Okay. channel five." U> yes S> "What time do you want to stop recording? " U> turn on the t v S> "turn on the TV… Sorry, I cannot control the TV set. What time do you want to stop recording? “ U> seven thirty S> "Okay. until seven thirty…The program has been added. " + Feedback Task clarification Load plan Issue clarification Requesting other action ? Feedback Reraising - Feedback (by User) Rejection

22. Demonstrators: Home device control

23. Siridus Summary IS approach has provided a stimulus for theoretical work –use of IS to improve intonation of spoken output recognition –exploration of different dialogue genres and phenomena systems which are –modular –reconfigurable –between FS/form filling and BDI/planning an automated telephone operator demonstrator research tool for dialogue developers (TrindiKit 3)