We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byChandler Wonson
Modified about 1 year ago
© Löckelt, Becker, Pfleger, Alexandersson; DFKI Edilog 2002 Workshop Jan Alexandersson (Tilman Becker, Markus Löckelt, Norbert Pfleger) German Research Center for Artificial Intelligence – DFKI GmbH Stuhlsatzenhausweg 3, Geb Saarbrücken phone: (+44) Overlay – a non-monotonic operation for discourse processing
© Alexandersson; DFKI Nancy What is this Talk about Dialogue systems must deal with utterances relating elliptically to previous dialogue User: „half past five“ System: „at what time would you like to start recording?“ User: „I would like to start recording at half past five“... or utterances only partially related to the previous discourse System: [Showing a list of films] „Here is a list of films“ User: „what films are on TV tonight?“ User: ``Thats a boring program. I‘d rather go to the movies´´ User: ``Thats a boring program. I‘d rather go to the movies tonight´´
© Alexandersson; DFKI Nancy What is this Talk about? ``...an expensive large portrait of the queen by Wainright hanging in the library´´ ``and one of the princess over the mantel´´ There are similar elliptical phenomena like one-anaphora ``and an expensive large portrait of the princess hanging over the mantel´´
© Alexandersson; DFKI Nancy Overview SmartKom Domain Model Overlay Discourse Modelling Action Planning Processing Partial Utterances Conclusion
© Alexandersson; DFKI Nancy What is the challenge? General tasks for a dialogue manager –Enrichment: Given incomplete information, enrich it from other sources in order to proceed. –Validation: Validate hypotheses against the current discourse state
© Alexandersson; DFKI Nancy Our Approach One general operation – Overlay –Domain representation: (Typed) Feature Structures –Non-monotonic: Always succeeds –Computes a score: Reflecting the structural consistency of the two arguments of overlay
© Alexandersson; DFKI Nancy SmartKom: Task-Oriented Dialogue System MM Dialogue Back- Bone Home: Consumer Electronics EPG Public: Cinema, Phone, Fax, Mail, Biometrics Mobile: Car and Pedestrian Navigation Application Layer SmartKom-Mobile SmartKom-Public SmartKom-Home/Office
© Alexandersson; DFKI Nancy Module Overview on the SmartKom Control GUI
© Alexandersson; DFKI Nancy The True Story
© Alexandersson; DFKI Nancy The DFKI Dialogue Back-Bone Communication pools Main data flow Context information Analysers External Services Modality Fusion Discourse Modelling DiM Action Planning AP Presentation Manager Generators Speech Gesture Speech Graphics Gesture
© Alexandersson; DFKI Nancy theater: MovieTheater movie: Movie reservationNumber: PositiveInteger Domain Model Used for communication in the back-bone Frame-based ontology; representation as Typed Feature Structures in M3L (XML) name: String director: Person cast: PersonList yearOfProduction: PositiveInteger… address: Address seats: SeatStructure… CinemaReservation Application objects composed of subobjects Slots: Feature paths meaningful for the dialogue (entities that can be talked about / referenced to); e.g. movie:director:lastName in a CinemaReservation object Slots can recursively contain other slots firstName: String lastName: String…
© Alexandersson; DFKI Nancy Discourse Modeling Lattice of intention hypothesis sequences from Modality Fusion Enrichment and Validation –compare and enrich with a selected number of discourse states: fill in consistent information compute a score –for each hypothesis - background pair: Overlay (covering, background) Overlay gives a score representing how well the covering fits the background. Scores of overlay, speech interpretation etc. give overall score One augmented hypothesis sequence with best score is passed on to Action Planning Covering: Background: Intention lattice Selected augmented hypothesis sequence
© Alexandersson; DFKI Nancy Example for Overlay with TFS User: What films are on TV tonight? System: [presents list of films] User: That‘s a boring program, I‘d rather go to the movies. How do we inherit “tonight” ?
© Alexandersson; DFKI Nancy The Domain Model A named entertainment at some time A named TV program at some time on some channel A named Movie at some time at some cinema
© Alexandersson; DFKI Nancy Unification Simulation Films on TV tonightGo to the movies Fail – type clash
© Alexandersson; DFKI Nancy Overlay Simulation Go to the moviesFilms on TV tonight Assimilation Background Covering
© Alexandersson; DFKI Nancy ``Formal´´ Definition Overlay Let –co be covering –bg be background Step 1: –Assimilate(co,bg) T bg co Step 2: –Overlay(co,assimilate(co,bg)) If co and bg are frames: recursion If co is empty: use bg If bg is empty: use co If conflict: use co
© Alexandersson; DFKI Nancy Domain Models with Multiple Inheritance Assimilate(co,bg) –Compute the set of minimal upper bounds (MUB) –Specialize the MUBs –Unify the specialized MUBs T cobg Overlay remains untouched MUB
© Alexandersson; DFKI Nancy Processing Partial User Contributions Our Dialogues contain a lot of partial contributions –``Three´´ –``Three o‘clock´´ –``At three o‘clock´´ –``At three [ ] o‘clock´´ –``Later´´ –``Information [ ]´´ –... Requirement –Advanced discourse modelling and action planning –Bidirection communication DiM - AP
© Alexandersson; DFKI Nancy Discourse Modelling - Three-Tiered Context Representation DO 1 DO 2 LO 1 DO 10 DO 3 DO 9 Modality layer Discourse layer System: Here [pointing gesture] I show you a list of films running in Heidelberg. show heidelberg list LO 2 LO 3... Domain layer DomainObject 1 ticketfirst DO 11 DO 12 reserve LO 4 LO 5 LO 6 DomainObject 2 GO 1... User: Reserve a ticket for the first!
© Alexandersson; DFKI Nancy Action Planner – Regression Planning AP processes plan operators –Uniform description of mixed initiative with user or applications based on Games and Moves. Example: User: request – System: response System: request – Application: response Processing steps 1.Constructs a plan from plan operators –A Plan specifies a partial order of games 2.Interprets the plan –Execute the games Expectations are published during execution of the plan –Guide the interpretation of input
© Alexandersson; DFKI Nancy Action Planning - Regression planning provides(Y) needs(D) condition(D=x) Goal provides() needs(X,Y) provides(X) needs(A,B) provides(C) needs() provides(B) needs(D) provides(A) needs(C) provides(F,G) needs(H) provides(D) needs(E) For Example: Slot D=y Current State provides(?) needs(?) successor states
© Alexandersson; DFKI Nancy AP - Example Plan: VCR Goal: VCR_record VCR_checkMediumVCR_getChannelVCR_getStartTimeVCR_getEndTime VCR_rewindMedium
© Alexandersson; DFKI Nancy Action Planning Initiation of sub-dialogues to gather necessary information Order proposed by the system, but the user may not follow the trail Several cases of user responses to requests: –Providing a matching answer, –Supplying information not (yet) asked for, –Changing already established information –Acting uncooperatively or changing goals
© Alexandersson; DFKI Nancy Expectations When AP initiates a sub-dialogue, it expects a certain answer An Expectation structure is published to help the analysis modules –Expected Slots: corresponding to the request –Possible Slots: plausible in context –Filled Slots: already provided –The currently active Application Allows for a fine-grained distinction between types of possible utterances Discourse Modelling can employ different rankings for each category
© Alexandersson; DFKI Nancy Integration of Partial Partial Utterances are analyzed as subobjects –validation and enrichment include the integration into the preceding context (to achieve a coherent discourse) –we consider three types of partial utterances: 1.The system has the initiative and the user responds elliptically to a system request 2.An elliptical user utterance does not correspond to a request but can be interpreted in context 3.Misinterpretation or non-cooperative user behaviour
© Alexandersson; DFKI Nancy Integration of Partial - Possible Situations User Initiative System Initiative Expected slotN A expected, unify, plan continues Possible slot plausible, unify, replanning plausible, unify, replanning Filled slot plausible, overlay, replanning plausible, overlay, replanning Otherimplausible – recover strategies User contribution DiM processing AP processing
© Alexandersson; DFKI Nancy DiM - Processing User Responses User provides expected slots –Bridging: a new application object is created (of same type as the current application object) for each subobject in the hypotheses its type is compared to the expected slots –matching subobjects are integrated into the new application object –remaining subobjects are interpreted as plausible partial utterances (next slide) –Overlay: The extended application object is overlayed over the application object currently in focus
© Alexandersson; DFKI Nancy Processing Plausible Partial Utterances User provides possible slots –processed like expected slots (but get a score penalty) User changes filled slots –search for a discourse object DO of the same type in the discourse context –create application object AO1 of the same type as the application object AO2 within which DO was mentioned –overlay AO1 over AO2 –gets a score penalty based on position in discourse history If integration fails, all subobjects are passed on unchanged (integration fails)
© Alexandersson; DFKI Nancy Example: Expected and Possible Slots (1) User: I want to record a film. (2) System: When should I start recording? (3) User: 1:30pm on channel two. Analysis of (1) contains the setting of a goal VCR_record (empty application object of type VCR introduced) AP triggers presentation (2) and publishes an expectation containing: –VCR as the type of the current application object –VCR_startTime as the (only) expected slot –some possible slots (e.g., VCR_endTime and VCR_channel ) –and maybe some filled slots (e.g. VCR_mediumPresent ) Analysis of (3) contains two subobjects –the expected slot is filled by “1:30pm” –but also one possible slot is filled by “channel two”
© Alexandersson; DFKI Nancy Example: Filled Slots (1) User: What is currently running on TV? (2) System: The following [Smartakus points at a list of programmes] programmes are currently running. (3) User: And tonight?
© Alexandersson; DFKI Nancy Example: Filled Slots show programcurrently Modality layer Discourse layer Domain layer DO 1 DO 2 LO 1 LO 2 DO 3 LO 3 tonight DO 4 LO 4 DO 3
© Alexandersson; DFKI Nancy Conclusion Uniform, flexible and robust mechanism for interpreting possible partial utterances Pragmatical classification of partial utterances from the point of view of the action planner Modality independent Approach integrates seamlessly with other aspects of discourse processing, e.g. anaphora resolution Implemented and running in, e.g., a large multimodal dialogue system
© Alexandersson; DFKI Nancy Papers Jan Alexandersson and Tilman Becker. Overlay as the Basic Operation for Discourse Processing in a Multimodal Dialogue System. In: Proceedings of the IJCAI Workshop ``Knowledge and Reasoning in Practical Dialogue Systems,´´ Seattle, Norbert Pfleger, Jan Alexandersson, and Tilman Becker. Scoring Functions for Overlay and their Application in Discourse Processing. In Proceedings of ``KONVENS 2002,´´ Saarbrücken, Germany, Markus Loeckelt, Tilman Becker, Norbert Pfleger and Jan Alexandersson Making Sense of Partial. In: Bos, Foster & Matheson (eds): ``Proceedings of the sixth workshop on the semantics and pragmatics of dialogue (EDILOG 2002),´´ Edinburgh, UK, Pages Jan Alexandersson and Tilman Becker. The Formal Foundations Underlying Overlay. Submitted to IWCS5
© Alexandersson; DFKI Nancy Thank you very much for your attention! Merci infiniment de votre attention! Babelfish.altavista.com
DFKI Approach to Dialogue Management Norbert Reithinger, Elsa Pecourt, Markus Löckelt
German Research Center for Artificial Intelligence DFKI GmbH Stuhlsatzenhausweg Saarbruecken, Germany phone: ( ) /4162 fax: (+49.
Media Coordination in SmartKom Norbert Reithinger Dagstuhl Seminar “Coordination and Fusion in Multimodal Interaction” Deutsches Forschungszentrum für.
Multimodal Information Access Using Speech and Gestures Norbert Reithinger
German Research Center for Artificial Intelligence DFKI GmbH Saarbruecken, Germany WWW:http://www.dfki.de/~wahlster Eurospeech.
A comprehensive framework for multimodal meaning representation Ashwani Kumar Laurent Romary Laboratoire Loria, Vandoeuvre Lès Nancy.
An Ontological Framework for Web Service Processes By Claus Pahl and Ronan Barrett.
Dept. of Computer Science University of Rochester Rochester, NY By: James F. Allen, Donna K. Byron, Myroslava Dzikovska George Ferguson, Lucian Galescu,
Artificial Intelligence Knowledge Representation.
Dialogue Managers in two projects: Comic and Amitie Roberta Catizone University of Sheffield.
Use-Cases. Outline Introduction Use Case Diagrams Writing Use Cases.
Goteborg University Dialogue Systems Lab Comments on ”A Framework for Dialogue Act Specification” 4th Workshop on Multimodal Semantic Representation January.
Towards multimodal meaning representation Harry Bunt & Laurent Romary LREC Workshop on standards for language resources Las Palmas, May 2002.
ENTERFACE 08 Project 2 “multimodal high-level data integration” Mid-term presentation August 19th, 2008.
Understanding Naturally Conveyed Explanations of Device Behavior Michael Oltmans and Randall Davis MIT Artificial Intelligence Lab.
Designing Help… Mark Johnson Providing Support Issues –different types of support at different times –implementation and presentation both important.
1 Case Study: Starting the Student Registration System Chapter 3.
Interpreting Dictionary Definitions Dan Tecuci May 2002.
Referring to Objects with Spoken and Haptic Modalities Frédéric LANDRAGIN Nadia BELLALEM & Laurent ROMARY LORIA Laboratory Nancy, FRANCE.
Author: James Allen, Nathanael Chambers, etc. By: Rex, Linger, Xiaoyi Nov. 23, 2009.
SAB ReviewFebruary 2004Pervasive 2004April 2004 Using an Extended Episodic Memory Within a Mobile Companion Alexander Kröner, Stephan Baldes, Anthony Jameson,
Mastering the AS/400, Third Edition, author Jerry Fottral 1 Week 2 The System The AS/400 is a multi-user, multi-tasking system -- a system on which many.
The roots of innovation Future and Emerging Technologies (FET) Future and Emerging Technologies (FET) The roots of innovation Proactive initiative on:
1 Levi Lúcio © A Test Selection Language for CO-OPN Specifications Levi Lúcio, Luis Pedro and Didier Buchs University of Geneva.
ARCHITECTURES FOR ARTIFICIAL INTELLIGENCE SYSTEMS.
Katanosh Morovat. This concept is a formal approach for identifying the rules that encapsulate the structure, constraint, and control of the operation.
Service Service metadata what Service is who responsible for service constraints service creation service maintenance service deployment rules rules processing.
Chapter 4 Requirements Engineering. Requirements Engineering It helps software engineers to better understand the problem they will work to solve It encompasses.
Working group on multimodal meaning representation Dagstuhl workshop, Oct
Writing an ERG mal-rule David Mott IBM Emerging Technology Services.
© DATAMAT S.p.A. – Giuseppe Avellino, Stefano Beco, Barbara Cantalupo, Andrea Cavallini A Semantic Workflow Authoring Tool for Programming Grids.
Personalized Recommendation of Related Content Based on Automatic Metadata Extraction Andreas Nauerz 1, Fedor Bakalov 2, Birgitta.
Guided tours and on-line presentations: how authors make existing hypertext intelligible for readers C. C. Marshall, P. M. Irish, Guided tours and on-line.
© 2005 Prentice Hall12-1 Stumpf and Teague Object-Oriented Systems Analysis and Design with UML.
Role-plays for CALL: System Architecture and Resources Sabrina Wilske & Magdalena Wolska Saarland University ICL, Villach, September.
DQA meeting: : Learning more effective dialogue strategies using limited dialogue move features Matthew Frampton & Oliver Lemon, Coling/ACL-2006.
Name : Emad Zargoun Id number : EASTERN MEDITERRANEAN UNIVERSITY DEPARTMENT OF Computing and technology “ITEC547- text mining“ Prof.Dr. Nazife Dimiriler.
Spoken dialog for e-learning supported by domain ontologies Dario Bianchi, Monica Mordonini and Agostino Poggi Dipartimento di Ingegneria dell’Informazione.
1 Where do spatial context-models end and where do ontologies start? A proposal of a combined approach Christian Becker Distributed Systems Daniela Nicklas.
Creating Architectural Descriptions. Outline Standardizing architectural descriptions: The IEEE has published, “Recommended Practice for Architectural.
Introduction to Earth Science Section 2 Section 2: Science as a Process Preview Key Ideas Behavior of Natural Systems Scientific Methods Scientific Measurements.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 13 Slide 1 Application architectures.
U1, Speech in the interface:2. Dialogue Management1 Module u1: Speech in the Interface 2: Dialogue Management Jacques Terken HG room 2:40 tel. (247) 5254.
Architecture View Models A model is a complete, simplified description of a system from a particular perspective or viewpoint. There is no single view.
UML – Class Diagrams. Objectives What is UML? UML Diagrams Use Case Diagram Class Diagram.
© 2017 SlidePlayer.com Inc. All rights reserved.