Presentation is loading. Please wait.

Presentation is loading. Please wait.

Software Development Processes, Reuse and Knowledge Sources in Spoken Dialog System 11761 Dialog Seminar Rohit Kumar Friday, Dec 1, 2006.

Similar presentations


Presentation on theme: "Software Development Processes, Reuse and Knowledge Sources in Spoken Dialog System 11761 Dialog Seminar Rohit Kumar Friday, Dec 1, 2006."— Presentation transcript:

1 Software Development Processes, Reuse and Knowledge Sources in Spoken Dialog System Dialog Seminar Rohit Kumar Friday, Dec 1, 2006

2 Some basic ideas / Ground rules –Practical Dialogue Hypothesis –Domain Independence Hypothesis Rapid Prototyping of Dialogue Systems –Dialogue system frameworks –Re-use SLDS development methodologies & Best practices –Iterative Method –(DISC) Dialogue Engineering Life Cycle –Model View Controller (MVC) approach –Rapid Dialogue Prototyping (RDP) Methodology –User involvement in development –Methodologies i have used : 2 cases Some “potential” rules of thumb ? Knowledge Sources view of Dialogue Systems –Overview –Template based –Tools and Methods required Some ideas Also included: Full list of readings

3 Some basic ideas / Ground rules –Practical Dialogue Hypothesis –Domain Independence Hypothesis Rapid Prototyping of Dialogue Systems –Dialogue system frameworks –Re-use SLDS development methodologies & Best practices –Iterative Method –(DISC) Dialogue Engineering Life Cycle –Model View Controller (MVC) approach –Rapid Dialogue Prototyping (RDP) Methodology –User involvement in development –Methodologies i have used : 2 cases Some “potential” rules of thumb ? Knowledge Sources view of Dialogue Systems –Overview –Template based –Tools and Methods required Some ideas Also included: Full list of papers surveyed

4 Some basic ideas / Ground rules Spoken Dialog System development is costly and time consuming Two optimistic hypothesis (Allen et. al., 2000) –Practical Dialogue Hypothesis “The conversational competence required for practical dialogues, while still complex is significantly simpler to achieve than general human conversational competence” –Domain Independence Hypothesis “Within the genre of practical dialogue the bulk of complexity in the language interpretation and dialogue management is independent of the task being performed” –Agreeable ?! (I say “Yes”)

5 Some basic ideas / Ground rules –Practical Dialogue Hypothesis –Domain Independence Hypothesis Rapid Prototyping of Dialogue Systems –Dialogue system frameworks –Re-use SLDS development methodologies & Best practices –Iterative Method –(DISC) Dialogue Engineering Life Cycle –Model View Controller (MVC) approach –Rapid Dialogue Prototyping (RDP) Methodology –User involvement in development –Methodologies i have used : 2 cases Some “potential” rules of thumb ? Knowledge Sources view of Dialogue Systems –Overview –Template based –Tools and Methods required Some ideas Also included: Full list of papers surveyed

6 Rapid Prototyping of Dialogue Systems Dialogue System Frameworks –Can be considered a consequence or a partial proof of the domain independence hypothesis Various Frameworks –TRIPS –Adriane / Ravenclaw / Olympus (CMU) –Communicator (SRI, CMU, U. Colorado, …) –Speech Builder (MIT) –CSLU Rapid Application Developer –Industrial Frameworks: Voice Platforms (IBM, Microsoft, Nuance…) –Tutorial Dialog: TuTalk –Many more…

7 Rapid Prototyping of Dialogue Systems

8

9

10 (Denecke, 2002) Able to develop very simple dialog systems with 8 – 10 hours of effort using the Adriane framework –Domain independent Interaction Patterns implemented Question Undo, Correction, State –Development comprises of 7 steps Backend Application Databases Ontology Dialogue Goals Database Conversion Rules Parsing Grammars Generation Templates

11 Rapid Prototyping of Dialogue Systems Observations Effort calculated as per the knowledge source. Rapid prototyping can be considered a method in which knowledge is added to a framework to customize it. All design exhibit lack of coverage in Language resources. Direct consequence of not doing any data collection ahead of time.

12 Rapid Prototyping of Dialogue Systems To reduce development costs: Re-Use –Can be considered to be the driving force being the Domain Independence Hypothesis (Hanna et. al. 2005) While Domain Independence Hypothesis is acceptable, the challenge is to develop such an domain independent system –Part of the challenge is difference in terms of complexity of applications which in turn requires different degrees and forms of domain independence Object oriented methodology to support extensibility and re-use of (dialog) components –All discourse object contain capabilities –On request, domain spotter polls and gets scores from each discourse object indicating object’s ability for successfully handle the request –Supports maintainability and re-use –Idea of hierarchies of increasing degrees of specialism

13 Rapid Prototyping of Dialogue Systems Dynamic Linking of discourse objects –Keeps frames sizes small –Facilitates delegation and delimitation of agent responsibility –Open ended structure which can grow and adapt depending on objectives set by user on run time

14 Some basic ideas / Ground rules –Practical Dialogue Hypothesis –Domain Independence Hypothesis Rapid Prototyping of Dialogue Systems –Dialogue system frameworks –Re-use SLDS development methodologies & Best practices –(DISC) Dialogue Engineering Life Cycle –Iterative Method –Model View Controller (MVC) approach –Rapid Dialogue Prototyping (RDP) Methodology –User involvement in development –Methodologies i have used : 2 cases Some “potential” rules of thumb ? Knowledge Sources view of Dialogue Systems –Overview –Template based –Tools and Methods required Some ideas Also included: Full list of papers surveyed

15 SLDS Development Methodologies Dialogue Engineering Life Cycle –Based on DISC project on best practices for development and evaluation of SLDSs –Includes: Life Cycle model, Grid (set of issues and options), Set of Evaluation Criteria

16 SLDS Development Methodologies Grid comprises of 6 aspects of SLDS –For each aspect, the grid lists various issues along with options (based on the available state of the art at that time) along with pros and cons –Choices to be made by the developers Essentially and Iterative Life Cycle with 2 legs –Development Phases Grid Issues are addressed in these phases –Evaluation Evaluation criteria derived from choices made for the various Grid issues are applied here This life cycle does not talk much about the phase after integration i.e. maintenance. Also scope for re-use is not discussed

17 SLDS Development Methodologies Evaluation –Analysis: Sufficiency and clarity of documents –Design: Design goals and constraints are sound, non-contradictory and feasible –Simulation: WOZ with questionnaires –Construction: Glass box and black box tests –Integration: Collected user system interaction is analysed

18 SLDS Development Methodologies Iterative Method –Work iteratively from 2 angles Conceptual Design Framework customization Solid Lines –Creative Progression Dotted Line –How levels of the method correlate E.g. Interpretation Module –Conceptual design starts with Parsing theory and Requirement Specifications –Framework customization starts with Parser and builds on corpus created from WOZ or other approaches

19 SLDS Development Methodologies Iterative Method: Recommended Pre-requisites –Classification of possible dialogues and identification of main use-cases –Specification of requirements for the system behavior in each class of dialogues Considering Dialog Management in Iterative Development –Design Modularization Knowledge Representation Interfaces –Customization Tools Framework templates Code Patterns Identified Design choices –Dialogue History, User Request Handling, Sub-Dialogue Control

20 SLDS Development Methodologies Model View Controller Approach: Very Industrial –Derived from well established GUI development practice Possible to upload the user interface without modifying the application data or control logic

21 SLDS Development Methodologies

22 Rapid Dialogue Prototyping Methodology (RDPM) –Focuses on Dialogue components only 5 steps –Producing the Task model Draw relations between attributes and task to be performed –Deriving the initial dialogue model Generic Dialogue Nodes (GDN) –Simple (Static): User inputs –List (Dynamic): Selects from menu –Internal: Start/resets, etc. Local / Global dialogue flow management strategy –Using a WoZ experiment to instantiate the initial dialogue model –Using an Internal field test to refine the dialogue model –Using an external field test to evaluate the final dialogue model

23 SLDS Development Methodologies (Atwell et. al., 2000) Experience with user involvement in development –Particular case for CALL applications Involve users at every stage Two distinct classes of users –Prospective End-Users –Meta Level Experts Interesting findings reported in the paper within the application domain which was possible due to user involvements at all stages

24 SLDS Development Methodologies Methodologies I have used –Case 1: ConQuest dialogue system development –Case 2: CycleTalk dialogue system development Sorry no time to make the slides for the above. But lets discuss

25 Some basic ideas / Ground rules –Practical Dialogue Hypothesis –Domain Independence Hypothesis Rapid Prototyping of Dialogue Systems –Dialogue system frameworks –Re-use SLDS development methodologies & Best practices –(DISC) Dialogue Engineering Life Cycle –Iterative Method –Model View Controller (MVC) approach –Rapid Dialogue Prototyping (RDP) Methodology –User involvement in development –Methodologies i have used : 2 cases Some “potential” rules of thumb ? Knowledge Sources view of Dialogue Systems –Overview –Template based –Tools and Methods required Some ideas Also included: Full list of papers surveyed

26 Some basic ideas / Ground rules –Practical Dialogue Hypothesis –Domain Independence Hypothesis Rapid Prototyping of Dialogue Systems –Dialogue system frameworks –Re-use SLDS development methodologies & Best practices –Iterative Method –(DISC) Dialogue Engineering Life Cycle –Model View Controller (MVC) approach –Rapid Dialogue Prototyping (RDP) Methodology –User involvement in development –Methodologies i have used : 2 cases Some “potential” rules of thumb ? Knowledge Sources view of Dialogue Systems –Overview –Tools and Methods required Some ideas Also included: Full list of papers surveyed

27 Knowledge Sources View Dialogue system development can be seen as a process of putting knowledge into an existing framework in order to customize the framework Tools and methods must –Provide visual languages for specification of structured dialogue data –Represent all relevant data and the dependences between them –Support formalism for defining constraints on the models and informing the user of violation of these models –Generate code that can be interpreted be generic dialogue system –Support the reuse for formerly developed domain models

28 Development Team & Requirements User Logs Randomly created Corpus Dialog Task Specification GrammarTemplates Voice Domain Knowledge Lexicon Language Model Acoustic Model

29 Knowledge Sources View Dialogue system development can be seen as a process of putting knowledge into an existing framework in order to customize the framework Tools and methods must –Provide visual languages for specification of structured dialogue data –Represent all relevant data and the dependences between them –Support formalism for defining constraints on the models and informing the user of violation of these models –Generate code that can be interpreted be generic dialogue system –Support the reuse for formerly developed domain models

30 Some basic ideas / Ground rules –Practical Dialogue Hypothesis –Domain Independence Hypothesis Rapid Prototyping of Dialogue Systems –Dialogue system frameworks –Re-use SLDS development methodologies & Best practices –Iterative Method –(DISC) Dialogue Engineering Life Cycle –Model View Controller (MVC) approach –Rapid Dialogue Prototyping (RDP) Methodology –User involvement in development –Methodologies i have used : 2 cases Some “potential” rules of thumb ? Knowledge Sources view of Dialogue Systems –Overview –Tools and Methods required Some ideas –Template based Also included: Full list of papers surveyed

31 Some Ideas: Template based development Somewhere in between the spectrum of domain independence and domain dependence –Based on extensive in domain re-use –Changes in knowledge sources with every re-use –Issues Incorporating knowledge changes Propagation of knowledge changes while maintaining constraints –Hence view of the template system development would be Which knowledge source is to be changed ?

32 Readings –An architecture for Generic Dialogue Shell, J. Allen, D. Byron, M. Dzikovska, G. Ferguson, L. Galescu, A. Stent –Rapid Prototyping for Spoken Dialogue Systems, Matthias Denecke –Developing Extensible and Reusable Spoken Dialogue Components: An examination of the Queen’s Communicator, Philip Hanna, Ian O’Neil, Xingkun Liu, Michael McTear –Iterative implementation of Dialogue System Modules, Lars Degerstedt Arne Jonsson –The Dialogue Engineering Life Cycle, Laila Dybkjaer, Niels Ole Bernsen –DISC Website, –Rapid Dialogue Prototyping Methodology, Trung H. Bui, Martin Rajman, Miroslav Melichar –User Guided System Development in Interactive Spoken Language Education, E. Atwell et. al. –OpenSpeech Dialog: More powerful applications at lower cost, SpeechWorks white paper –A survey of knowledge sources in dialogue systems, Annika Flycht-Erikssn –Universal Dialogue Specification for Conversational Systems, Anke Kolzer


Download ppt "Software Development Processes, Reuse and Knowledge Sources in Spoken Dialog System 11761 Dialog Seminar Rohit Kumar Friday, Dec 1, 2006."

Similar presentations


Ads by Google