1. The semantics of dialogue acts Harry Bunt Oxford, IWCS 2011

2. The semantics of dialogue acts bridging the gap between computational semantics and pragmatics Harry Bunt Oxford, IWCS 2011

3. Plan 1Dialogue acts: Basics 2DIT, DIT ++, and ISO 24617-2 3DiAML: Dialogue Act Markup Language 4Context modelling 5Context model update semantics for dialogue acts 6Conclusions and perspectives

4. Dialogue acts Why dont you start? question/suggestion Do you know what time it is? question/reproach Its nearly five oclock. statement/answer/warning/reproach Action-based semantics: dialogue acts as addressees information- state update operators. (or context update) Dialogue acts have a communicative function and a semantic content (propositional/referential/action-al); the communicative function specifies a type of update operation; the semantic content is the material for updating the IS with. A communicative function can be applied to a semantic content in order to produce a dialogue act. Study of the semantics of communicative functions complements traditional compositional semantics

5. Dialogue act analysis: current state Speech Act Theory Communication as Cooperation (Grice) (Austin, Searle) Communicative Activity Analysis (Allwood) ( HCRC TRAINS MRDA … GBG-IMDIT Verbmobil DAMSL MATEDIT++ LIRICS ISO 24617-2 (DIS) DIT++ Release 5

6. Multidimensionality Participating in a dialogue involves more than just pursuing a certain goal, task or activity: -Giving and eliciting feedback -Taking turns -Managing the use of time -Establishing and maintaining contact -Dealing with social obligations (greeting, thanking, apologizing,…) -….. Communication has many dimensions An utterance may have parts or aspects that have a communicative function relating to different dimensions – be multifunctional

7. Dimensions for dialogue acts DIT Definition of a multimensional taxonomy of dialogue acts: Set of dialogue acts, partitioned into clusters of which each member addresses the same aspect of communication, and where each of these aspects can be addressed independently of the other dimensions. (In other words, dimensions are independent or orthogonal.) DIT: Every two communicative functions which can be used in a certain dimension are either mutually exclusive or one is a specialization of the other (e.g. Answer Confirmation) a functional segment can have a communicative function in each dimension, but never more than one (modulo entailed functions)

8. DIT: 10 dimensions of communication Task: Performing a certain task or activity through the dialogue Feedback: a.auto-feedback; providing information about ones processing (perception, understanding, evaluation,…) of previous utterances; b.allo-feedback: asserting or eliciting information the partners processing of previous utterances Interaction Management: managing contact turn allocation (speaker role) use of time structuring of the discourse editing of ones own and of partners speech social obligations - greeting, thanking, apologising, saying goodbye,… All these types of information must be represented in information states for ISU semantics of dialogue acts !

9. DIT++ taxonomy of communicative functions Communicative functions: A.specific for particular dimensions, e.g. Turn Grab, Stalling, Apology,...:dimension-specific functions B.general-purpose functions, applicable to any dimension of communication, e.g. YN-Question, Inform, Request,...: Total of 93 (basic) communicative functions: - 33 general-purpose functions - 60 dimension-specific functions + function qualifiers for variation due to uncertainty, conditionality, sentiment See http://dit.uvt.nlhttp://dit.uvt.nl

10. general-purpose communicative functions Information-transfer functions action-discussion functions (8 )Info-seeking (9) info-providing (8) commissives (7) directives Offer Address Suggestion Request Question Inform Suggestion Promise Instruct Prop. Choice Q Set Q Agreement Disagreement Accept Decline Q. Answer Suggestion Suggestion Address Offer Address Request Check Q. Confirm Disconfirm Correction Accept Decline Accept Req. Decline Req. Offer Offer

11. Dimension-specific communicative functions Auto- Allo- Contact Time Partner Turn Own Discourse Social Feedback Feedback Speech Speech Structuring Obligations Editing Editing Managem. Establish C Positive Positive Check C Opening Pos. Attention … Stalling Completion Pre-closing Pos. Perception Negative Pausing Correct- Error signalling I-Greeting … … misspeaking Retraction R-Greeting Pos. Execution Elicitation Self-correction I-Self-Introd. Negative Elic. Attention R-Self-Introd. Neg. Attention … Turn-initial Turn-final I-Apology … Elic. Execution Accept-Apology Neg. Execution Thanking Turn Take Turn Keep Accept-Thanking Turn Grab Turn Release I-Goodbye Turn Accept Turn Assign R-Goodbye

12. Communicative function qualifiers Variations w.r.t. uncertainty, conditionality, incompleteness, and sentiment, e.g.: A: Would you like to have some coffee? B1: Only if you have it ready. (Conditional Accept Offer) B2: That would be wonderful! (Happily Accept Offer) A: Do you know what time the meeting will end? B: Not sure, maybe something like two oclock. (Uncertain Anwer)

13. ISO standard 24617-2 (DIS) Dialogue act annotation Slightly simplified version of DIT++ taxonomy Communicative functions defined as ISO data categories, following ISO standard 12620 (see http://semantic-annotation.uvt.nl/)http://semantic-annotation.uvt.nl/ Annotation language DiAML (Dialogue Act Markup Language), with abstract syntax + formal semantics and concrete XML-based syntax Annotated multilingual test suites (English, Dutch, Italian) Established as Draft International Standard, January 11, 2011 Expected to become ISO standard later in 2011

14. ISO Project 24617-2 Project team: Jan Alexandersson Harry Bunt (PL) Jean Carletta Alex Chengyu Fang Jae-Woong Choe Koiti Hasida Volha Petukhova Andrei Popescu-Belis Claudia Soria David Traum Consulting group: Jens Allwood James Allen Nick Campbell Roberta Catizone Anna Esposito Thierry Declerck Raquel Fernandez Giacomo Ferrari Gil Francopoulo Dirk Heylen Julia Hirschberg Kristiina Jokinen Maciej Karpinski Staffan Larsson Kiyong Lee Oliver Lemon Carlos Martinez-Hinarejos Paul Mc Kevitt Mike McTear David Novick Tim Paek Patrizia Paggio Catherine Pelachaud Massimo Poesio German Rigau Laurent Romary Nicla Rossini Milan Rusko Candice Sidner Marieke van Erp Ielka van der Sluis Kristinn Thorisson Aesoon Yoon Yorick Wilks

15. ISO requirements for annotation standards ISO Linguistic Annotation Framework (Ide & Romary, 2005): annotations: the linguistic information that is added to segments of language data, independent of the format in which the information is represented; representations: particular formats in which annotations are rendered, (e.g. in XML, in typed feature structure AVMs, or in graphs in graphical form) independent of their content. Standards should be formulated at the level of annotations.

16. Traditional language definition semantics syntax

17. Annotation/representation distinction: abstract/concrete syntax abstract syntaxconcrete syntax semantics

18. Ideal Concrete Syntax A concrete syntax defines an ideal representation format iff: 1.Every annotation structure, defined by the abstract syntax, has a representation according the concrete syntax; 2.Every representation, defined by the concrete syntax, is the rendering of a unique annotation structure according to the abstract syntax.

19. Ideal concrete syntax abstract syntax ideal concrete syntax-1 semantics F 1 F 1 -1 IaIa ideal concrete syntax-2 F 2 -1 F2F2 C 12 C 21

20. DiAML abstract syntax An annotation structure is a pair consisting of a set E of entity structures and set L of link structures, which connect entity structures. Entity structures contain semantic information about a segment of source data; link structures describe semantic relations between segments of source data. Most important type of entity structure in DiAML: dialogue act structure, consisting of: - speaker S; addressee A; - dimension D; communicative function f or a pair with qualifier(s) q

21. DiAML example 1. P1: What time does the next train to Tilburg leave? Task: fs1: What time does the next train to Tilburg leave? Set-Question (WH-question) 2. P2: The next train to Tilburg leaves I think at 8:32. Task: fs2: The next train to Tilburg leaves I think at 8:32. Answer [uncertain] AuFB: fs3: The next train to Tilburg Positive AutoFeedback

22. DiAML concrete syntax, example <dialogueAct xml:id="da1" target=#fs1 sender="#p1" addressee="#p2 communicativeFunction="setQuestion" dimension="task /> <dialogueAct xml:id="da2" target="#fs2 sender="#p2" addressee="#p1 communicativeFunction="answer dimension="task" certainty="uncertain/> <dialogueAct xml:id="da3" target="#fs3 sender="#p2" addressee="#p1 communicativeFunction="autoPositive dimension="autoFeedback"/>

23. Context models (information states) Content of a context model / information state: All and exactly that information which has to be updated when interpreting a dialogue act. (And/or: all that information that is involved in the generation of dialogue acts) Formalization and implementation of a context model: DRSs (Poesio & Traum, 1998) Contexts in Constructive Type Theory (Ahn, 2001) Modular Partial Models (Bunt, 2002) Dependent Record Types (Cooper, 2004) Typed Feature Structures (Keizer & Bunt, 2007) Common assumption: context models are highly structured, to facilitate efficient updating.

24. DIT context models Content of context models containing the kinds of information required by the 10 dimensions of DIT is best structured into 5 components to facilitate efficient representation and updating: Linguistic context: dialogue history; dialogue future Cognitive context: agents own processing of previous utterances; beliefs about partners processing Semantic context: information about the task (domain) Perceptual/physical context: perceptual information (e.g. visual) Social context: communicative obligations and permissions Pending context: information under consideration, which has yet to be evaluated for being consistent with content of permanent context. Context update = addition of elements to pending context.

25. DIT context model implementation

26. Dialogue act semantics Remember: dialogue act structure. The (old) idea: the interpretation of a DA structure is a function that can be applied to a semantic content, resulting in the specification of an context update operation. Implementation: recursive valuation function V, recursion ending at application of model assignment F: V( = V(f)( ). Taxonomy structure Update specification structure Update specifications as combinations of elementary update schemes Formalization of informal definitions, e.g. S wants to make p known to A Want(S, Bel(A, p)) S believes p is correct Bel(S,p)

27. Dialogue act semantics (contd) Elementary update schemes: e.g. U 1 : add to the addressees pending context that the speaker believes that p Example: V( = = V(PropQ)( ) = = λp. [U 10 (Usr, Sys, Sys* SemC, p) υ U 11 (Usr, Sys, Sys* SemC, p)]. e.g. if p = ArrTime(KL476, 19:15) then: (Sys* is Sys pending context). U 10 (Usr, Sys, Sys* SemC, p) update: Sys believes that Usr wants to know whether ArrTime(KL476, 19:15); U 11 (Usr, Sys, Sys* SemC, p) update: Sys believes that Usr assumes that Sys knows whether ArrTime(KL476, 19:15)

28. Example: feedback act with general- purpose function B: (u1) A: Could you please repeat that? F(Request) = λC. λX. λY. λD. λα. [U 23 (X, Y, D, α, C) υ U 26 (X, Y, D, α, C)] Unconditional request: C = Τ (the universally true statement) F(Request)(A, B, Autofeedback, Repeat(u1), Τ) = = U 23 (A, B, CogContext, Repeat(u1), Τ) υ U 26 (A, B, CogContext, Repeat(u1), Τ) = B* CogC =+ Bel(B, Want(A, [WillDo(B, Repeat(u1) -> CommitDo(B, Repeat(u1))]); B* CogC =+ Bel(B, Bel(A, CanDo(B, Repeat(u1))))

29. Example: Turn management act with dimension-specific function A: Charlie? F(TurnAssign)(A, B) =[λX. λY. U 101 (X,Y,TurnM) υ U 102 (X,Y, TurnM](A, B) = = U 101 (A,B,TurnM) υ U 102 (X,Y,TurnM) = = B* LingC =+ Bel(B, Current-Speaker(A)) B* LingC =+ Bel(B, Want(A, Next-Speaker(B))) i.e. B believes that A currently has the speaker role; B believes that A wants B to next have the speaker role

30. Semantics of qualifiers Communicative function qualifiers: - q-specifiers: make the preconditions of the function that they qualify more specific, e.g. taking expression of (un-)certainty or (un-)conditionality into account. Semantics: V( ) = f i (qs j ) - q-additives: enrich a communicative function with additional information, e.g.: with sentiment information V( ) = λS. λz. [f i (S,z) υ qa k (S,z)] Combination of q-specifier(s) and q-additive(s): V( ) = λS. λz. [(f i (qs j ))(S,z) υ (qa k (S,z)]

31. Semantics of qualifiers: example A: How about a cup of coffee? B: That would be wonderful ! V( ) = = λS. λz. [[F(AcceptOffer)(F(unconditional))](S,z) υ [F(happy)](S,z)] = λS. λz. [[[λC. λX. λY. λD. λα. [[U 24 (X,Y,D_i, α) υ U 25 (X,Y,D, α, C) υ U 25b (X,Y,D_i, α, C)](Τ)](S,z) υ HAPPY (S,z)) = λS. λY. λD. λz.U 24 (S,Y,D, z) υ U 25 (S,Y,D_i, z, Τ) υ U 25b (S,Y,D, z, Τ)] υ HAPPY (S,z) Applied to A, B, and the action coffee: A* Task =+ Bel(A, Want(B, CommitDo(A, coffee))) A* Task =+ Bel(A, Bel(B, WillDo(A, coffee))) A* Task =+ Bel(A, HAPPY (B, coffee))

32. Conclusions and Perspectives Dialogue acts consist (essentially) of a communicative function and a semantic content. Communicative functions can be given a formal semantics in terms of updates of a context model. The semantic of communicative functions can be effectively implemented using a structured context model implementation, for example as a typed feature structure, and combinations of elementary update schemes. This works well for the fine-grained sets of ISO 24617-2 and DIT++ communicative functions, which distinguish around 90 basic functions plus specifying and additive qualifiers for making more fine-grained distinctions. These communicative functions can effectively be recognized automatically (Petukhova & Bunt, IWCS 2011 paper) and open the perspective of effective full-blown incremental semantic interpretation of dialogue acts when integrated with incremental construction of a representation of the semantic content.

33. References Bunt, Harry (2000) Dialogue pragmatics and context specification. In Harry Bunt and William Black (eds.), Abduction, Belief and Context in Dialogue. Studies in Computational Pragmatics, pp. 81- 150. Amsterdam: John Benjamins. Bunt, Harry (2009) Multifunctionality and muldimensional dialogue semantics. In Proceedings of DiaHolmia, 13th Workshop on the Semantics and Pragmatics of Dialogue, Stockholm, pp. 3-14. Bunt, Harry (2010) A methodology for designing semantic annotation languages. In Proceedings of the 2nd International Conference on Global Interoperability for Language Resources (ICGL-2), Hong Kong, January 2010, pp. 29-46. Bunt, Harry (in press) Interpretation and generation of dialogue with multidimensional context models. In Anna Esposito (ed.), Toward Autonomous, adaptive, and context-aware multimedia interfaces, pp. 81--131. Berlin: Springer. Bunt, Harry (2011) Multifunctionality in dialogue. Computer, Speech and Language 25, 225-245. Bunt, Harry, Jan Alexandersson, Jean Carletta, Jae-Woong Choe, Alex Fang, Koiti Hasida, Kiyong Lee, Volha Petukhova, Andrei Popescu-Belis, Laurent Romary, Claudia Soria, and David Traum (2010). Towards an ISO standard for dialogue act annotation. In Proceedings 7th International Conference on Language Resources and Evaluation (LREC 2010). Paris: ELRA. Ide, Nancy and Harry Bunt (2010) Anatomy of semantic annotation schemes: Mappings to GrAF. In Proceedings of the 4th Linguistic Annotation Workshop (LAW-IV), Uppsala. Poesio, Massimo and David Traum (1998) Towards an axiomatisation of dialogue acts. In Proceedings of the Twente Workshop on the Semantics and Pragmatics of Dialogue, Enschede, pp. 207-222.