1. 1 How To Annotate Interactions Using Dialog Function Units (Part 2) by Michal Novemsky CCLS, Columbia U. and Dept. of CS, NYU (with the help of Becky Passonneau)

2. 2 Items To Be Covered Transcription note: overlapping speech Link topics: multiple links per DFU; dangling Flinks Special DFU types: Inform-Answer-Offline; Backchannel: grounding Less Common DFUs (and requests to hold on) Broken-up DFUs in Loqui Headers (Request Table for Loqui; Thread summary for Enron) Enron issues: copied/pasted text; charts; message IDs and depth; duplicate messages Examples to practice on Homework assignment

3. 3 Overlapping Speech (Loqui only) In some Loqui dialogs, you will see [os…eo] in the transcription. This stands for “overlap start…end overlap”, and is used when two people are talking simultaneously For example, from Loqui dialog 78:

4. 4 When a DFU gets more than one link Sometimes a DFU is both a request and a response to something else; it gets a Blink as well as a (S)flink For example, from Loqui dialog 63: Here, line 3.2 is a response to the librarian’s offer to help, as well as an (implicit) request in its own right. Therefore, it receives both a Blink and a Sflink.

5. 5 Dangling Flinks (particularly Enron) Sometimes, a Request-Information that has an obligatory Flink is not responded to- we informally call this unanswered Flink “dangling” More likely in email threads, because immediate response is not obligatory For example (from Enron thread 15713): (trust that it doesn’t get answered, as M2 is the last message in this thread)

6. 6 A special kind of Inform (Enron only) Inform-Answer-Offline is sometimes used in Enron emails, when a DFU is clearly an answer to a question, but the question occurred outside of the email thread. Example from Enron thread 108: (Since this is the first line of the first message in the thread, it must be the case that the question was asked outside the thread.)

7. 7 A special kind of Backchannel (Loqui only) When a caller gives a request (often by RC number), sometimes the librarian repeats part or all of the request back to the caller, indicating comprehension. The label for this is [Backchannel: grounding, ]. It comes from Herbert H. Clark’s book Using Language, in which speakers communicate their current state of understanding to establish a degree of “common ground”- therefore, it is called “grounding” behavior.* *from Dialog Function Units for Studying Interaction across Modalities and Genres: Overview and Annotation Manual, Section 7

8. 8 Example of Backchannel: grounding From Loqui dialog 45 (also includes other kind of Backchannel):

9. 9 List of Less Common DFUs Request-Action Commit Affirmative (newly added) Reject Perform

10. 10 Request-Action When said by one person, this DFU asks another person to perform some non-communicative action (that therefore cannot be part of the dialog) A Flink is obligatory Examples include a request to write a report, send something by email, make coffee, or mow the lawn For example (from Enron thread 136629):

11. 11 Commit This DFU obligates the speaker or writer to perform some kind of non-communicative action Often a response to Request-Action (or possibly Request-Information) When a response to Request-Action or Request- Information, a Blink is necessary For example, from Loqui dialog 33:

12. 12 Asking someone to hold on (Loqui only) When a librarian asks a caller to hold on, sometimes the librarian seems to expect confirmation from the caller (a “Commit”) and sometimes seems to be informing the caller that another call is coming in. This can depend in part on how it sounds in the audio. Here (Loqui dialogue 60) the cases in 35 and 38 have been given the Inform DA label (You might remember from the last presentation; it means “incoming call ring”)

13. 13 Affirmative This DFU is for a positive response to a yes/no question Often a response to Request-Action (or sometimes Request-Information) A Blink is necessary when it is such a response For example, from Loqui dialog 78:

14. 14 Reject This DFU is a refusal to answer what is asked, or a refusal to perform a requested task Often a response to Request-Action (or sometimes Request-Information) A Blink is necessary when it is such a response For example, if one person asked “Could you make me some coffee?” and the second answered “Sorry, you have to make it yourself”, this would be a Request-Action - Reject pair

15. 15 Perform As the name would suggest, something with this DFU is a performative, meaning that the very act of saying it makes it true. Examples of this include “You’re fired,” “I promise,” or “I now pronounce you husband and wife” (only if the speaker has the proper authority) Links are possible- depends on context For example, from Enron thread 59580:

16. 16 When a DFU is broken up (Loqui only) Sometimes, a DFU spans several lines that are not sequential- they are usually separated by the other party interrupting, or saying something at the same time The most common DFUs this happens to are Inform and Request-Information The labeling is (e.g.): Inform-Start for the first line, Inform-Finish for the last line, and Inform-Continue for any lines in between (there may be many of these!) Include a description only after -Finish (the last one), like this, from Loqui dialog 55 (the #’s in parentheses are all the line numbers of -Start, -Continue [if there are any], and -Finish):

17. 17 First example of a broken-up DFU from Loqui dialog 60:

18. 18 Second example of a broken-up DFU Here is a broken-up DFU within another broken-up DFU, from Loqui dialog 42:

19. 19 Request Table (Loqui only) At the beginning of each dialog, insert a table that looks like this (this one is from Dialog 45): These are lines in which the caller requests an item- NOT when the librarian offers something, which sometimes occurs. Some dialogs contain no requests, so these do not require this table.

20. 20 Types of Requests Auth (by author) Title (by title) RC (by RC number, for recorded books) BR (by BR number, for Braille books) Other (if does not fit an above category) Or, any combination of the above, e.g. Auth/Title, or RC/Title, etc.

21. 21 Were requests successful? There are 3 answers to “Successful?”: “yes” (y), “no” (n), or “no/grounding” (n/g) “y” if librarian explicitly or implicitly indicates that s/he has understood and is looking up the item (“ok”, “got it”, etc.) “n” if librarian has not understood request- if s/he explicitly or implicitly requests clarification or confirmation “n/g” if the librarian repeats back request without question intonation (see slides 5-6, about Backchannel: grounding) ***NOTE: “successful” has nothing to do with whether books were actually sent out or not!

22. 22 Summarizing the Thread Type (Enron only) At the top of the thread, after writing the thread’s name (number), write its general purpose and/or content type, followed by a short description (in square brackets, as a DFU type would be) Sometimes the purpose or type can fall into more than one category, so write both For Loqui dialogs, the type is always External-event- planning, so a summary is unnecessary to include For example, from Enron thread 130806train:

23. 23 Types of email threads Information-Fact: exchange of factual information Information-Opinion: exchange of opinions Information-Unsure: annotator is unsure whether exchange is of facts or opinions External-event-planning: planning events outside email thread Collaboration-on-information-product: collaboration on docs/information (work is “inside” email thread) Social: main purpose is being social Other: if none of the above applies

24. 24 Copied-and-pasted Text (Enron only) In some Enron threads, someone copies and pastes something received from a party outside the email exchange This does not get annotated! For example, from Enron thread 158811train:

25. 25 Chart annotation (Enron only) Sometimes, in Enron emails, you will see a chart or list of items You do not need to annotate each line- this is considered one DFU (and if there is a title, it might be a DFU also) For example, from Enron thread 2641:

26. 26 Message ID, Parent ID, and Depth (Enron only) In Enron threads, (usually in xml format), you will see something like this at the top of each message (this example is from thread 181398train): This can be helpful in determining linking! If you see that one message is a parent of another, you may see that there is a Flink in the parent and a Blink in the “child”. Looking at the depth helps too: the higher the depth, the later the message, so in this case, depth 1 means that it is a response to the first message (which is of depth 0).

27. 27 Duplicate messages (Enron only) Sometimes, an email is (probably accidentally) sent more than once. If this happens, label by writing “DUPLICATE of [whatever the message ID is]” right after where it says “ ” For a response to a duplicate message, write “ORIGINAL PARENT: [whatever the original parent’s message ID is]” right after where it says “ ”

28. 28 Example of a duplicate message and a response to it from Enron thread 181398train, where to write “DUPLICATE”: and where to write “ORIGINAL PARENT”:

29. 29 Examples to Practice On-L1 Identify the types of requests and whether they were successful (from Loqui dialog 45):

30. 30 Examples to Practice On-L2 Identify the type of request and whether successful for these 2 examples (from Loqui dialog 73)

31. 31 Examples to Practice On-E1 ID thread type:

32. 32 Examples to Practice On-E2 ID thread type:

33. 33 Examples to Practice On-E3 ID thread type:

34. 34 Examples to Practice On-E4 Annotate this Enron thread:

35. 35 Homework Assignment #2 Annotate Enron email thread 42685train (the “train” just means it was originally intended as part of a training set) For this thread, make sure to look at the depth, message ID, and parent ID for each message Use a plain text editor, and again, create a.log.txt file noting any questions or things you were unsure about Make sure to format DFU labels correctly! No abbreviations (e.g. “request-info”), and include square brackets! (Links go underneath brackets)