1. Emily Pitler, Annie Louis, Ani Nenkova University of Pennsylvania

2. 2

3.  I am in Singapore, but I live in the United States. ◦ Explicit Comparison  The main conference is over Wednesday. I am staying for EMNLP. ◦ Implicit Comparison 3

4.  I am here because I have a presentation to give at ACL. ◦ Explicit Contingency  I am a little tired; there is a 13 hour time difference. ◦ Implicit Contingency 4

5.  Focus on implicit discourse relations ◦ in a realistic distribution  Better understanding of lexical features ◦ Showed do not capture semantic oppositions  Empirical validation of new and old features ◦ Polarity, verb classes, context, and some lexical features indicate discourse relations 5

6.  Classify both implicits and explicits ◦ Same sentence [Soricut and Marcu, 2003] ◦ Graphbank corpus: doesn’t distinguish implicit and explicit [ Wellner et al., 2006]  Create artificial implicits by deleting connective ◦ I am in Singapore, but I live in the United States. ◦ [Marcu and Echihabi, 2001; Blair-Goldensohn et al., 2007; Sporleder and Lascarides, 2008] 6

7. 7

8.  Most basic feature for implicits  I_there, I_is, …, tired_time, tired_difference 8 IamaIittletired thereisa13hourtimedifference Marcu and Echihabi, 2001

9.  The recent explosion of country funds mirrors the “closed-end fund mania of the 1920s, Mr. Foot says, when narrowly focused funds grew wildly popular.  They fell into oblivion after the 1929 crash. 9

10.  Using just content words reduces performance (but has steeper learning curve) ◦ Marcu and Echihabi, 2001  Nouns and adjectives don’t help at all ◦ Lapata and Lascarides, 2004  Filtering out stopwords lowers results ◦ Blair-Goldensohn et al., 2007 10

11.  Synthetic implicits: Cause/Contrast/None sentences ◦ Explicit instances from Gigaword with connective deleted ◦ Because  Cause, But  Contrast ◦ At least 3 sentences apart  None ◦ Blair-Goldensohn et al., 2007  Random selection ◦ 5,000 Cause ◦ 5,000 Other  Computed information gain of word pairs 11

12.  The government says it has reached most isolated townships by now, but because roads are blocked, getting anything but basic food supplies to people remains difficult.  but because  Comparison  but because  Contingency 12

13.  Maybe even with lots and lots of data, we won’t see “popular…but…oblivion” that often  What are we trying to get at?  PopularDesirableMollify  Oblivion AbhorrentEnrage 13

14. 14

15.  Multi-perspective Question Answering Opinion Corpus ◦ Wilson et. al, 2005  Sentiment words annotated as ◦ Positive ◦ Negative ◦ Both ◦ Neutral 15

16.  Similar to word pairs, but words replaced with polarity tags  Arg1: Executives at Time Inc. Magazine Co., a subsidiary of Time Warner, have said the joint venture with Mr. Lang wasn’t a good one.  Arg2: The venture, formed in 1986, was supposed to be Time’s low- cost, safe entry into women’s magazines. Arg1NegatePositiveArg2Positive 16

17.  General Inquirer lexicon ◦ Stone et al., 1966 ◦ Semantic categories of words  Complementary classes ◦ “Understatement” vs. “Overstatement” ◦ “Rise” vs. “Fall” ◦ “Pleasure” vs. “Pain”  Features ~ Tag pairs, only verbs 17

18.  Newsweek's circulation for the first six months of 1989 was 3,288,453, flat from the same period last year  U.S. News' circulation in the same time was 2,303,328, down 2.6%  Probably WSJ-specific 18

19.  Levin verb class level in LCS database ◦ Levin, 1993; Dorr, 2001 ◦ More related verbs ~ Expansion  Average length of verb chunk ◦ They [are allowed to proceed] ~ Contingency ◦ They [proceed] ~ Expansion, Temporal  POS tags of the main verb ◦ Same tense ~ Expansion ◦ Different tense ~ Contingency, Temporal 19

20.  Prior work found first and last words very helpful in predicting sense ◦ Wellner et al., 2006 ◦ Often explicit connectives 20

21.  Was preceding/following relation explicit? ◦ If so, which sense? ◦ If so, which connective?  Does Arg1 begin a paragraph? 21

22.  Largest available annotated corpus of discourse relations ◦ Penn Treebank WSJ articles ◦ 16,224 implicit relations between adjacent sentences  I am a little tired; [because] there is a 13 hour time difference. ◦ Contingency.cause.reason 22

23. Relation Sense Proportion of implicits Expansion53% Contingency26% Comparison15% Temporal 6% 23

24.  Developed features on sections 0-1  Trained on sections 2-20  Tested on sections 21-22  Binary classification task for each sense  Trained on equal numbers of positive and negative examples  Tested on natural distribution  Naïve Bayes classifier 24

25. 25

26.  Motivation in prior work ◦ Train on synthetic implicits 26 17.13 31.10 20.96 43.79 21.96 45.60  What works better ◦ Train on actual implicits Synthetic examples can still help! Comp. Cont. ◦ With only best features selected from synthetic implicits

27. Featuresf-score First-Last, First321.01 Context19.32 Money/Percent/Num19.04 Random9.91 27 Polarity is actually the worst feature 16.63

28. Comparison Not Comparison Positive-Negative or Negative-Positive Pairs 30%31% 28

29. Featuresf-score First-Last, First336.75 Verbs36.59 Context29.55 Random19.11 29

30. Featuresf-score Polarity Tags71.29 Inquirer Tags70.21 Context67.77 Random64.74 30 Expansion is majority class precision more problematic than recall These features all help other senses

31. Featuresf-score First-Last, First315.93 Verbs12.61 Context12.34 Random5.38 31 Temporals often end with words like “Monday” or “yesterday”

32.  Comparison ◦ Selected word pairs  Contingency ◦ Polarity, Verb, First/Last, Modality, Context, Selected word pairs 32

33.  Expansion ◦ Polarity, Inquirer Tags, Context  Temporal ◦ First/Last+word pairs 33

34. Comparison 21.96 (17.13) Contingency 47.13 (31.10) Expansion 76.41 (63.84) Temporal 16.76 (16.21) 34 Comparison/Contingency baseline: synthetic implicits word pairs Expansion/Temporal baseline: real implicits word pairs

35.  Results from classifying each relation independently ◦ Naïve Bayes, MaxEnt, AdaBoost  Since context features were helpful, tried CRF  6-way classification, word pairs as features ◦ Naïve Bayes accuracy: 43.27% ◦ CRF accuracy: 44.58% 35

36.  Focus on implicit discourse relations ◦ in a realistic distribution  Better understanding of word pairs ◦ Showed do not capture semantic oppositions  Empirical validation of new and old features ◦ Polarity, verb classes, context, and some lexical features indicate discourse relations 36