1. 1 Extraction and Summarization of Opinions

2. 2 Subjectivity: opinions, emotions, motivations, speculations, sentiments Information Extraction of –NL expressions –Components –Properties Angolans are terrified of the Marburg virus Source Attitude Target Negative Emotion Intensity: High Opinion Frame Source: Angolans Polarity: negative Attitude: emotion Intensity: high Target: Marburg virus

3. 3 Fine-grained Opinions Australian press has launched a bitter attack on Italy after seeing their beloved Socceroos eliminated on a controversial late penalty. Italian coach Lippi has also been blasted for his comments after the game. In the opposite camp Lippi is preparing his side for the upcoming game with Ukraine. He hailed 10- man Italy's determination to beat Australia and said the penalty was rightly given. [Stoyanov & Cardie, 2006]

4. 4 Fine-grained Opinion Extraction “The Australian Press launched a bitter attack on Italy” Opinion Frame Source: Australian Press Polarity: negative Attitude: sentiment Intensity: high Target: Italy

5. 5 Opinion Summary Australian Press Italy Marcello Lippi penalty Socceroos

6. 6 Summarization of Opinions + Events Summary Representation Direct Subjective Source: Polarity: Intensity: Direct Subjective Source: Polarity: Intensity: Opinion Frame Source: Polarity: Intensity: Disease Outbreak Victim: Location: Disease: Date: …

7. 7 Why Opinions? Provide technology that can aid analysts in their –extracting socio-behavioral information from text –monitoring public health awareness, knowledge and speculations about disease outbreaks, … Enrich Information Extraction, Question Answering, and Visualization tools

8. 8 Distinguish Objective from Subjective Language Speculations, hyperbole, emotions –The Chilean president admitted that “the US had been attacked” because “their initial measure was hasty. It amounted to using a tank to kill a flea.” Reflects argument credibility

9. 9 E.g., are people extremely afraid or angry? Opinion Frame Source: Polarity: negative Attitude: Intensity: high Target:

10. 10 The industry is scared and so, even if they do find an ornamental carp with KHV, they will keep it secret Recognize motivations Predict actions Opinion Frame Source: Polarity: Attitude: Intensity: Target:

11. 11 Brugere-Picoux backs the decision to ban British Beef Search for opinions about particular named targets Opinion Frame Source: Polarity: Attitude: Intensity: Target: Ban on British beef

12. 12 Brugere-Picoux backs the decision to ban British Beef Search for opinions held by particular named sources Opinion Frame Source: Brugere-Picoux Polarity: Attitude: Intensity: Target:

13. 13 Motivation for the Summaries Quickly determine the opinions of a person, organization, community, region, etc. Quickly determine the opinions toward a person, organization, issue, event, … –Across an entire document –Across multiple documents –Over time Reveal relationships and identify cliques and communities of interest Complement work in social network analysis

14. 14 Outline Motivations for opinion extraction Extracting opinion frames and components –Lexicon of subjective expressions –Contextual disambiguation –Enriched tasks Opinion summarization

15. 15 Lexicon Explore different uses of words, to zero in on the subjective ones Example: benefit

16. 16 Lexicon Example: benefit Very often objective, as a Verb: Children with ADHD benefited from a 15-course of fish oil

17. 17 Lexicon Noun uses look more promising: The innovative economic program has shown benefits to humanity

18. 18 Lexicon However, there are objective noun uses too: …tax benefits. …employee benefits. …tax benefits to provide a stable economy. …health benefits to cut costs.

19. 19 Lexicon Pattern: benefits as the head of a noun phrase containing a prepositional phrase Matches this: The innovative economic program has shown proven benefits to humanity But none of these: …tax benefits. …employee benefits. …tax benefits to provide a stable economy. …health benefits to cut costs.

20. 20 Lexicon Longer Constructions be soft on crime be 2 soft J 1 3 on 1 4 crime N

21. 21 The entry contains a pattern for finding instances of the construction Matches variations: –When I look into his past I see a man who is very soft on crime. –The data could also weaken her authority to criticize Patrick for being soft on crime.

22. 22 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

23. 23 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

24. 24 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

25. 25 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

26. 26 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

27. 27 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

28. 28 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

29. 29 Attributive information be soft on crime true h sen The Obama campaign rejected the notion that the senator might be vulnerable to accusations that he is soft on crime. vp s n m h 1:[morph:[lemma="be"] order:[distance="2" landmark="2"]] 2:[morph:[word="soft" majorClass="J"] order:[distance="1" landmark="3"]] 3:[morph:[word="on"] order:[distance="1" landmark="4"]] 4:[morph:[word="crime" majorClass="N"]] ngramPattern

30. 30 Lexicon: Summary Uniform representation for different types of subjectivity clues –Word stem: benefit –Word: benefits –Word/POS: benefits/nouns –Fixed n-grams: benefits to –Syntactic patterns –Combinations of the above Learn subjective uses from corpora (bodies of texts) Capture longer subjective constructions Add relevant knowledge about expressions Riloff, Wiebe, Wilson 2003; Riloff & Wiebe 2003; Wiebe & Riloff 2005; Riloff, Patwardhan, Wiebe 2006; Ruppenhofer, Akkaya, Wiebe in preparation

31. 31 Outline Motivations for opinion extraction Extracting opinion frames and components –Lexicon of subjective expressions –Contextual disambiguation –Enriched tasks Opinion summarization

32. 32 Polarity Contextual polarity There is no reason at all to believe that he’s the right choice Interacts with opinion topics Example: argument for one type of design is simultaneously an argument against an alternative design

33. 33 Polarity Recognizing contextual polarity using rich feature sets and machine learning Modeling and recognizing discourse relations among opinions and their targets in a text Wilson, Wiebe, Hoffmann EMNLP05 Wilson, Wiebe, Hoffmann, submitted Somasundaran, Wiebe, Ruppenhofer, submitted

34. 34 Opinion Frame Extraction via CRFs and ILP [Choi et al., EMNLP 2006] [Roth & Yih, 2004] CRFs [Lafferty et al., 2001] Joint extraction of entities and relations

35. 35 Opinion-Frame Extraction Joint extraction of entities and relations for opinion recognition (previous slide) –Choi, Break, Cardie EMNLP 2006 Linking sources referring to the same entity –Stoyanov and Cardie ACL 2006 Workshop on Sentiment and Subjectivity in Text Identifying expressions of opinions in context –Breck, Choi, Cardie IJCAI 2007

36. 36 Outline Motivations for opinion extraction Extracting opinion frames and components –Lexicon of subjective expressions –Contextual disambiguation –Enriched tasks Opinion summarization

37. 37 Targets and Attitude Types Wilson PhD Dissertation 2008 I think people are happy because Chavez has fallen. direct subjective span: are happy source: attitude: inferred attitude span: are happy because Chavez has fallen type: neg sentiment intensity: medium target: target span: Chavez has fallen target span: Chavez attitude span: are happy type: pos sentiment intensity: medium target: direct subjective span: think source: attitude: attitude span: think type: positive arguing intensity: medium target: target span: people are happy because Chavez has fallen

38. 38 Current Work: Topics Topic annotations added to the MPQA corpus –Annotations indicate the closest phrase to the opinion expression that adequately describes the topic of the opinion –Include topic “coreference” chains to link all phrases that describe the same topic concept –IAG results Stoyanov and Cardie LREC 2008

39. 39 Current Work: Topics Topic coreference resolution –Treat as an NP coreference resolution task –Modify our existing NP coref approach –Initial results look promising Using topic spans from gold standard –B 3 =.709 –MUC =.917 Topic span = opinion sentence –B 3 =.573 –MUC =.914 Topic span identified automatically –B 3 =.574 –MUC =.924 Best baseline system –B 3 =.554 –MUC =.793

40. 40 Subjectivity Types Wilson PhD Dissertation 2008

41. 41 Subjectivity Types Arguing and sentiment in the news and conversations –Manually annotating –Automatically detecting –Exploiting results of automatic detection to improve question answering Somasundaran, Wiebe, Hoffmann, Litman, ACL workshop 2006 Somasundaran, Wilson, Wiebe, Stoyanov ICWSM 2007 Somasundaran, Ruppenhofer, Wiebe SIGdial 2007 Ruppenhofer, Somasundaran, Wiebe LREC 2008

42. 42 Text Extraction and Data Visualization for Animal Health Surveillance Collaborative project between CERATOPS, PURVAC, and the Veterinary Information Network (VIN), with funding from LLNL. Goal: Study of subjectivity in health surveillance texts CERATOPS

43. 43 Method Manual Annotation Study –Identify relevant types of topic, source, and subjectivity –Annotate 16 texts from the ProMED (Program for Monitoring Emerging Diseases) mailing list

44. 44 Hypothesis A fine-grained study of subjectivity will show that health-surveillance texts contain significant amounts of subjectivity recognizing this subjectivity can enhance information extraction and question answering applications

45. 45 Example Whilst the present tragedy in the UK is extremely distressing to farmers …, so far the number of animals culled is only a miniscule portion of the national herd. Opinion Sentence-level annotation

46. 46 Example Whilst the present tragedy in the UK is extremely distressing to farmers …, so far the number of animals culled is only a miniscule portion of the national herd. Opinion Type: Sentiment Sentence-level annotation

47. 47 Example Whilst the present tragedy in the UK is extremely distressing to farmers …, so far the number of animals culled is only a miniscule portion of the national herd. Opinion Type: Sentiment Source: Affected individuals Sentence-level annotation

48. 48 Example Whilst the present tragedy in the UK is extremely distressing to farmers …, so far the number of animals culled is only a miniscule portion of the national herd. Opinion Type: Sentiment Source: Affected individuals Topic: Outbreak Sentence-level annotation

49. 49 Source types the writer medical experts media (non-media) organizations, including governments and agencies individuals affected by an outbreak members of the general public other explicitly mentioned entities implicit entities

50. 50 Source type example It has become clear that the UK has been importing significant animal products from areas where FMD is known to be endemic. Opinion Type: Knowledge / Awareness Source: Writer Topic: Other

51. 51 Topic types Occurrence of a disease outbreak Danger/severity of an outbreak Cause of a disease Symptoms Treatment Prevention Diagnosis Attitudes of others Development/progression of outbreak Other

52. 52 Topic type example The crisis has been caused by the koi herpes virus, commonly referred to as KHV, a disease harmless to other animals, but invariably fatal to carp. ‘Opinion’ Type: Objective Report Source: Writer Topic: Cause

53. 53 Topic type example The crisis has been caused by the koi herpes virus, commonly referred to as KHV, a disease harmless to other animals, but invariably fatal to carp.

54. 54 Topic type example The crisis has been caused by the koi herpes virus, commonly referred to as KHV, a disease harmless to other animals, but invariably fatal to carp. ‘Opinion’ Type: Objective Report Source: Writer Topic: Cause

55. 55 Subjectivity types (1) Sentiment Belief, distinguishing two sub-types – Beliefs about what is the case – Belief about what should or should not be done Knowledge & Awareness of facts Uncertainty & Speculation

56. 56 Subjectivity types (2) Agreement & Disagreement between various sources in the text Confirmation & Denial of contested statements Intention & Purpose Policies & Actions reflecting the above attitudes, for example, restrictions on the use, manufacture, distribution of substances

57. 57 Subjectivity type example Professor Jeanne Brugere-Picoux... said although France has officially registered 75 cases of BSE in the past 10 years, she believed the real figure to be “far higher than that”. Opinion Type: Belief About Facts Source: Medical expert Topic: Danger / Severity

58. 58 Subjectivity type example Nor did the FSA consider that there would be any need to label meat products derived from animals that have been vaccinated with the FMD vaccine. Opinion Type: Belief About Course Of Action Source: Organization Topic: Other

59. 59 Frequency of subjective types

60. 60 Querying the annotations 1 I am afraid people don’t know enough about this disease.

61. 61 Perform a query looking for sentences with –type = KnowledgeAwareness –topic = symptoms –source = member-of-public –polarity = negative

62. 62 Because the infection is much more likely in the summer, officials worry that this year’s tally may increase. One problem, Andrews said, is that many people don’t know that they have liver disease. As a result, she encourages people not to eat raw oysters from the Gulf Coast in the summer unless the oysters have been treated.

63. 63 Querying the annotations 2 What can we expect? How will this outbreak unfold?

64. 64 The analyst could query the annotations to retrieve sentences in which –type = UncertaintySpeculation –topic = DangerSeverity –source = expert or organization

65. 65 The number of infections this year is the highest since 5 people died within 3 weeks in 1996, Dassey said. Because the infection is much more likely in the summer, officials worry that this year’s tally may increase.

66. 66 Outline Motivations for opinion extraction Extracting opinion frames and components Opinion summarization

67. 67

68. 68 Querying the Opinion Frames & Summaries

69. 69 Timeline Format

70. 70 Expand to Reveal Opinion Holders

71. 71 DHS Expresses (neutral) Opinion

72. 72 Sortable List Format

73. 73 Drill Down to Original Article

74. 74 Juxtapose Opinions w/Other Info

75. 75 Summarization of Opinions + Events Summary Representation Direct Subjective Source: Polarity: Intensity: Direct Subjective Source: Polarity: Intensity: Opinion Frame Source: Polarity: Intensity: Disease Outbreak Victim: Location: Disease: Date: …