1. Learning Narrative Schemas Nate Chambers, Dan Jurafsky Stanford University IBM Watson Research Center Visit

2. Two Joint Tasks Events in a NarrativeSemantic Roles suspect, criminal, client, immigrant, journalist, government, … police, agent, officer, authorities, troops, official, investigator, …

3. Scripts Background knowledge for language understanding Restaurant Script Schank and Abelson. 1977. Scripts Plans Goals and Understanding. Lawrence Erlbaum. Mooney and DeJong. 1985. Learning Schemata for NLP. IJCAI-85. Hand-coded Domain dependent

4. Applications Coreference Resolve pronouns (he, she, it, etc.) Summarization Inform sentence selection with event confidence scores Aberration Detection Detect surprise/unexpected events in text Story Generation McIntyre and Lapata, (ACL-2009) Textual Inference Does a document infer other events Selectional Preferences Use chains to inform argument types

5. The Protagonist protagonist: (noun) 1.the principal character in a drama or other literary work 2.a leading actor, character, or participant in a literary work or real event

6. Inducing Narrative Relations 1.Dependency parse a document. 2.Run coreference to cluster entity mentions. 3.Count pairs of verbs with coreferring arguments. 4.Use pointwise mutual information to measure relatedness. Chambers and Jurafsky. Unsupervised Learning of Narrative Event Chains. ACL-08 Narrative Coherence Assumption Verbs sharing coreferring arguments are semantically connected by virtue of narrative discourse structure.

8. Chain Example (ACL-08)

9. Schema Example (new) Police, Agent, Authorities Judge, Official Prosecutor, Attorney Plea, Guilty, Innocent Suspect, Criminal, Terrorist, …

10. Narrative Schemas

11. Integrating Argument Types Use verb relations to learn argument types. Record head nouns of coreferring arguments. Use argument types to learn verb relations. Include argument counts in relation scores. The typhoon was downgraded Sunday as it moved inland from the coast, where it killed two people. downgrade-o, move-s, typhoon move-s, kill-s, typhoon downgrade-o, kill-s, typhoon

12. Learning Schemas

13. Argument Induction Suspect Government Journalist Monday Member Citizen Client … Induce semantic roles by scoring argument head words.

14. Training Data 1.2 million New York Times articles NYT portion of the Gigaword Corpus David Graff. 2002. English Gigaword. Linguistic Data Consortium. Stanford Parser http://nlp.stanford.edu/software/lex-parser.shtml OpenNLP coreference http://opennlp.sourceforge.net Lemmatize verbs and noun arguments.

15. Learned Examples court, judge, justice, panel, Osteen, circuit, nicolau, sporkin, majority law, ban, rule, constitutionality, conviction, ruling, lawmaker,

16. Learned Examples company, inc, corp, microsoft, iraq, co, unit, maker, … drug, product, system, test, software, funds, movie, …

17. Database of Schemas ~500 unique schemas, 10 events each Temporal ordering data Available online soon.

18. Evaluations Compared to FrameNet High precision when overlapping New type of knowledge not included Cloze Evaluation Predict missing events Far better performance than vanilla distributional approaches

19. Future Work Improved information extraction Extract information across multiple predicates. Knowledge Organization Link news articles describing subsequent events. Core AI Reasoning Automatic approach to learning causation? NLP specific tasks Coreference, summarization, etc.

20. Thanks! Unsupervised Learning of Narrative Schemas and their Participants Nathanael Chambers and Dan Jurafsky ACL-09, Singapore. 2009. Unsupervised Learning of Narrative Event Chains Nathanael Chambers and Dan Jurafsky ACL-08, Ohio, USA. 2008. Jointly Combining Implicit Constraints Improves Temporal Ordering Nathanael Chambers and Dan Jurafsky EMNLP-08, Waikiki, Hawaii, USA. 2008. Classifying Temporal Relations Between Events Nathanael Chambers, Shan Wang, Dan Jurafsky ACL-07, Prague. 2007.

21. Cloze Evaluation 1.Choose a news article at random. 2.Identify the protagonist. 3.Extract the narrative event chain. 4.Randomly remove one event from the chain. Predict which event was removed.

22. Cloze Results Outperform the baseline distributional learning approach by 36% Including participants improves further by 10%

23. Comparison to FrameNet Narrative Schemas Focuses on events that occur together in a narrative. FrameNet (Baker et al., 1998) Focuses on events that share core roles.

24. Comparison to FrameNet Narrative Schemas Focuses on events that occur together in a narrative. Schemas represent larger situations. FrameNet (Baker et al., 1998) Focuses on events that share core roles. Frames typically represent single events.

25. Comparison to FrameNet 1.How similar are schemas to frames? Find “best” FrameNet frame by event overlap 2.How similar are schema roles to frame elements? Evaluate argument types as FrameNet frame elements.

26. FrameNet Schema Similarity 1.How many schemas map to frames? 13 of 20 schemas mapped to a frame 26 of 78 (33%) verbs are not in FrameNet 2.Verbs present in FrameNet 35 of 52 (67%) matched frame 17 of 52 (33%) did not match

27. FrameNet Schema Similarity trade rise fall Exchange Change Position on a Scale Two FrameNet FramesOne Schema Why were 33% unaligned? FrameNet represents subevents as separate frames Schemas model sequences of events.

28. FrameNet Argument Similarity 2.Argument role mapping to frame elements. 72% of arguments appropriate as frame elements law, ban, rule, constitutionality, conviction, ruling, lawmaker, tax INCORRECT FrameNet frame: Enforcing Frame element: Rule

29. XX Event Scoring

30. XX Argument Induction Induce semantic roles by scoring argument head words. How often do events share any coreferring arguments? How often do they share argument ? = criminal?

31. Results Chains Schemas Typed Chains Typed Schemas 10.1%

32. Results 1.We learned rich narrative structure. 10.1% improvement over previous work 2.Induced semantic roles characterizing the participants in a narrative. 3.Verb relations and their semantic roles can be jointly learned and improve each other’s results. Selectional preferences improve verb relation learning.

33. XX Semantic Role Induction Supervised Learning PropBank (Palmer et al., 2005), FrameNet (Baker et al., 1998), VerbNet (Kipper et al., 2000) Bootstrapping from a seed corpus (Swier and Stevenson, 2004), (He and Gildea, 2006) Unsupervised, pre-defined roles (Grenegar and Manning 2006) WordNet inspired (Green and Dorr, 2005), (Alishahi and Stevenson, 2007) Suspect Government Journalist Monday Member Citizen Client …