1. Easy-First Coreference Resolution Veselin Stoyanov and Jason Eisner Johns Hopkins University

2. Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity

3. Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

4. Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

5. Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

6. Coreference Resolution Typical pairwise approaches But, inherently global task Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

7. Coreference Resolution Inherently global task Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

8. Coreference Resolution Inherently global task Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

9. Global Approaches to Coreference Resolution A difficult search problem – Exponentially many possible clusterings Recent approaches have been successful: – [Culotta et al., 2007] – train a global coherence measure; use greedy inference at test time – [Poon and Domingos, 2008] – use an unsupervised approach based on MLNs to enforce global properties

10. A Different Approach A Multi-Pass Sieve [Raghunathan et al., 2010] – Build coreference clusters incrementally – Applying sieves (hand-specified rules) of decreasing precision and increasing recall – Winner of the CoNLL-2011 shared task

11. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

12. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

13. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

14. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

15. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

16. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

17. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match …………. 7. Pronoun Matching Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

18. A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match …………. 7. Pronoun Matching Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

19. Easy-First Coreference Resolution Use the same idea: – Greedily build clusters – Using more information for the less confident decisions But learn automatically what constitutes an “easy” (confident) decision – Leads to empirical improvement over the Multi- Pass Sieve approach and a competitive baseline

20. Easy-First Coreference Resolution Incrementally and greedily build clusters – Using machine learning Learn policy for the test time inference procedure

21. Easy-First Coreference: Talk Overview Test-Time Inference Training Algorithm Experimental results

22. TEST-TIME INFERENCE

23. Test-Time Inference Begin with each NP in a separate cluster Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

24. Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson Cluster 3 SemType: person Gender: male Number: single Names: NIL Cluster 3 SemType: person Gender: male Number: single Names: NIL Begin with each NP in a separate cluster

25. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson

26. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False

27. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True

28. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True

29. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True RoleAppositive = True

30. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True RoleAppositive = True SentDistance = 0

31. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True RoleAppositive = True SentDistance = 0 Global features Local features

32. Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson f(join ij )

33. Compute a weight for each join action Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij )

34. Compute a weight for each join action Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) weight ij =

35. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) a ij =argmax ij

36. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) a 12 =argmax ij

37. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 15 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) a 12 =argmax ij

38. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

39. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

40. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

41. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

42. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

43. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

44. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

45. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

46. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

47. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

48. Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

49. Until the action with the highest weight is the special action HALT Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

50. Time complexity Let n = the number of mentions in a document At most n clusters at any time s = number of joins performed by the algorithm s<=n at most n joins before HALT (usually much less)

51. Time complexity Each step requires: – Popping the top action -- O(logn) (priority queue) – Performing the join – O(1) – Deleting O(n) old actions – O(nlogn) – Computing scores for O(n) new actions -- O(n) – Inserting those -- O(nlogn) or O(n) Total runtime = O(n 2 logn)+O(snlogn)

52. TRAINING ALGORITHM

53. Learning Learn weight vector w that leads the Easy-First algorithm to good clusterings At each step we are given a state s and a set of possible actions {join ij } ∪ {HALT} We want to learn what is “the best” action in each state But the current state s depends on the action taken so far

54. Reinforcement Learning A classic reinforcement learning setting Act in the world and learn “good” actions in each state We tried several reinforcement learning algorithms – Policy gradient [Sutton et al., 2000] – Q-learning [Watkins and Dayan, 1992] – No success: distant supervision; too many possible states

55. Perceptron-Style Training A version of the early update structured perceptron [Collins and Roark, 2004] – Inspired by the success of easy-first parsing [Goldberg and Elhadad, 2010] The algorithm: – Keep acting in the world until the next action will result in an error (reduction in score) – Update weights away from the top action and towards the highest scoring positive action

56. Training algorithm Perform N iterations: – For each document d in the training set do: Initialize C so that each mention in its own cluster Initialize A = {join ij } ∪ {HALT} repeat until a max != HALT: – for k steps do: » a max =A.peekMax() » if a max.isPositive() break » a pos =A.peekMaxPositive() » w+=(feat(a pos,C)-feat(a max,C)) » Recompute scores in A using new w update weight

57. Training algorithm Perform N iterations: – For each document d in the training set do: Initialize C so that each mention in its own cluster Initialize A = {join ij } ∩ {HALT} repeat until a max != HALT: – Update weights – a max =A.popMax() – if a max is a join » perform a max

58. EXPERIMENTAL EVALUATION

59. Experimental Setup Five datasets: – MUC6, MUC7, ACE03, ACE04, ACE05 Two evaluation metrics: – MUC score – B^3 score Implemented in the Reconcile research platform [Stoyanov et al. 2010] – Code available upon request Two settings: – Automatic mentions – Gold (manual) mentions

60. Baselines A pairwise baseline [Stoyanov et al. 2009] Reimplementation of the pairwise baseline using different features Easy-First coreference resolution – Easy-First percep – weights from the pairwise baseline – Easy-First struct – weights from our training algorithm Only for the gold mentions data – Entity-centered approach [Haghighi and Klein, 2009] – Multi-pass sieve [Raghunathan et al., 2010]

61. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4

62. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4 Pairwise baseline70.767.962.257.981.773.579.468.075.770.2

63. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4 Pairwise baseline70.767.962.257.981.773.579.468.075.770.2

64. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4 Pairwise baseline70.767.962.257.981.773.579.468.075.770.2

65. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4 Pairwise baseline70.767.962.257.981.773.579.468.075.770.2 Easy-First percep 71.568.464.458.182.874.180.368.476.070.0

66. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4 Pairwise baseline70.767.962.257.981.773.579.468.075.770.2 Easy-First percep 71.568.464.458.182.874.180.368.476.070.0 Easy-First struct 72.168.264.657.783.174.580.968.875.970.9

67. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4 Pairwise baseline70.767.962.257.981.773.579.468.075.770.2 Easy-First percep 71.568.464.458.182.874.180.368.476.070.0 Easy-First struct 72.168.264.657.783.174.580.968.875.970.9

68. Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile70.968.565.962.879.467.976.562.073.767.4 Pairwise baseline70.767.962.257.981.773.579.468.075.770.2 Easy-First percep 71.568.464.458.182.874.180.368.476.070.0 Easy-First struct 72.168.264.657.783.174.580.968.875.970.9

69. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1

70. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1 Stoyanov et al.76.188.277.876.2

71. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1 Stoyanov et al.76.188.277.876.2

72. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1 Stoyanov et al.76.188.277.876.2

73. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1 Stoyanov et al.76.188.277.876.2 Pairwise baseline72.788.275.978.9

74. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1 Stoyanov et al.76.188.277.876.2 Pairwise baseline72.788.275.978.9

75. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1 Stoyanov et al.76.188.277.876.2 Pairwise baseline72.788.275.978.9 Easy-First percep 73.388.278.479.8

76. Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein75.081.976.976.5 Raghunathan et al.73.277.778.978.1 Stoyanov et al.76.188.277.876.2 Pairwise baseline72.788.275.978.9 Easy-First percep 73.388.278.479.8 Easy-First struct 77.588.281.880.1

77. Conclusions A new approach to coreference using greedy incremental clustering – More accurate clustering by utilizing global information as it becomes available for less confident decisions Learning policy for the test-time inference procedure leads to improvements

78. THANK YOU!