1. Probabilistic Equational Reasoning Arthur Kantor akantor@uiuc.edu

2. 2 Outline The Problem Solution –Undirected Graphical Model approach

3. 3 The Problem - example Objects She Powell References Mr. Powell … ? =

4. 4 The Problem - example Objects She Powell References Mr. Powell … = References can disambiguate each other

5. 5 … The Problem Objects X1X1 X2X2 XiXi XnXn References Specify the arrows … …

6. 6 Outline The Problem Solution –Undirected Graphical Model approach

7. 7 Undirected Graphical Model Objects are implicit – we deal only with references Given: –References x 1 … x i … x n –Y ij are binary random variables Y ij =1 iff x i co-references x j –f l (x i,x j,y ij ) are feature or potential functions measure particular similarity facet between x i and x j have the property f l (x i,x j,1) = - f l (x i,x j,0) non-zero, if informative

8. 8 Objective function Maximize Becomes -  if are linked but don’t form a clique. Penalizes triangle inconsistencies. Biggest if all the similar (x i,x j ) are connected and all dissimilar (x i,x j ) are separated ‘s are weights assigned to features

9. 9 Objective function are learned by maximum likelihood over the training datalearned The function is concave, so we can use our favorite learning algorithm (e.g. stochastic gradient ascent)

10. 10 Graph partitioning Maximizing is equivalent to finding an optimal graph partitioning of a complete graph The nodes are The edges are the log- potential functions applied to that pair of references

11. 11 X5X5 X3X3 X1X1 X2X2 X4X4 Correlation Clustering Graph G=(V,E) Try to partition V into clusters s.t. + edges are within clusters –edges are across clusters. Number of clusters is not specified. Edges are weighted + + – – – – + – – –

12. 12 Agreements and Disagreements Agreements + edges inside clusters AND – edges outside clusters. X5X5 X3X3 X1X1 X2X2 X4X4 + + – – – – + – – –

13. 13 Can either Maximize Agreements OR Minimize Disagreements Disagreements (mistakes) + edges outside clusters AND – edges inside clusters. Agreements and Disagreements Agreements + edges inside clusters AND – edges outside clusters. X5X5 X3X3 X1X1 X2X2 X4X4 + + – – – – + – – –

14. 14 X5X5 X3X3 X1X1 X2X2 X4X4 Q>0.5 is always possible If Q can be 1, algorithm is trivial + + – – – – + – – – This partition is worse than the previous one Q=13/15 Partition quality measure

15. 15 Clustering Algorithms Let be an optimal partition Finding is NP-Hard Algorithms exist to find partition so that Algorithm complexity:complexity

16. 16 A few observations Number of objects is determined automatically –Bound only by n Features are defined pairwise, but decision to join a clique involves all references

17. The End Thank you

18. 18 Gradient Ascent Log-likelihood: Update rule: Objective function with currently best ‘s Calculating is expensive. Can also randomly pick some and use Where are given by the MLE of back

19. 19 Complexity back A probabilistic inference problem cannot be constant-factor-approximated in polynomial time. –We can reduce a 3-SAT problem to it Our Clustering Algorithm guarantees absolute error less than  in Are we a probabilistic inference problem in polynomial time. No. –The algorithm is polynomial in the graph size, not in accuracy.

20. 20 Proper Noun co-reference ‘s:

21. 21 Proper Nouns performance Tested on –30 news wire articles –117 stories from broadcast news portion of DARAPA’s ACE set –Hand-annotated nouns (non-proper nouns ignored) –Identical feature functions on all three sets! –5-fold validation –Only 60% accuracy if proper nouns are included