1. Web People Search via Connection Analysis Authors : Dmitri V. Kalashnikov, Zhaoqi (Stella) Chen, Sharad Mehrotra, and Rabia Nuray-Turan From : IEEE Trans. on Knowledge and Data Engineering 2008 Presenter : 陳仲詠 Citation : 21 (Google Scholar) 1

2. Outline 1. Introduction 2. Overview of the approach 3. Generating a graph representation 4. Disambiguation algorithm 5. Interpreting clustering results 6. Related work 7. Experimental Results 8. Conclusions and Future work 2

3. Introduction (1/7) Searching for web pages related to a person accounts for more than 5 percent of the current Web searches [24]. A search for a person such as say “Andrew McCallum” will return pages relevant to any person with the name Andrew McCallum. [24] R. Guha and A. Garg, Disambiguating People in Search. Stanford Univ., 2004. 3

4. Introduction (2/7) Assume (for now) that for each such web page, the search-engine could determine which real entity (i.e., which Andrew McCallum) the page refers to. Provide a capability of clustered person search, the returned results are clustered by associating each cluster to a real person. 4

5. Introduction (3/7) The user can hone in on the cluster of interest to her and get all pages in that cluster. For example, only the pages associated with that Andrew McCallum. 5

6. Introduction (4/7) In reality, it is not obvious that it indeed is a better option compared to searching for people using keyword-based search. If clusters identified by the search engine corresponded to a single person, then the clustered-based approach would be a good choice. 6

7. Introduction (5/7) The key issue is the quality of clustering algorithms in disambiguating different web pages of the namesake. 7

8. Introduction (6/7) 1. Develop a novel algorithm for disambiguating among people that have the same name. 2. Design a cluster-based people search approach based on the disambiguation algorithm. 8

9. Introduction (7/7) The main contributions of this paper are the following : A new approach for Web People Search that shows high-quality clustering. A thorough empirical evaluation of the proposed solution (Section 7), and A new study of the impact on search of the proposed approach (Section 7.3). 9

10. Overview of the approach (1/4) The processing of a user query consists of the following steps: 1. User input : A user submits a query. 2. Web page retrieval : Retrieves a fixed number (top K) of relevant web pages. 10

11. Overview of the approach (2/4) 3. Preprocessing : – TF/IDF. noun phrase identification. – Extraction. Named entities (NEs) and Web-related information. 4. Graph creation : The entity-relationship (ER) graph is generated based on data extracted. 11

12. Overview of the approach (3/4) 5. Clustering : The result is a set of clusters of these pages with the aim being to cluster web pages based on association to real person. 12

13. Overview of the approach (4/4) 6. Cluster processing : – Sketches : A set of keywords that represent the web pages within a cluster. – Cluster ranking. – Web page ranking. 7. Visualization of results 13

14. Generating a graph representation (1/6) Extracted : 1)the entities 2)relationships 3)hyperlinks 4)e-mail addresses from the web pages. 14

15. Generating a graph representation (2/6) For example, a person “John Smith” might be extracted from two different web pages. 15 Doc 1 Doc 2 John Smith Regardless whether the two pages refer to the same person or to two different people.

16. Generating a graph representation (3/6) 16

17. Generating a graph representation (3/6) 17

18. Generating a graph representation (3/6) 18

19. Generating a graph representation (3/6) 19

20. Generating a graph representation (4/6) The relationship edges are typed. Any hyperlinks and e-mail addresses extracted from the web page are handled in an analogous fashion. 20

21. Generating a graph representation (5/6) A hyperlink has the form : For example, for the URL : www.cs.umass.edu/~ mccallum/ have d 3 = cs, d 2 = umass, d 1 = edu p 1 = ~mccallum. 21

22. Generating a graph representation (6/6) 22

23. Disambiguation algorithm 1. Input the entity relationship graph. 2. Uses a Correlation Clustering (CC) algorithm to cluster the pages. 3. The outcome is a set of clusters with each cluster corresponding to a person. 23

24. Disambiguation algorithm Correlation Clustering (1/3) CC has been applied in the past to group documents of the same topic and to other problems. It assumes that there is a similarity function s(u, v) learned on the past data. Each (u, v) edge is assigned a “+” (similar) or “-” (different) label, according to the similarity function s(u, v). 24

25. Disambiguation algorithm Correlation Clustering (2/3) The goal is to find the partition of the graph into clusters that agrees the most with the assigned labels. The CC does not take k (the number of the resulting clusters) as its input parameter. 25

26. Disambiguation algorithm Correlation Clustering (3/3) The goal of CC is formulated formally : – maximize the agreement – minimize the disagreement. The problem of CC is known to be NP-hard. 26

27. Disambiguation algorithm Connection Strength (1/6) Use the notion of the Connection Strength c(u, v) between two objects u and v to define the similarity function s(u, v). The disambiguation algorithm is based on analyzing : – object features and – the ER graph for the data set. 27

28. Disambiguation algorithm Connection Strength (2/6) A path between u and v semantically captures interactions between them via intermediate entities. If the combined attraction of all these paths is sufficiently large, the objects are likely to be the same. 28

29. Disambiguation algorithm Connection Strength (3/6) Analyzing paths : The assumption is that each path between two objects carries in itself a certain degree of attraction. 29

30. Disambiguation algorithm Connection Strength (4/6) The attraction between two nodes u and v via paths is measured using the connection strength measure c(u, v). Defined as the sum of attractions contributed by each path: 30

31. Disambiguation algorithm Connection Strength (5/6) P uv denotes the set of all L-short simple paths between u and v. – A path is L-short if its length does not exceed L and is simple if it does not contain duplicate nodes. w p denotes the weight contributed by path p. – The weight path p contributes is derived from the type of that path. 31

32. Disambiguation algorithm Connection Strength (6/6) Let P uv consist of c 1 paths of type 1, c 2 paths of type 2,... ; c n paths of type n. 32

33. Disambiguation algorithm Similarity Function (1/4) The goal is to design a powerful similarity function s(u, v) that would minimize mislabeling of the data. Design a flexible function s(u, v), such that it will be able to automatically self-tune itself to the particular domain being processed. 33

34. Disambiguation algorithm Similarity Function (2/4) The similarity function s(u, v) labels data by comparing the s(u, v) value against the threshold γ. Use the δ - band (“clear margin”) approach, label the edge (u, v). To avoid committing to + or - decision, when it does not have enough evidence for that. 34

35. Disambiguation algorithm Similarity Function (3/4) Employs the standard TF/IDF scheme to compute its feature-based similarity f(u, v). – Noun phrases – Larger terms The entire document corpus consists of K documents N distinct terms T = {t 1, t 2,...,t N }. 35

36. Disambiguation algorithm Similarity Function (4/4) Each document u : w ui is the weight 36

37. Disambiguation algorithm Training the Similarity Function (1/2) For each (u, v) edge, require that : In practice, s(u, v) is unlikely to be perfect and that would manifest itself in cases where the inequalities in (5) will be violated for some of the (u, v) edges It can be resolved in a similar manner by adding slack to each inequality in (5). 37

38. Disambiguation algorithm Training the Similarity Function (2/2) The task becomes to solve the linear programming problem (6) to determine the optimal values for path type weights w 1, w 2,…,w n and threshold γ. 38

39. Disambiguation algorithm Choosing Negative Weight (1/7) A CC algorithm will assign an entity u to a cluster if the number of positive edges between u and the other entities in the cluster outnumbers that of the negative edges. The number of positive edges is more than half (i.e., 50 percent). 39

40. Disambiguation algorithm Choosing Negative Weight (2/7) To keep an entity in a cluster, it is sufficient to have only 25 percent of positive edges. Using the w + =+1 weight for all positive edges and w - =-1/3 weight for all negative edges will achieve the desired effect. 40

41. Disambiguation algorithm Choosing Negative Weight (3/7) One solution for choosing a good value for the weight of negative edges w is to learn it on past data. The number of namesakes n in the top k web pages. – If n = 1, w - = 0 – All the pair connected via positive edges will be merged. 41

42. Disambiguation algorithm Choosing Negative Weight (4/7) – If n = k, it is best to choose w - = 1. – This would produce maximum negative evidence for pairs not to be merged. w - = w - (n) 42

43. Disambiguation algorithm Choosing Negative Weight (5/7) This observation raises two issues : – 1) n is not known to the algorithm beforehand. – 2) how to choose the w - (n) function. 43

44. Disambiguation algorithm Choosing Negative Weight (6/7) n is not known, compute its estimated value ^n by running the disambiguation algorithm with a fixed value of w -. The algorithm would output certain number of clusters ^n, which can be employed as an estimation of n. 44

45. Disambiguation algorithm Choosing Negative Weight (7/7) The value of w - (^n) : – when ^n < threshold, w - (^n) = 0. – when ^n > threshold, w - (^n) = -1. This threshold is learned from the data. 45

46. A brief Summary 46

47. Interpreting Clustering Results (1/4) Now describe how these clusters are used to build people search. The goal is to provide the user with a set of clusters based on association to real person. – 1. Rank the clusters. – 2. Provide a summary description with each cluster. 47

48. Interpreting Clustering Results (2/4) Cluster rank : – Select the highest ranked page. Cluster sketch : – The set of terms above a certain threshold is selected and used as a summary for the cluster. 48

49. Interpreting Clustering Results (3/4) Web page rank : – These pages are displayed according to their original search engine order. 49

50. Interpreting Clustering Results (4/4) Affinity to cluster : – Defined as the sum of the similarity values between the page p and each page v in the cluster C : The remainder pages are displayed, the user has the option to get to these web pages too. 50

51. Experimental Results Experimental Setup (1/8) The three data sets : – 1. WWW 2005 data set[8] : 12 different people names. – 2. WEPS data set : SemEval workshop [3], consist of : Trail : 9 person names. Training : 49 person names. Test : 30 persons names. 51 [3] J. Artiles, J. Gonzalo, and S. Sekine, “The SemEval-2007 WePS Evaluation: Establishing a Benchmark for the Web People Search Task,” Proc. Int’l Workshop Semantic Evaluations (SemEval ’07), June 2007. [8] R. Bekkerman and A. McCallum, “Disambiguating Web Appearances of People in a Social Network,” Proc. Int’l World Wide Web Conf. (WWW), 2005.

52. Experimental Results Experimental Setup (2/8) – 3. Context data set : Issuing nine queries to Google, each in the form of a person name along with context keywords. The top 100 returned web pages of the Web search were gathered for each person. 52

53. Experimental Results Experimental Setup (3/8) To get the “ground truth” for these data sets, the pages for each person name have then been assigned to distinct real persons by manual examination. 53

54. Experimental Results Experimental Setup (4/8) Used the GATE [19] system for the extraction of NEs from the web pages in the data set. To train the free parameters of algorithm, apply leave-one-out cross validation on WWW 2005, WEPS Trial, and Context data sets. 54 [19] H. Cunningham, D. Maynard, K. Bontcheva, and V. Tablan, “GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications,” Proc. Ann. Meeting of the Assoc. Computational Linguistics (ACL), 2002.

55. Experimental Results Experimental Setup (5/8) Before the “ground truth” for its WEPS Test portion was released, tested the approach on the WEPS Training set by a twofold cross validation. 55

56. Experimental Results Experimental Setup (6/8) After the “ground truth” of the WEPS Test portion became available, trained the algorithm on the whole WEPS Training portion and tested on the WEPS Test portion. 56

57. Experimental Results Experimental Setup (7/8) Quality evaluation measures : – the B-cubed [6] and F P measures. Baseline methods : – the Agglomerative Vector Space clustering algorithm with TF/IDF as the Baseline method. – The threshold parameter for this method is trained the same way 57

58. Experimental Results Experimental Setup (8/8) Statistical significance test : – 1-tailed paired t-test, with α = 0.05. 58

59. Testing Disambiguation Quality Experiment 1 (Disambiguation quality : overall) (1/7) 59

60. Testing Disambiguation Quality Experiment 1 (Disambiguation quality : overall) (2/7) * s(u, v) = c(u, v) represents the approach where only the connection strength is employed for disambiguation. * Relies only on the extracted NEs and hyperlink information, and it does not use the TF/IDF. 60

61. Testing Disambiguation Quality Experiment 1 (Disambiguation quality : overall) (3/7) * With the analysis of the features of web pages f(u, v), in the form of their TF/ IDF similarity. 61

62. Testing Disambiguation Quality Experiment 1 (Disambiguation quality : overall) (4/7) Picks w - according to the function w - (^n) of the predicted number of namesakes. Gains 7.8 percent improvement in terms of B-cubed over the baseline (WWW 2005 ). Gets 6.1 percent improvement (WEPS Training) and 10.7 percent improvement (WEPS Test). 62

63. Testing Disambiguation Quality Experiment 1 (Disambiguation quality : overall) (5/7) Also compare the results with the top runners in the WEPS challenge [3]. The first runner in the challenge reports 0.78 for F p and 0.70 for B-cubed measures. 63 [3] J. Artiles, J. Gonzalo, and S. Sekine, “The SemEval-2007 WePS Evaluation: Establishing a Benchmark for the Web People Search Task,” Proc. Int’l Workshop Semantic Evaluations (SemEval ’07), June 2007.

64. Testing Disambiguation Quality Experiment 1 ( Disambiguation quality per namesake ) (6/7) The “#” field shows the number of namesakes for a particular name in the corresponding 100 web pages. [4]J. Artiles, J. Gonzalo, and F. Verdejo, “A Testbed for People Searching Strategies in the WWW,” Proc. SIGIR, 2005. (C : 39) 64

65. Testing Disambiguation Quality Experiment 1 ( Disambiguation quality per namesake ) (7/7) The table shows that the proposed approach outperforms that in [4] by 9.5 percent in terms of the F P measure. 65 [4]J. Artiles, J. Gonzalo, and F. Verdejo, “A Testbed for People Searching Strategies in the WWW,” Proc. SIGIR, 2005. (C : 39)

66. Testing Disambiguation Quality Experiment 2 ( Disambiguation quality : group identification ) The 1,085 web pages of the WWW 2005 data set. The task is to find the web pages related to the meant N people. 66

67. Testing Disambiguation Quality Experiment 2 ( Disambiguation quality : group identification ) The field “#W” in Table 3 is the number of the to-be-found web pages related to the namesake of interest. The field “#C” is the number of web pages found correctly. The field “#I” is the number of pages found incorrectly in the resulting groups. 67

68. Testing Disambiguation Quality Experiment 3 ( Disambiguation quality: queries with context ) Generated a data set by querying Google with a person name and context keyword(s) that is related to that person. Used nine different queries. 68

69. Testing Disambiguation Quality Experiment 4 ( Quality of generating cluster sketches) The set of terms above a certain threshold (or top N terms) is selected and used as a summary for the cluster. If the search is for UMass professor Andrew McCallum, his cluster can easily be identified with the terms like “machine learning” and “artificial intelligence.” 69

70. Impact on Search In case of a traditional search interface, at each observation i, where i = 1, 2,…,K, the user looks at the sketch provided for the i-th returned web page. 70

71. Impact on Search For the new interface, supported by a cluster-based people search, the user first looks at the “people search” interface. 1.The user sequentially reads cluster sketches/ descriptions, until on the m-th observation the user find the cluster of interest. 2.Clicks on that cluster. 3.Shows the original set of K web pages returned by the search engine. 71

72. Impact on Search Measures : Compare the quality of the new and standard interface using Precision, Recall, and F-measure. In general, the fewer observations are needed in a given interface, the faster the user can find the related pages. 72

73. Experiment 5 (Impact on search) Case 1 : First-dominant cluster Observation StandardNew interface To discover 50 percent of the relevant pages. 4433 To discover 90 percent of the relevant pages. 9255 73

74. Experiment 5 (Impact on search) Case 2 : Regular cluster Observation StandardNew interface To discover 50 percent of the relevant pages. 5116 To discover 90 percent of the relevant pages. 7917 Andrew McCallum the Customer Support person. His cluster consists of three pages. 74

75. Experiment 5 (Impact on search) Case 3 : Average The average of Recall, Precision, and F measures for search impact on the WWW 2005. Some of the person names have many namesakes. Show that, even with the imperfect clustering, the curves for the new interface largely dominate those for the standard interface. 75

76. Experiment 5 (Impact on search) Impact on search with context In that case, one can expect to see no difference between the new and the standard interface. The query is “Andrew McCallum” music The number of namesakes for that query is surprisingly large: 23. Andrew McCallum the UMass professor, who is interested in music. 76

77. Experiment 5 (Impact on search) Impact on search with context Andrew McCallum the DJ/ musician. In both cases, the new interface performs better than the standard one. Observation StandardNew interface To discover 90 percent of the prof. 9060 To discover 90 percent of the DJ. 9020 77

78. Experiment 6 (efficiency) That takes 3.82 seconds per web page (downloads and preprocesses pages.) The clustering algorithm itself executes in 4.7 seconds on the average per queried name. 78

79. CONCLUSIONS AND FUTURE WORK Attempted to answer the question of which maximum quality the approach can get if it uses only the information stored in the top-k web pages being processed. Future work : 1. Employ external data sources for disambiguation. 2. Use more advances extraction capabilities. 3. Work on algorithms for a generic entity search, where entities are not limited to people. 79

80. Related Work Disambiguation and entity resolution techniques are key to any Web people search applications. 80

81. The differences among the disambiguation methodology in this paper and most related existing work are multilevel (see Table 1). 81

82. Level 1: Problem type. Two different common types of the disambiguation challenge: (fuzzy) Lookup [27], [28], and (fuzzy) Grouping [10], [13]. 82

83. Level 2: Data with respect to G L uv. *The methodology is based on analyzing G L uv in this paper. *The majority of the existing techniques do not analyze G L uv. 83

84. Name co-occurrence. [12] analyzes only co-occurrences of names of authors via publications for a publication data set. 84 [12]I. Bhattacharya and L. Getoor, “Collective Entity Resolution in Relational Data,” Bull. IEEE CS Technical Committee Data Eng., vol. 29, no. 2, pp. 4-12, June 2006.

85. Name co-occurrence. When analyzing authors A1 and A5, the approach in [10], [11], and [13] would only be interested in author A3, which is a co-occurring author in publications P1 and P2, which are connected to A1 and A5, respectively. 85 [10] I. Bhattacharya and L. Getoor, “Iterative Record Linkage for Cleaning and Integration,” Proc. ACM SIGMOD Workshop Research Issues in Data Mining and Knowledge Discovery (DMKD), 2004. [11] I. Bhattacharya and L. Getoor, “Relational Clustering for Multi- Type Entity Resolution,” Proc. Multi-Relational Data Mining Workshop (MRDM), 2005. [13] I. Bhattacharya and L. Getoor, “A Latent Dirichlet Model for Unsupervised Entity Resolution,” Proc. SIAM Data Mining Conf. (SDM), 2006.

86. Name co-occurrence. [12] would be interested only in the sub-graph shown in Fig. 5. The methodology in this paper instead analyzes the whole G L uv. 86 [12]I. Bhattacharya and L. Getoor, “Collective Entity Resolution in Relational Data,” Bull. IEEE CS Technical Committee Data Eng., vol. 29, no. 2, pp. 4-12, June 2006.

87. Restrictions on types. [12] understands only one type of relationship. The approach proposed here can analyze all of the types of relationships and entities. 87

88. *[26], [31], and [33] often still analyzes just portions of G L uv. * The adaptive approach in [33] analyzes G 2 uv, see Fig. 7. 88 [26] R. Holzer, B. Malin, and L. Sweeney, “Email Alias Detection Using Social Network Analysis,” Proc. ACM SIGKDD, 2005. [31] B. Malin, “Unsupervised Name Disambiguation via Social Network Similarity,” Proc. Workshop Link Analysis, Counterterrorism, and Security, 2005. [33] E. Minkov, W. Cohen, and A. Ng, “Contextual Search and Name Disambiguation in Email Using Graphs,” Proc. SIGIR, 2006.

89. *[31] simply looks at people and connects them via “are-related” 89 [31] B. Malin, “Unsupervised Name Disambiguation via Social Network Similarity,” Proc. Workshop Link Analysis, Counterterrorism, and Security, 2005.

90. *Level 3: Analysis of G L uv. *The methodology in this paper is based on analyzing paths in P uv and building mathematical models for c(u, v). * The existing work (e.g., [27], [28]) analyze the direct neighbors and [26] analyzes the shortest u-v path. 90 [26] R. Holzer, B. Malin, and L. Sweeney, “Email Alias Detection Using Social Network Analysis,” Proc. ACM SIGKDD, 2005. [27] D.V. Kalashnikov and S. Mehrotra, “Domain-Independent Data Cleaning via Analysis of Entity-Relationship Graph,” ACM Trans. Database Systems, vol. 31, no. 2, pp. 716-767, June 2006. [28] D.V. Kalashnikov, S. Mehrotra, and Z. Chen, “Exploiting Relationships for Domain- Independent Data Cleaning,” Proc. SIAM Int’l Conf. Data Mining (SDM ’05), Apr. 2005.

91. *Level 4 : Way to use c(u, v). *[10] and [11] employ agglomerative clustering. *[27], [28], the disambiguation problem is converted into an optimization problem, which is then solved iteratively. 91 [10] I. Bhattacharya and L. Getoor, “Iterative Record Linkage for Cleaning and Integration,” Proc. ACM SIGMOD Workshop Research Issues in Data Mining and Knowledge Discovery (DMKD), 2004. [11] I. Bhattacharya and L. Getoor, “Relational Clustering for Multi- Type Entity Resolution,” Proc. Multi- Relational Data Mining Workshop (MRDM), 2005. [27] D.V. Kalashnikov and S. Mehrotra, “Domain-Independent Data Cleaning via Analysis of Entity- Relationship Graph,” ACM Trans. Database Systems, vol. 31, no. 2, pp. 716-767, June 2006. [28] D.V. Kalashnikov, S. Mehrotra, and Z. Chen, “Exploiting Relationships for Domain-Independent Data Cleaning,” Proc. SIAM Int’l Conf. Data Mining (SDM ’05), Apr. 2005.

92. *Level 5: Domain independence. *Some of the existing techniques are applicable to only certain types of domains or just one domain. 92

93. Related Work WSD (1/3) Word Sense Disambiguation : – determine the exact sense of an ambiguous word given a list of word senses. Word Sense Discrimination : – determine, which instances of the ambiguous word can be clustered as sharing the same meaning. 93

94. Related Work WSD (2/3) External knowledge sources : – Using lexical knowledge associated with a dictionary and WordNet. Approach : – supervised – unsupervised 94

95. Related Work WSD (3/3) If view the ambiguous word as a reference and the word sense as an entity. The two instances of WSD problem are similar to the Lookup and Grouping instances of Entity Resolution/WePS. 95

96. Related Work WePS (1/4) WePS can be implemented in two different settings. – Server-side setting : the disambiguation mechanism is integrated into the search-engine directly. – Middleware approach : build people search capabilities on top of an existing search-engine such as Google by “wrapping” the original engine. 96

97. Related Work WePS (2/4) Clusty (http://www.clusty.com)http://www.clusty.com Grokker (http://www.grokker.com)http://www.grokker.com Kartoo (http://www.kartoo.com)http://www.kartoo.com 97

98. Related Work WePS (3/4) ZoomInfo (http://www.zoominfo.com) 98

99. Related Work WePS (4/4) But, this system has a high cost and low scalability. Because the person information in the systems is collected primarily manually. Does not rely on any such pre-compiled knowledge and thus will scale to person search for any person on the Web. 99