1. Exploiting Wikipedia as External Knowledge for Document Clustering
Xiaohua Hu1, Xiaodan Zhang1, Caimei Lu1, E.K. Park2, Xiaohua Zhou1
1College of Information Science and Technology, Drexel University, Philadelphia, PA 19104, USA
2School of Computing and Engineering, University of Missouri at Kansas City, Kansas City, MO 64110, USA

2. Outline Introduction Framework
Concept Mapping: Exact and Relatedness Match
Category Mapping
Clustering
Experiments & Results
Conclusions & Future work

3. Introduction Problems of BOW-based clustering:
Ignores the relationships among words
If two documents use different collections of core words to represent the same topic, they would be assigned to different clusters.
Solution: enrich document representation with the background knowledge represented by an ontology.

4. Introduction
Two issues for enhancing text clustering by leveraging ontology semantics:
An ontology which can cover the topical domain of individual document collections as completely as possible.
A proper matching method which can enrich the document representation by fully leveraging ontology terms and relations without introducing more noise.
This paper aims to address both issues.

5. Wikipedia as Ontology The free encyclopedia that anyone can edit
Wikipedia is a free, multilingual encyclopedia project supported by the non-profit Wikimedia Foundation.
Wikipedia's articles have been written collaboratively by volunteers around the world.
Almost all of its articles can be edited by anyone who can access the Wikipedia website.
----

6. Wikipedia as Ontology
Unlike other standard ontologies, such as WordNet and Mesh, Wikipedia itself is not a structured thesaurus.
However, it is more…
Comprehensive: it contains 12 million articles (2.8 million in the English Wikipedia)
Accurate : A study by Giles (2005) found Wikipedia can compete with Encyclopædia Britannica in accuracy*.
Up to date: Current and emerging concepts are absorbed timely.
* Giles, J Internet encyclopaedias go head to head. Nature 438: 900–901.

7. Wikipedia as Ontology Moreover, Wikipedia has a well-formed structure
Each article only describes a single concept.
The title of the article is a short and well-formed phrase like a term in a traditional thesaurus.

8. Wikipedia Article that describes the Concept Artificial intelligence

9. Wikipedia as Ontology Moreover, Wikipedia has a well-formed structure
Each article only describes a single concept
The title of the article is a short and well-formed phrase like a term in a traditional thesaurus.
Equivalent concepts are grouped together by redirected links.

10. AI is redirected to its equivalent concept Artificial Intelligence

11. Wikipedia as Ontology Moreover, Wikipedia has a well-formed structure
Each article only describes a single concept
The title of the article is a short and well-formed phrase like a term in a traditional thesaurus.
Equivalent concepts are grouped together by redirected links.
It contains a hierarchical categorization system, in which each article belongs to at least one category.

12. The concept Artificial Intelligence belongs to four categories: Artificial intelligence, Cybernetics, Formal sciences & Technology in society

13. Wikipedia as Ontology Moreover, Wikipedia has a well-formed structure
Each article only describes a single concept
The title of the article is a short and well-formed phrase like a term in a traditional thesaurus.
Equivalent concepts are grouped together by redirected links.
It contains a hierarchical categorization system, in which each article belongs to at least one category.
Polysemous concepts are disambiguated by Disambiguation Pages.

14. The different meanings that Artificial intelligence may refer to are listed in its disambiguation page.

15. The Framework of leveraging Wikipedia for document clustering
Similarity metric between two documents:
SIM(d1, d2)= Sim(W1, W2)+ α∙Sim(C1, C2)
+ β∙Sim(Cat1, Cat2)

16. Concept Mapping Match documents to Wikipedia concepts.
Result: A document-concept Matrix
Matching Schemes:
Exact Match
Relatedness Match D1 D2
C C2
w11
w12
w21
w22
Wikipedia Concept
Documents

17. artificial Intelligence Wikipedia concept dictionary
1. Concept Mapping ---Exact Match d1
Artificial Intelligence, or AI for short, is a combination of computer science, physiology, and philosophy. …
document-concept Vector
Exact match
Wiki Concepts C
artificial Intelligence …
computer science
physiology
philosophy d1 2 1
Wikipedia concept dictionary
artificial Intelligence, AI …
computer science
physiology
philosophy

18. Concept Mapping Schemes: Exact Match
Each document is scanned to find Wikipedia concepts (article titles).
The searched Wikipedia concepts are used to comprise the concept vector of the corresponding document.
Synonymous phrases to the same concept are grouped together through the redirect links in Wikipedia.

19. Concept Mapping Schemes: Exact Match
A dictionary is constructed, with each entry corresponding to a topic covered by Wikipedia.
Each entry includes not only the preferred Wikipedia concept which is used as the title of the article, but also all the redirected concepts representing the same topic.
Based on the dictionary, both preferred concepts and redirected concepts are retrieved from documents.
Only preferred concepts are used to build the concept vector for each document. The weight of each preferred concept equals to the frequency of itself plus the frequencies of all the redirected concepts appearing in a document.
The document-concept TFIDF matrix is further calculated based on the document-concept frequency matrix.

20. Concept Mapping Schemes: Exact Match
High efficiency
Low recall. Only the concepts which explicitly appear in a document are extracted and used to construct the concept vector of the document.

21. 2. Concept Mapping –Relatedness Match
Wikipedia collection
Artificial Intelligence, or AI for short, is a combination of computer science, physiology, and philosophy. …
Wikipedia term concept matrix
term vector of document d1
Wiki concepts C c1 c2 … cM
Wiki articleterms W w1
tfidf11
tfidf12
tfidf1N w2
tfidf21
tfidf22
tfidf2N wN
tfidfN1
tfidfN2
tfidfNM
terms W w1 w2 … wN d1
tfidf11
tfidf12
tfidf1N
document-concept Vector
Wiki concepts C c1 c2 … cM d1
r11
r12
r1N

22. Concept Mapping Schemes: Relatedness Match
Step 1: A Wikipedia term-concept matrix is constructed from Wikipedia article collection.
The values in the matrix are TFIDF scores, which denote the relatedness of each term to each Wikipedia concept article.
For each word, only top k concepts with highest TFIDF scores are selected. (In this study, k = 5)

23. Concept Mapping Schemes: Relatedness Match
Step 2: The word-concept matrix is used as a bridge to build Document-Wikipedia concept Matrix. The relatedness of a Wikipedia concept to a given document is calculated as:
: the TFIDF score of word in document ; representing the importance of to
: the TFIDF score of word in concept ; representing the importance of to
For each document, only the top M concepts with highest relatedness score are selected. (In this experiment, M is set to 200)

24. Concept Mapping Schemes: Relatedness Match
More time consuming
Helpful for identify relevant Wikipedia concepts which are not explicitly present in a document.
Especially useful when Wikipedia concepts have less coverage for a dataset.

25. Category Mapping w11 w12 w21 w22 1 v11 v12 v21 v22 Wikipedia Category
C C2
w11
w12
w21
w22
Wikipedia Concept
Documents
CAT CAT2 C1 C2 1
Wikipedia Concepts
Wikipedia Category
v11
v12
v21
v22

26. (Exact Match or Relatedness Match)
3. Category Mapping
Concept Mapping
(Exact Match or Relatedness Match)
The hierarchical categorization system of Wikipedia
Wiki Concept-Category Matrix
document-concept Vector
Wiki categories CAT
cat1
cat2 …
catk
Wiki Concepts C c1 1 c2 cN
Wiki concepts C c1 c2 … cM d1
r11
r12
r1N
document-category Vector
Wiki categories CAT
cat1
cat2 …
catk d1
v11
v12
v1k

27. Category Mapping Exact match:
A document-category frequency matrix is first derived from the document-concept frequency matrix by replacing each concept with its corresponding categories.
If a category is mapped to a document through more than one concept, the sum of the frequencies of these concepts is the category’s frequency.
Based on the generated document-category frequency matrix, we further derive the document-category TFIDF matrix

28. Category Mapping Relatedness Match
The document-category matrix is generated by replacing each concept in the document-concept matrix with its corresponding categories, which share the same normalized relatedness score as the concept.
If a category is mapped to a document through more than one concept, its relatedness score to the document is the sum of the scores of all these concepts.

29. Document Clustering Agglomerative Clustering Partitional Clustering
and respectively indicate the importance of concept vector and category vector in measuring the similarity between two documents.

30. Experiments
Wikipedia data: 911,028 articles and about categories after pre-processing and filtering.
Clustering dataset: TDT2, LA Times (from TREC), and 20-newsgroups (20NG).
Evaluation Metrics:
Purity
F-score
NMI (normalized mutual information )

31. Experiments Clustering Schemes under Comparison Notation Explanation
Word
Clustering solely based on word vector
Concept
Clustering solely based on concept vector
Category
Clustering solely based on category vector
Word_Concept
Clustering based on the linear combination of word vector and concept vector
Word_Category
Clustering based on the linear combination of word vector and category vector
Concept_Category
Clustering based on the linear combination of concept vector and category vector
Word_Concept_Category
Clustering based on the linear combination of word vector, concept vector and category vector

32. Experiments
Parameter Setting (α and β that indicate the importance of concept vector and category vector in measuring the similarity between two documents.)
For Word_Concept scheme, β is set to zero and α is set to 0.1, 0.2, ∙∙∙, 1.0 respectively. The average result of the ten runs is used as the final clustering results for Word_Concept scheme.
For Word_Category scheme, α is set to zero and β is set to 0.1, 0.2, ∙∙∙, 1.0 respectively. The average result of the ten runs is used as the final clustering results for Word_Category scheme.
For Word_Concept_Category scheme, α is set to the value which produces best results for Word_Concept based clustering, and β is set to the value that generates best results for Word_Category based clustering.

33. Agglomerative clustering results
20 News Group
NMI
F-Score
Purity
Match Scheme EM RM
Word (BaseLine)
0.144
0.146
0.132
Concept
0.134
0.157
0.102
0.062
0.103
0.081
Cateogory
0.111
0.160*
0.128
0.143
0.114
Concept_Category
0.131
0.148
0.084
0.160
0.095
Word_Concept
0.150
0.153
0.168
0.136
Word_Category
0.166*
0.171*
0.189*
0.209*
0.201*
0.180*
Word_Concept_Category
0.154
0.196*
0.195*
0.206*
0.165
* indicates the improvement is significant according to the paired-sample T-test at the level of p<0.05.

34. Agglomerative clustering results
LA Times
NMI
F-Score
Purity
Match Scheme EM RM
Word (BaseLine)
0.048
0.066
0.124
Concept
0.060
0.073
0.057
0.044
0.120
0.113
Cateogory
0.071*
0.053
0.174
0.054
0.111
0.118
Concept_Category
0.177
0.202
Word_Concept
0.051
0.052
0.064
0.072
0.128
Word_Category
0.101*
0.049
0.210*
0.097*
0.238*
0.142*
Word_Concept_Category
0.103*
0.204*
0.100*
0.232*
0.144*
* indicates the improvement is significant according to the paired-sample T-test at the level of p<0.05.

35. Agglomerative clustering results
TDT2
NMI
F-Score
Purity
Match Scheme EM RM
Word (BaseLine)
0.537
0.622
0.600
Concept
0.296
0.372
0.398
0.483
0.368
0.463
Cateogory
0.577*
0.448
0.637
0.539
0.649*
0.549
Concept_Category
0.581*
0.444
0.656
0.543
0.659*
0.560
Word_Concept
0.563
0.609
0.689*
0.620
0.678*
Word_Category
0.695*
0.660*
0.754*
0.721*
0.769*
0.737*
Word_Concept_Category
0.675*
0.661*
0.734*
0.726*
0.751*
0.747*
* indicates the improvement is significant according to the paired-sample T-test at the level of p<0.05.

36. Partitional Clustering Results
20 News Group
NMI
F-Score
Purity
Match Scheme EM RM
Word (BaseLine)
0.390
0.382
0.411
Concept
0.288
0.313
0.312
0.372
0.302
0.401
Cateogory
0.291
0.326
0.332
0.383
0.341
0.391
Concept_Category
0.287
0.322
0.333
0.354
0.317
0.388
Word_Concept
0.380
Word_Category
0.409*
0.429*
0.402*
0.412*
0.430*
0.442*
Word_Concept_Category
0.398
0.412
0.400*
0.418*
* indicates the improvement is significant according to the paired-sample T-test at the level of p<0.05.

37. Partitional Clustering Results
LA Times
NMI
F-Score
Purity
Match Scheme EM RM
Word (BaseLine)
0.188
0.317
0.328
Concept
0.186
0.082
0.312
0.253
0.333
0.251
Cateogory
0.185
0.097
0.315
0.241
0.327
0.249
Concept_Category
0.190
0.112
0.310
0.242
0.329
0.245
Word_Concept
0.159
0.128
0.292
0.264
0.304
0.275
Word_Category
0.194
0.179
0.325
0.335
0.322
Word_Concept_Category
0.189
0.140
0.319
0.276
0.330
0.286
* indicates the improvement is significant according to the paired-sample T-test at the level of p<0.05.

38. Partitional Clustering Results
TDT2
NMI
F-Score
Purity
Match Scheme EM RM
Word (BaseLine)
0.790
0.825
0.848
Concept
0.556
0.447
0.622
0.522
0.647
0.544
Cateogory
0.577
0.448
0.637
0.539
0.649
0.549
Concept_Category
0.543
0.442
0.630
0.523
0.643
0.545
Word_Concept
0.787
0.766
0.815
0.792
0.840
0.819
Word_Category
0.804
0.737
0.830
0.720
0.854
0.763
Word_Concept_Category
0.802
0.833
0.846
0.876
* indicates the improvement is significant according to the paired-sample T-test at the level of p<0.05.

39. Conclusions
Category information is most useful for improving clustering results. In both agglomerative clustering and partitional clustering, combining category information with document content information generates the best results in most cases.
Clustering based on all three document vectors (word vector, concept vector, category vector) also gets significantly better results than the baseline, but does not outperform clustering based only on word vector and category vector.
Concept information is not as useful as category information for improving clustering performance due to the noisy information it contains and sense ambiguity problem.
The effect of category and concept information on k-means clustering is not as significant as it on agglomerative clustering. But, in most cases, Word_Category based clustering still achieves best performance among all clustering schemes.
The effect of the two mapping schemes depends on the dataset, quality metric and clustering approach.

40. Future Work
Introduce sense disambiguation functions into the concept mapping process by leveraging Wikipedia disambiguation pages.
Explore how to utilize the link structure among Wikipedia concepts for document clustering.

41. Q & A