Text Mining: Fast Phrase-based Text Indexing and Matching Khaled Hammouda, Ph.D. Student PAMI Research Group University of Waterloo Waterloo, Ontario, Canada LORNET Theme 4

The Problem Web / LOR Text Documents Web Documents Discussion Articles... Automatic Clustering/Grouping ProgrammingLanguages Database Systems Pattern Recognition How do we judge similarity? Data Mining

Group Similar Documents Together Maximize intra-cluster similarity Maximize intra-cluster similarity Minimize inter-cluster similarity Minimize inter-cluster similarity Need to accurately calculate document similarity Clustering Documents

Document Similarity How similar each document is to every other document? Very time consuming! O(n 2 )

Document Similarity Information Theoretic Measure (Dekang’98): How do we intersect every pair of documents without sacrificing efficiency? What features should we intersect? Words Words Phrases Phrases

Fast Phrase-based Document Indexing and Matching Document Index Graph Structure A model based on a digraph representation of the phrases in the document set A model based on a digraph representation of the phrases in the document set Nodes correspond to unique terms Nodes correspond to unique terms Edges maintain phrase representation Edges maintain phrase representation A phrase is a path in the graph A phrase is a path in the graph The model is an inverted list (terms  documents) The model is an inverted list (terms  documents) Nodes carry term weight information for each document in which they appear Nodes carry term weight information for each document in which they appear Shared phrases can be matched efficiently Shared phrases can be matched efficiently Phrase-based Features Phrases: more informative feature than individual words  local context matching Phrases: more informative feature than individual words  local context matching Represent sentences rather than words Represent sentences rather than words Facilitate phrase-matching between documents Facilitate phrase-matching between documents Achieves accurate document pair-wise similarity Achieves accurate document pair-wise similarity Avoid high-dimensionality of vector space model Avoid high-dimensionality of vector space model Allow incremental processing Allow incremental processing Document Index Graph

- river rafting - river - vacation plan - river - trips

Phrase-based Document Indexing Document Index Graph (internal structure) Document Index Graph (size scalability) Document Index Graph (time performance)

Effect of using phrase-based similarity over individual words Effect of using phrase similarity (F-measure)Effect of using phrase similarity (Entropy)

Applications Grouping search engine results on-the-fly (incremental processing) Creating taxonomies of documents (Yahoo! and Open Directory style) Implementing “Find Related” or “Find Similar” features of information retrieval systems Automatic generation of descriptive phrases about a set of documents (i.e. labeling clusters) Detecting plagiarism

Collaboration Provide Data Mining services (primarily text mining) for other groups Opportunity for collaboration with U of Saskatchewan: I-Help Discussion System I-Help Discussion System Course Delivery Tools Course Delivery Tools Others are welcome

Publications Journal Publications K. Hammouda and M. Kamel, “Efficient Phrase-based Document Indexing for Web Document Clustering”, IEEE Transactions on Knowledge and Data Engineering. Accepted, September K. Hammouda and M. Kamel, “Efficient Phrase-based Document Indexing for Web Document Clustering”, IEEE Transactions on Knowledge and Data Engineering. Accepted, September K. Hammouda and M. Kamel, “Document Similarity Using a Phrase Indexing Graph Model”, Knowledge and Information Systems. Springer. Accepted, May K. Hammouda and M. Kamel, “Document Similarity Using a Phrase Indexing Graph Model”, Knowledge and Information Systems. Springer. Accepted, May Conference Publications K. Hammouda and M. Kamel, “Incremental Document Clustering Using Cluster Similarity Histograms”, The 2003 IEEE/WIC International Conference on Web Intelligence (WI 2003), pp , Halifax, Canada, October 2003 K. Hammouda and M. Kamel, “Incremental Document Clustering Using Cluster Similarity Histograms”, The 2003 IEEE/WIC International Conference on Web Intelligence (WI 2003), pp , Halifax, Canada, October 2003 K. Hammouda and M. Kamel, “Phrase-based Document Similarity Based on an Index Graph Model”, The 2002 IEEE International Conference on Data Mining (ICDM'02), pp , Maebashi, Japan, December K. Hammouda and M. Kamel, “Phrase-based Document Similarity Based on an Index Graph Model”, The 2002 IEEE International Conference on Data Mining (ICDM'02), pp , Maebashi, Japan, December Available at: Available at:

Questions Instant Messaging MSN Messenger: MSN Messenger:

Text Documents Phrasal Indexing and Cohesive Document Clustering Phrase-based Text Indexing and Matching A model based on a digraph representation of the phrases in the document set A model based on a digraph representation of the phrases in the document set Nodes correspond to unique words Nodes correspond to unique words Edges maintain phrase representation Edges maintain phrase representation Phrases: more informative feature than individual words  local context matching Phrases: more informative feature than individual words  local context matching Facilitate phrase-matching between documents Facilitate phrase-matching between documents Achieves accurate document pair-wise similarity Achieves accurate document pair-wise similarity Avoid high-dimensionality of traditional vector space model Avoid high-dimensionality of traditional vector space model Allows incremental processing Allows incremental processing Document Clustering Similarity Histogram-based Clustering (SHC) Similarity Histogram-based Clustering (SHC) Clusters are represented using concise statistical representation called similarity histograms Clusters are represented using concise statistical representation called similarity histograms Maximize clusters coherency by maintaining high similarity distributions in clusters histograms Maximize clusters coherency by maintaining high similarity distributions in clusters histograms Enhance a cluster any time by re-distributing documents among clusters Enhance a cluster any time by re-distributing documents among clusters Both original and receiving clusters benefit from more tight similarity distributions Both original and receiving clusters benefit from more tight similarity distributions Phrase-based Document Index Graph Document Clustering Using Similarity Histograms