1. Document Clustering Based on Non-negative Matrix Factorization
Wei Xu et al, SIGIR’03
Presented by Weilin Xu, 11/06/2014

3. Motivation Document clustering applications
Narrow
Broad
Variety of information needs
Document clustering applications
News clustering
Extractive summarization …
Why traditional clustering methods not applicable?
Agglomerative clustering (bottom-up hierarchical)
Cons: Computationally expensive
Document partitioning (flat clustering), e.g. K-means
Cons: Harsh simplifying assumptions on distributions
Hierarchical: computationally expensive (O(n2logn))
Flat: based on weak assumptions, compact shape clusters in K-means, independent dimensions in Naïve Bayes, Gaussian distribution assumption in Gaussian mixture model.

4. Motivation Existing document clustering methods
Latent semantic indexing (LSI)
Singular Value Decomposition
Negative coefficients
Graph-based spectral clustering
Presented by Jinghe
Leads to computing singular vectors or eigenvectors
Equivalent to each other under certain conditions.
Both need additional clustering methods.
LSI: project each document into the singular vector space, then conducts document clustering using traditional clustering
Graph: lead to the computation of singular vectors or eigenvectors of certain graph affinity matrices.

5. Proposed method Non-negative Matrix Factorization-based clustering
𝑋≈𝑈 𝑉 𝑇
U: term-concept matrix; V: document-concept matrix.
Non-negative coefficients
Latent semantic space is not necessarily orthogonal
Factorization, Approximation

6. Roadmap
Representations
Method
NMF vs. SVD
Experiment
Summary

7. Representations
𝑋 𝑚×𝑛 = 𝑥 11 ⋯ 𝑥 1𝑛 ⋮ ⋱ ⋮ 𝑥 𝑚1 ⋯ 𝑥 𝑚𝑛 , term-document matrix
E.g. TF-IDF vector of document i
𝑋 𝑚×𝑛 ≈ 𝑈 𝑚×𝑘 𝑉 𝑛×𝑘 𝑇
𝑈 𝑚×𝑘 : term-topic matrix,
𝑉 𝑛×𝑘 : document-topic matrix

8. Method 𝑋≈𝑈 𝑉 𝑇 Minimize the objective function:
Derivation process (omitted)
Updating equations:
Normalize U:
Normalize U to get unique solution of U and V,
Frobenius norm
Convergence is guaranteed

9. NMF vs. SVD

10. Experiment - Corpora
#doc
#cluster

11. Experiment – Compared methods
Spectral clustering: Average Association (AA)
Equivalent to [LSI + K-means], if <Xi,Xj> similarity
Spectral clustering: Normalized Cut (NC)
Standard form of NMF (NMF)
NC weighted form (NMF-NCW)

12. Experiment - Evaluations
Accuracy
Mutual information
Normalized:
Map(), find by Kuhn-Munkres algorithm.

13. Experiment - Results 4 2 3 1
NMF-NCW > NC > NMF > AA

14. Experiment - Results 4 2 3 1
NMF-NCW > NC > NMF > AA

15. Summary NMF-based document partitioning method
Differ from SVD-based LSI and spectral clustering
Latent semantic space not need to be orthogonal
Non-negative values in latent semantic directions
Without additional clustering
Outperform the best methods.

16. Thanks