1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University

2 Contextual pattern and co-occurrences ? Spatial contexts provide useful cues for clustering

3 K-means revisit Assumption: data samples are independent Binary label indicator Limitation: contextual information of spatial dependency is not considered in clustering data samples EM Update

4 Clustering higher-level patterns Regularized k-means Distortion in original feature space Distortion in hamming space charactering contextual patterns Same as traditional K-means clustering Regularization term due to contextual patterns Not a smooth term!

5 Chicken and Egg Problem Hamming distance in clustering contextual patterns Matrix form Cannot minimize J1 and J2 separately ! J1 is coupled with J2

6 Decoupling Fix Update Fix Update

7 Nested-EM solution Nested E-step M-step Update and separately the nested-EM algorithm can converge in finite steps. Theorem of convergence

8 Simulation results (feature space)

9 Simulation results (spatial space)

10

11 K-means Initialization

12 1st round Final Phrases

13

14

15

16 Multiple-feature clustering Dataset: handwritten numerical (‘0’-‘9’) from UCI data set –Each digit has three different types of features –Contextual pattern corresponds to compositional feature Different types of features serve as contexts of each other –Clustering each type of features into 10 “words” –Clustering 10 “phrases” based on a word-lexicon of size 3x10

17 Conclusion A context-aware clustering formulation proposed –Targets on higher-level compositional patterns in terms of co-occurrences –Discovered contextual patterns can feed back to improve the primitive feature clustering An efficient nested-EM solution which is guaranteed to converge in finite steps Successful applications in image pattern discovery and multiple-feature clustering –Can be applied to other general clustering problems