GRAPH-BASED HIERARCHICAL CONCEPTUAL CLUSTERING by Istvan Jonyer, Lawrence B. Holder and Diane J. Cook The University of Texas at Arlington

Outline What is hierarchical conceptual clustering? Overview of Subdue Conceptual clustering in Subdue Evaluation of hierarchical clusterings Experiments and results Conclusions

What is clustering?

What is hierarchical conceptual clustering? Unsupervised concept learning Generating hierarchies to explain data Applications – Hypothesis generation and testing – Prediction based on groups – Finding taxonomies

Example hierarchical conceptual clustering Animals BodyTemp: unregulated HeartChamber: four BodyTemp: regulated Fertilization: internal Fertilization: external Name: mammal BodyCover: hair Name: bird BodyCover: feathers Name: reptile BodyCover: cornified-skin HeartChamber: imperfect-four Fertilization: internal Name: fish BodyCover: scales HeartChamber: two Name: amphibian BodyCover: moist-skin HeartChamber: three

The Problem Hierarchical conceptual clustering in discrete-valued structural databases Existing systems: – Continuous-valued – Discrete but unstructured – We can do better! (Field under explored)

Related Work Cobweb Labyrinth AutoClass Snob In Euclidian space: Chameleon, Cure Unsupervised learning algorithms

The Solution Take Subdue and extend it!

Overview of Subdue Data mining in graph representations of structural databases A C BD A C BD F E f c b a d e a b c g

Overview of Subdue Iteratively searching for best substructure by MDL heuristic A C BD c b a

Overview of Subdue Compress using best substructure S S F E f d e g

Overview of Subdue Fuzzy match – Inexact matching of subgraphs – Applications: Defining fuzzy concepts Evaluation of clusterings

Conceptual Clustering with Subdue Use Subdue to identify clusters – The best subgraph in an iteration defines a cluster When to stop within an iteration? 1) Use –limit option 2) Use –size option 3) Use first minimum heuristic (new)

The First Minimum Heuristic Use subgraph at first local minimum – Detect it using –prune2 option

The First Minimum Heuristic Not a greedy heuristic! – Although first local minimum is usually the global minimum – First local minimum is caused by a smaller, more frequently occurring subgraph – Subsequent minima are caused by bigger, less frequently occurring subgraphs => First subgraph is more general

The First Minimum Heuristic A multi-minimum search space:

Lattice vs. Tree Previous work defined classification trees – Inadequate in structured domains Better hierarchical description: classification lattice – A cluster can have more than one parent – A parent can be at any level (not only one level above)

Hierarchical Clustering in Subdue Subdue can compress by a subgraph after each iteration Subsequent clusters may be defined in terms of previously defined clusters This results in a hierarchy

Hierarchical Conceptual Clustering of an Artificial Domain

Root

Evaluation of Clusterings Traditional evaluation: – Not applicable to hierarchical domains No known evaluation for hierarchical clusterings – Most hierarchical evaluations are anecdotal

New Evaluation Heuristic for Hierarchical Clusterings Properties of a good clustering: – Small number of clusters Large coverage  good generality – Big cluster descriptions More features  more inferential power – Minimal or no overlap between clusters More distinct clusters  better defined concepts

New Evaluation Heuristic for Hierarchical Clusterings Big clusters: bigger distance between disjoint clusters Overlap: less overlap  bigger distance Few clusters: averaging comparisons

Experiments and Results Validation in an artificial domain Validation in unstructured domains Comparison to existing systems Real world applications

The Animal Domain NameBody Cover Heart ChamberBody Temp.Fertilization mammalhairfourregulatedinternal birdfeathersfourregulatedinternal reptilecornified-skinimperfect-fourunregulatedinternal amphibianmoist-skinthreeunregulatedexternal fishscalestwounregulatedexternal animal hair mammal BodyCover Fertilization HeartChamber BodyTemp internalregulated Name four

Hierarchical Clustering of the Animal Domain Animals BodyTemp: unregulated HeartChamber: four BodyTemp: regulated Fertilization: internal Fertilization: external Name: mammal BodyCover: hair Name: bird BodyCover: feathers Name: reptile BodyCover: cornified-skin HeartChamber: imperfect-four Fertilization: internal Name: fish BodyCover: scales HeartChamber: two Name: amphibian BodyCover: moist-skin HeartChamber: three

Hierarchical Clustering of the Animal Domain by Cobweb animals amphibian/fish mammal/bird reptile mammalbird fishamphibian

Comparison of Subdue and Cobweb Quality of Subdue’s lattice (tree): 2.60 Quality of Cobweb’s tree: 1.74 Therefore Subdue is better Reasons for a higher score: – Better generalization resulting in less clusters – Eliminating overlap between (reptile) and (amphibian/fish)

Chemical Application: Clustering of a DNA sequence

Coverage – 61% – 68% – 71% DNA O | O == P — OH C — NC — C \ O | O == P — OH | O | CH 2 C \ N — C \ C O \ C / \ C — C N — C / \ O C

Conclusions Goal of hierarchical conceptual clustering of structured databases was achieved Synthesized classification lattice Developed new evaluation heuristic for hierarchical clusterings Good performance in comparison to other systems, even in unstructured domains

Future Work More experiments on real-world domains Comparison to other systems Incorporation of evaluation tool into Subdue