Presentation is loading. Please wait.

Presentation is loading. Please wait.

DATA MINING Introductory and Advanced Topics Part II - Clustering

Published byÁgoston Bodnár Modified over 5 years ago

Similar presentations

Presentation on theme: "DATA MINING Introductory and Advanced Topics Part II - Clustering"— Presentation transcript:

1 DATA MINING Introductory and Advanced Topics Part II - Clustering
Margaret H. Dunham Department of Computer Science and Engineering Southern Methodist University Companion slides for the text by Dr. M.H.Dunham, Data Mining, Introductory and Advanced Topics, Prentice Hall, 2002. Part II - Clustering © Prentice Hall

2 Clustering Outline Clustering Problem Overview Hierarchical Algorithms
Goal: Provide an overview of the clustering problem and introduce some of the basic algorithms Clustering Problem Overview Hierarchical Algorithms Partitional Algorithms Categorical Data Summary © Prentice Hall

3 Clustering Examples Segment customer database based on similar buying patterns. Group houses in a town into neighborhoods based on similar features. Identify new plant species Identify similar Web usage patterns © Prentice Hall

4 Clustering Example © Prentice Hall

5 Geographic Distance Based
Clustering Houses Size Based Geographic Distance Based © Prentice Hall

6 Clustering vs. Classification
No prior knowledge Number of clusters Meaning of clusters Unsupervised learning © Prentice Hall

7 Clustering Issues Outlier handling Dynamic data Interpreting results
Evaluating results Number of clusters Data to be used Scalability © Prentice Hall

8 Impact of Outliers on Clustering
© Prentice Hall

9 Clustering Problem Given a database D={t1,t2,…,tn} of tuples and an integer value k, the Clustering Problem is to define a mapping f:Dg{1,..,k} where each ti is assigned to one cluster Kj, 1<=j<=k. A Cluster, Kj, contains precisely those tuples mapped to it. Unlike classification problem, clusters are not known a priori. © Prentice Hall

10 Types of Clustering Hierarchical – Nested set of clusters created.
Partitional – One set of clusters created. Incremental – Each element handled one at a time. Simultaneous – All elements handled together. Overlapping/Non-overlapping © Prentice Hall

11 Clustering Approaches
Hierarchical Partitional Categorical Large DB Agglomerative Divisive Sampling Compression © Prentice Hall

12 Cluster Parameters © Prentice Hall

13 Distance Between Clusters
Single Link: smallest distance between points Complete Link: largest distance between points Average Link: average distance between points Centroid: distance between centroids © Prentice Hall

14 Hierarchical Clustering
Clusters are created in levels actually creating sets of clusters at each level. Agglomerative Initially each item in its own cluster Iteratively clusters are merge together Bottom Up Divisive Initially all items in one cluster Large clusters are successively divided Top Down © Prentice Hall

15 Hierarchical Algorithms
Single Link MST Single Link Complete Link Average Link © Prentice Hall

16 Dendrogram Dendrogram: a tree data structure which illustrates hierarchical clustering techniques. Each level shows clusters for that level. Leaf – individual clusters Root – one cluster A cluster at level i is the union of its children clusters at level i+1. © Prentice Hall

17 Levels of Clustering © Prentice Hall

18 Agglomerative Example
B C D E 1 2 3 4 5 A B E C D Threshold of 1 2 3 4 5 A B C D E © Prentice Hall

19 MST Example A B A B C D E 1 2 3 4 5 E C D © Prentice Hall

20 Agglomerative Algorithm
© Prentice Hall

21 Single Link View all items with links (distances) between them.
Finds maximal connected components in this graph. Two clusters are merged if there is at least one edge which connects them. Uses threshold distances at each level. Could be agglomerative or divisive. © Prentice Hall

22 MST Single Link Algorithm
© Prentice Hall

23 Single Link Clustering
© Prentice Hall

24 Partitional Clustering
Nonhierarchical Creates clusters in one step as opposed to several steps. Since only one set of clusters is output, the user normally has to input the desired number of clusters, k. Usually deals with static sets. © Prentice Hall

25 Partitional Algorithms
MST Squared Error K-Means Nearest Neighbor PAM BEA GA © Prentice Hall

26 MST Algorithm © Prentice Hall

27 Squared Error Minimized squared error © Prentice Hall

28 Squared Error Algorithm
© Prentice Hall

29 K-Means Initial set of clusters randomly chosen.
Iteratively, items are moved among sets of clusters until the desired set is reached. High degree of similarity among elements in a cluster is obtained. Given a cluster Ki={ti1,ti2,…,tim}, the cluster mean is mi = (1/m)(ti1 + … + tim) © Prentice Hall

30 K-Means Example Given: {2,4,10,12,3,20,30,11,25}, k=2
Randomly assign means: m1=3,m2=4 K1={2,3}, K2={4,10,12,20,30,11,25}, m1=2.5,m2=16 K1={2,3,4},K2={10,12,20,30,11,25}, m1=3,m2=18 K1={2,3,4,10},K2={12,20,30,11,25}, m1=4.75,m2=19.6 K1={2,3,4,10,11,12},K2={20,30,25}, m1=7,m2=25 © Prentice Hall

31 K-Means Algorithm © Prentice Hall

32 Nearest Neighbor Items are iteratively merged in the existing clusters that are closest. Incremental Threshold, t, used to determine if items are added to existing clusters or a new cluster is created. © Prentice Hall

33 Nearest Neighbor Algorithm
© Prentice Hall

34 PAM Partitioning Around Medoids (PAM) (K-Medoids)
Handles outliers well. Ordering of input does not impact results. Does not scale well. Each cluster represented by one item, called the medoid. Initial set of k medoids randomly chosen. © Prentice Hall

35 PAM © Prentice Hall

36 PAM Cost Calculation At each step in algorithm, medoids are changed if the overall cost is improved. Cjih – cost change for an item tj associated with swapping medoid ti with non-medoid th. © Prentice Hall © Prentice Hall

37 PAM Algorithm © Prentice Hall

38 BEA Bond Energy Algorithm Database design (physical and logical)
Vertical fragmentation Determine affinity (bond) between attributes based on common usage. Algorithm outline: Create affinity matrix Convert to BOND matrix Create regions of close bonding © Prentice Hall

39 BEA © Prentice Hall

40 Genetic Algorithm Example
{A,B,C,D,E,F,G,H} Randomly choose initial solution: {A,C,E} {B,F} {D,G,H} or , , Suppose crossover at point four and choose 1st and 3rd individuals: , , What should termination criteria be? © Prentice Hall

41 GA Algorithm © Prentice Hall

42 Clustering Large DB Most clustering algorithms assume a large data structure which is memory resident. Clustering may be performed first on a sample of the database then applied to the entire database. Algorithms BIRCH DBSCAN CURE © Prentice Hall

Download ppt "DATA MINING Introductory and Advanced Topics Part II - Clustering"

Similar presentations

CLUSTERING.

CLUSTERING.
CS 478 – Tools for Machine Learning and Data Mining Clustering: Distance-based Approaches.

CS 478 – Tools for Machine Learning and Data Mining Clustering: Distance-based Approaches.
SEEM4630 2011-2012 Tutorial 4 – Clustering. 2 What is Cluster Analysis?  Finding groups of objects such that the objects in a group will be similar (or.

SEEM Tutorial 4 – Clustering. 2 What is Cluster Analysis?  Finding groups of objects such that the objects in a group will be similar (or.
Clustering.

Clustering.
Cluster Analysis: Basic Concepts and Algorithms

Cluster Analysis: Basic Concepts and Algorithms
1 CSE 980: Data Mining Lecture 16: Hierarchical Clustering.

1 CSE 980: Data Mining Lecture 16: Hierarchical Clustering.
Hierarchical Clustering, DBSCAN The EM Algorithm

Hierarchical Clustering, DBSCAN The EM Algorithm
PARTITIONAL CLUSTERING

PARTITIONAL CLUSTERING
CS690L: Clustering References:

CS690L: Clustering References:
CS685 : Special Topics in Data Mining, UKY The UNIVERSITY of KENTUCKY Clustering CS 685: Special Topics in Data Mining Spring 2008 Jinze Liu.

CS685 : Special Topics in Data Mining, UKY The UNIVERSITY of KENTUCKY Clustering CS 685: Special Topics in Data Mining Spring 2008 Jinze Liu.
Data Mining Techniques: Clustering

Data Mining Techniques: Clustering
2004/05/03 Clustering 1 Clustering (Part One) Ku-Yaw Chang Assistant Professor, Department of Computer Science and Information.

2004/05/03 Clustering 1 Clustering (Part One) Ku-Yaw Chang Assistant Professor, Department of Computer Science and Information.
Clustering II.

Clustering II.
Clustering… in General In vector space, clusters are vectors found within  of a cluster vector, with different techniques for determining the cluster.

Clustering… in General In vector space, clusters are vectors found within  of a cluster vector, with different techniques for determining the cluster.
© University of Minnesota Data Mining for the Discovery of Ocean Climate Indices 1 CSci 8980: Data Mining (Fall 2002) Vipin Kumar Army High Performance.

© University of Minnesota Data Mining for the Discovery of Ocean Climate Indices 1 CSci 8980: Data Mining (Fall 2002) Vipin Kumar Army High Performance.
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,

Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach,
© Prentice Hall1 DATA MINING TECHNIQUES Introductory and Advanced Topics Eamonn Keogh (some slides adapted from) Margaret Dunham Dr. M.H.Dunham, Data Mining,

© Prentice Hall1 DATA MINING TECHNIQUES Introductory and Advanced Topics Eamonn Keogh (some slides adapted from) Margaret Dunham Dr. M.H.Dunham, Data Mining,
Cluster Analysis.  What is Cluster Analysis?  Types of Data in Cluster Analysis  A Categorization of Major Clustering Methods  Partitioning Methods.

Cluster Analysis.  What is Cluster Analysis?  Types of Data in Cluster Analysis  A Categorization of Major Clustering Methods  Partitioning Methods.
Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.

Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.
© Prentice Hall1 DATA MINING Introductory and Advanced Topics Part II Margaret H. Dunham Department of Computer Science and Engineering Southern Methodist.

© Prentice Hall1 DATA MINING Introductory and Advanced Topics Part II Margaret H. Dunham Department of Computer Science and Engineering Southern Methodist.

Similar presentations

Ads by Google