Clustering. 2 Outline  Introduction  K-means clustering  Hierarchical clustering: COBWEB.

Slides:

Advertisements

Similar presentations

Advertisements

Clustering k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein.

Data Set used. K Means K Means Clusters 1.K Means begins with a user specified amount of clusters 2.Randomly places the K centroids on the data set 3.Finds.

K-means algorithm 1)Pick a number (k) of cluster centers 2)Assign every gene to its nearest cluster center 3)Move each cluster center to the mean of its.

CS 478 – Tools for Machine Learning and Data Mining Clustering: Distance-based Approaches.

Unsupervised Learning Reading: Chapter 8 from Introduction to Data Mining by Tan, Steinbach, and Kumar, pp , , Chapter 8.

1 Machine Learning: Lecture 10 Unsupervised Learning (Based on Chapter 9 of Nilsson, N., Introduction to Machine Learning, 1996)

Unsupervised learning

Unsupervised learning: Clustering Ata Kaban The University of Birmingham

Clustering Shallow Processing Techniques for NLP Ling570 November 30, 2011.

Dimension reduction : PCA and Clustering Agnieszka S. Juncker Slides: Christopher Workman and Agnieszka S. Juncker Center for Biological Sequence Analysis.

1 Text Clustering. 2 Clustering Partition unlabeled examples into disjoint subsets of clusters, such that: –Examples within a cluster are very similar.

Unsupervised Learning: Clustering Some material adapted from slides by Andrew Moore, CMU. Visit for

Slide 1 EE3J2 Data Mining Lecture 16 Unsupervised Learning Ali Al-Shahib.

Introduction to Information Retrieval Introduction to Information Retrieval Hinrich Schütze and Christina Lioma Lecture 16: Flat Clustering 1.

Clustering. 2 Outline  Introduction  K-means clustering  Hierarchical clustering: COBWEB.

Unsupervised Learning and Data Mining

Semi-Supervised Clustering Jieping Ye Department of Computer Science and Engineering Arizona State University

Introduction to Bioinformatics - Tutorial no. 12

Dimension reduction : PCA and Clustering by Agnieszka S. Juncker Part of the slides is adapted from Chris Workman.

CLUSTERING (Segmentation)

Revision (Part II) Ke Chen COMP24111 Machine Learning Revision slides are going to summarise all you have learnt from Part II, which should be helpful.

Clustering Unsupervised learning Generating “classes”

Evaluating Performance for Data Mining Techniques

CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website:

Unsupervised Learning. CS583, Bing Liu, UIC 2 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate.

DATA MINING CLUSTERING K-Means.

Unsupervised Learning Reading: Chapter 8 from Introduction to Data Mining by Tan, Steinbach, and Kumar, pp , , (

START OF DAY 8 Reading: Chap. 14. Midterm Go over questions General issues only Specific issues: visit with me Regrading may make your grade go up OR.

Apache Mahout. Mahout Introduction Machine Learning Clustering K-means Canopy Clustering Fuzzy K-Means Conclusion.

Text Clustering.

Clustering Supervised vs. Unsupervised Learning Examples of clustering in Web IR Characteristics of clustering Clustering algorithms Cluster Labeling 1.

Basic Machine Learning: Clustering CS 315 – Web Search and Data Mining 1.

Clustering What is clustering? Also called “unsupervised learning”Also called “unsupervised learning”

MACHINE LEARNING 8. Clustering. Motivation Based on E ALPAYDIN 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2  Classification problem:

Unsupervised Learning. Supervised learning vs. unsupervised learning.

Information Retrieval Lecture 6 Introduction to Information Retrieval (Manning et al. 2007) Chapter 16 For the MSc Computer Science Programme Dell Zhang.

COMP Data Mining: Concepts, Algorithms, and Applications 1 K-means Arbitrarily choose k objects as the initial cluster centers Until no change,

Clustering I. 2 The Task Input: Collection of instances –No special class label attribute! Output: Clusters (Groups) of instances where members of a cluster.

CLUSTER ANALYSIS Introduction to Clustering Major Clustering Methods.

CLUSTERING AND SEGMENTATION MIS2502 Data Analytics Adapted from Tan, Steinbach, and Kumar (2004). Introduction to Data Mining.

Data Science and Big Data Analytics Chap 4: Advanced Analytical Theory and Methods: Clustering Charles Tappert Seidenberg School of CSIS, Pace University.

1Ellen L. Walker Category Recognition Associating information extracted from images with categories (classes) of objects Requires prior knowledge about.

K-Means Algorithm Each cluster is represented by the mean value of the objects in the cluster Input: set of objects (n), no of clusters (k) Output:

Data Mining Practical Machine Learning Tools and Techniques Chapter 4: Algorithms: The Basic Methods Section 4.8: Clustering Rodney Nielsen Many of these.

Clustering Clustering is a technique for finding similarity groups in data, called clusters. I.e., it groups data instances that are similar to (near)

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

Clustering. 2 Outline  Introduction  K-means clustering  Hierarchical clustering: COBWEB.

Machine Learning Queens College Lecture 7: Clustering.

Slide 1 EE3J2 Data Mining Lecture 18 K-means and Agglomerative Algorithms.

Compiled By: Raj Gaurang Tiwari Assistant Professor SRMGPC, Lucknow Unsupervised Learning.

Basic Machine Learning: Clustering CS 315 – Web Search and Data Mining 1.

1 Machine Learning Lecture 9: Clustering Moshe Koppel Slides adapted from Raymond J. Mooney.

Given a set of data points as input Randomly assign each point to one of the k clusters Repeat until convergence – Calculate model of each of the k clusters.

Machine Learning in Practice Lecture 21 Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute.

Data Mining and Text Mining. The Standard Data Mining process.

COMP24111 Machine Learning K-means Clustering Ke Chen.

Rodney Nielsen Many of these slides were adapted from: I. H. Witten, E. Frank and M. A. Hall Data Science Algorithms: The Basic Methods Clustering WFH:

Clustering. What is Clustering? Unsupervised learning Seeks to organize data into “reasonable” groups Often based on some similarity (or distance) measure.

Unsupervised Learning: Clustering

Unsupervised Learning: Clustering

Semi-Supervised Clustering

Constrained Clustering -Semi Supervised Clustering-

Machine Learning Lecture 9: Clustering

Data Mining 資料探勘分群分析 (Cluster Analysis) Min-Yuh Day 戴敏育

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Text Categorization Berlin Chen 2003 Reference:

Unsupervised Learning: Clustering

Data Mining CSCI 307, Spring 2019 Lecture 24

Presentation transcript:

Clustering

2 Outline  Introduction  K-means clustering  Hierarchical clustering: COBWEB

3 Classification vs. Clustering Classification: Supervised learning: Learns a method for predicting the instance class from pre-labeled (classified) instances

4 Clustering Unsupervised learning: Finds “natural” grouping of instances given un-labeled data

5 Clustering Methods  Many different method and algorithms:  For numeric and/or symbolic data  Deterministic vs. probabilistic  Exclusive vs. overlapping  Hierarchical vs. flat  Top-down vs. bottom-up

6 Clusters: exclusive vs. overlapping Simple 2-D representation Non-overlapping Venn diagram Overlapping a k j i h g f e d c b

7 Clustering Evaluation  Manual inspection  Benchmarking on existing labels  Cluster quality measures  distance measures  high similarity within a cluster, low across clusters

8 Simple Clustering: K-means Works with numeric data only 1)Pick a number (K) of cluster centers (at random) 2)Assign every item to its nearest cluster center (e.g. using Euclidean distance) 3)Move each cluster center to the mean of its assigned items 4)Repeat steps 2,3 until convergence (change in cluster assignments less than a threshold)

9 K-means example, step 1 k1k1 k2k2 k3k3 X Y Pick 3 initial cluster centers (randomly)

10 K-means example, step 2 k1k1 k2k2 k3k3 X Y Assign each point to the closest cluster center

11 K-means example, step 3 X Y Move each cluster center to the mean of each cluster k1k1 k2k2 k2k2 k1k1 k3k3 k3k3

12 K-means example, step 4 X Y Reassign points closest to a different new cluster center Q: Which points are reassigned? k1k1 k2k2 k3k3

13 K-means example, step 4 … X Y A: three points with animation k1k1 k3k3 k2k2

14 K-means example, step 4b X Y re-compute cluster means k1k1 k3k3 k2k2

15 K-means example, step 5 X Y move cluster centers to cluster means k2k2 k1k1 k3k3

16 Discussion  Result can vary significantly depending on initial choice of seeds  Can get trapped in local minimum  Example:  To increase chance of finding global optimum: restart with different random seeds instances initial cluster centers

17 K-means clustering summary Advantages  Simple, understandable  items automatically assigned to clusters Disadvantages  Must pick number of clusters before hand  All items forced into a cluster  Too sensitive to outliers

18 K-means variations  K-medoids – instead of mean, use medians of each cluster  Mean of 1, 3, 5, 7, 9 is  Mean of 1, 3, 5, 7, 1009 is  Median of 1, 3, 5, 7, 1009 is  Median advantage: not affected by extreme values  For large databases, use sampling