Distance-based Clustering Assign a distance measure between data Find a partition such that: –Distance between objects within partition (i.e. same cluster) is minimized –Distance between objects from different clusters is maximised Issues : –Requires defining a distance (similarity) measure in situation where it is unclear how to assign it –What relative weighting to give to one attribute vs another? –Number of possible partition is super-exponential
Hierarchical Clustering Techniques At the beginning, each object (gene) is a cluster. In each of the subsequent steps, two closest clusters will merge into one cluster until there is only one cluster left.
Hierarchical Clustering Given a set of N items to be clustered, and an NxN distance (or similarity) matrix, the basic process hierarchical clustering is this: 1.Start by assigning each item to its own cluster, so that if you have N items, you now have N clusters, each containing just one item. Let the distances (similarities) between the clusters equal the distances (similarities) between the items they contain. 2.Find the closest (most similar) pair of clusters and merge them into a single cluster, so that now you have one less cluster. 3.Compute distances (similarities) between the new cluster and each of the old clusters. 4.Repeat steps 2 and 3 until all items are clustered into a single cluster of size N.
The distance between two clusters is defined as the distance between Single-Link Method / Nearest Neighbor (NN): minimum of pairwise dissimilarities Complete-Link / Furthest Neighbor (FN): maximum of pairwise dissimilarities Unweighted Pair Group Method with Arithmetic Mean (UPGMA): average of pairwise dissimilarities Their Centroids. Average of all cross-cluster pairs.
Computing Distances single-link clustering (also called the connectedness or minimum method) : we consider the distance between one cluster and another cluster to be equal to the shortest distance from any member of one cluster to any member of the other cluster. If the data consist of similarities, we consider the similarity between one cluster and another cluster to be equal to the greatest similarity from any member of one cluster to any member of the other cluster. complete-link clustering (also called the diameter or maximum method): we consider the distance between one cluster and another cluster to be equal to the longest distance from any member of one cluster to any member of the other cluster. average-link clustering : we consider the distance between one cluster and another cluster to be equal to the average distance from any member of one cluster to any member of the other cluster.
Single-Link Method b a Distance Matrix Euclidean Distance (1) (2) (3) a,b,c ccd a,b dd a,b,c,d
Complete-Link Method b a Distance Matrix Euclidean Distance (1) (2) (3) a,b ccd d c,d a,b,c,d
Ordered dendrograms 2 n-1 linear orderings of n elements (n= # genes or conditions) Maximizing adjacent similarity is impractical. So order by: Average expression level, Time of max induction, or Chromosome positioning Eisen98
Self organizing maps Tamayo et al. 1999 PNAS 96:2907-2912
Partitioning vs. Hierarchical Partitioning –Advantage: Provides clusters that satisfy some optimality criterion (approximately) –Disadvantages: Need initial K, long computation time Hierarchical –Advantage: Fast computation (agglomerative) –Disadvantages: Rigid, cannot correct later for erroneous decisions made earlier
Generic Clustering Tasks Estimating number of clusters Assigning each object to a cluster Assessing strength/confidence of cluster assignments for individual objects Assessing cluster homogeneity
Clustering and promoter elements Harmer et al. 2000 Science 290:2110-2113
Standard resolutionStandard resolution | High resolution Figure 1 Pathway alignment for glycolysis, Entner–Doudoroff pathway and pyruvate processing Enzymes for each pathway part (top; EC numbers and enzyme subunits are given below these) are compared in 17 organisms and represented as small rectangles. Filled and empty rectangles indicate the presence and absence respectively of enzyme-encoding genes in the different species listed at the left. Further details are given in the text; different isoenzymes and enzyme families are listed in Table 2.