1. The following slides have been adapted from http://www.tm4.org/http://www.tm4.org/ to be presented at the Follow-up course on Microarray Data Analysis (Nov 20-24 2006, PICB Shanghai) by Peter Serocka

2. Analysis of Multiple Experiments TIGR Multiple Experiment Viewer (MeV)

3. The Expression Matrix is a representation of data from multiple microarray experiments. Each element is a log ratio (usually log 2 (Cy5 / Cy3) ) Red indicates a positive log ratio, i.e, Cy5 > Cy3 Green indicates a negative log ratio, i.e., Cy5 < Cy3 Black indicates a log ratio of zero, i. e., Cy5 and Cy3 are very close in value Exp 1Exp 2Exp 3Exp 4Exp 5Exp 6 Gene 1 Gene 2 Gene 3 Gene 4 Gene 5 Gene 6 Gray indicates missing data

4. Expression Vectors -Gene Expression Vectors encapsulate the expression of a gene over a set of experimental conditions or sample types. -0.8 0.8 1.5 1.8 0.5 -1.3 -0.4 1.5 Log2(cy5/cy3)

5. Expression Vectors As Points in ‘Expression Space’ Experiment 1 Experiment 2 Experiment 3 Similar Expression -0.8 -0.6 0.91.2 -0.3 1.3 -0.7 Exp 1Exp 2Exp 3 G1 G2 G3 G4 G5 -0.4 -0.8 -0.7 1.30.9 -0.6

6. Distance and Similarity -the ability to calculate a distance (or similarity, it’s inverse) between two expression vectors is fundamental to clustering algorithms -distance between vectors is the basis upon which decisions are made when grouping similar patterns of expression -selection of a distance metric defines the concept of distance

7. Distance: a measure of similarity between genes. Exp 1Exp 2Exp 3Exp 4Exp 5Exp 6 Gene A Gene B x 1A x 2A x 3A x 4A x 5A x 6A x 1B x 2B x 3B x 4B x 5B x 6B Some distances: (MeV provides 11 metrics) 1.Euclidean:  i = 1 (x iA - x iB ) 2 6 2.Manhattan:  i = 1 |x iA – x iB | 6 3. Pearson correlation p0p0 p1p1

8. Distance is Defined by a Metric Euclidean Pearson(r*-1)Distance Metric : 4.2 1.4 -0.90 D D

9. Algorithms…

10. Hierarchical Clustering (HCL) HCL is an agglomerative clustering method which joins similar genes into groups. The iterative process continues with the joining of resulting groups based on their similarity until all groups are connected in a hierarchical tree. (HCL-1)

11. Hierarchical Clustering g8g1g2g3g4g5g6g7 g1g8g2g3g4g5g6g7g1g8g4g2g3g5g6 g1 is most like g8 g4 is most like {g1, g8} (HCL-2)

12. g7g1g8g4g2g3g5g6 g1g8g4g2g3g5g7 g6g1g8g4g5g7g2g3 Hierarchical Clustering g5 is most like g7 {g5,g7} is most like {g1, g4, g8} (HCL-3)

13. g6g1g8g4g5g7g2g3 Hierarchical Tree (HCL-4)

14. Hierarchical Clustering During construction of the hierarchy, decisions must be made to determine which clusters should be joined. The distance or similarity between clusters must be calculated. The rules that govern this calculation are linkage methods. (HCL-5)

15. Agglomerative Linkage Methods Linkage methods are rules or metrics that return a value that can be used to determine which elements (clusters) should be linked. Three linkage methods that are commonly used are: Single Linkage Average Linkage Complete Linkage (HCL-6)

16. Cluster-to-cluster distance is defined as the minimum distance between members of one cluster and members of the another cluster. Single linkage tends to create ‘elongated’ clusters with individual genes chained onto clusters. D AB = min ( d(u i, v j ) ) where u  A and v  B for all i = 1 to N A and j = 1 to N B Single Linkage (HCL-7) D AB

17. Cluster-to-cluster distance is defined as the average distance between all members of one cluster and all members of another cluster. Average linkage has a slight tendency to produce clusters of similar variance. D AB = 1/(N A N B )  ( d(u i, v j ) ) where u  A and v  B for all i = 1 to N A and j = 1 to N B Average Linkage (HCL-8) D AB

18. Cluster-to-cluster distance is defined as the maximum distance between members of one cluster and members of the another cluster. Complete linkage tends to create clusters of similar size and variability. D AB = max ( d(u i, v j ) ) where u  A and v  B for all i = 1 to N A and j = 1 to N B Complete Linkage (HCL-9) D AB

19. Comparison of Linkage Methods SingleAve.Complete (HCL-10)

20. 1. Specify number of clusters, e.g., 5. 2. Randomly assign genes to clusters. G1G2G3G4G5G6G7G8G9G10G11G12G13 K-Means / K-Medians Clustering (KMC)– 1

21. K-Means Clustering – 2 3. Calculate mean / median expression profile of each cluster. 4. Shuffle genes among clusters such that each gene is now in the cluster whose mean / median expression profile (calculated in step 3) is the closest to that gene’s expression profile. G1G2G3G4G5G6 G7 G8G9G10 G11 G12 G13 5. Repeat steps 3 and 4 until genes cannot be shuffled around any more, OR a user-specified number of iterations has been reached. K-Means / K-Medians is most useful when the user has an a-priori hypothesis about the number of clusters the genes should group into.

22. Cluster Affinity Search Technique (CAST) -uses an iterative approach to segregate elements with ‘high affinity’ into a cluster -the process iterates through two phases -addition of high affinity elements to the cluster being created -removal or clean-up of low affinity elements from the cluster being created

23. Clustering Affinity Search Technique (CAST)-1 Affinity = a measure of similarity between a gene, and all the genes in a cluster. Threshold affinity = user-specified criterion for retaining a gene in a cluster, defined as %age of maximum affinity at that point 1. Create a new empty cluster C1. 3. Move the two most similar genes into the new cluster. Empty cluster C1 G2 G4 G9 G8 G12 G6 G1 G7 G13 G11 G14 G3 G5G15 G10 Unassigned genes 4. Update the affinities of all the genes (new affinity of a gene = its previous affinity + its similarity to the gene(s) newly added to the cluster C1) 2. Set initial affinity of all genes to zero 5. While there exists an unassigned gene whose affinity to the cluster C1 exceeds the user-specified threshold affinity, pick the unassigned gene whose affinity is the highest, and add it to cluster C1. Update the affinities of all the genes accordingly. ADD GENES:

24. CAST – 2 6. When there are no more unassigned high-affinity genes, check to see if cluster C1 contains any elements whose affinity is lower than the current threshold. If so, remove the lowest-affinity gene from C1. Update the affinities of all genes by subtracting from each gene’s affinity, its similarity to the removed gene. 7. Repeat step 6 while C1 contains a low-affinity gene. 8. Repeat steps 5-7 as long as changes occur to the cluster C1. REMOVE GENES: 9. Form a new cluster with the genes that were not assigned to cluster C1, repeating steps 1-8. 10. Keep forming new clusters following steps 1-9, until all genes have been assigned to a cluster Current cluster C1 G2 G4 G9 G8 G12 G6 G1 G7 G13 G11 G14 G3 G5 G15 G10 Unassigned genes

25. QT-Clust (from Heyer et. al. 1999) (HJC) -1 1.Compute a jackknifed distance between all pairs of genes (Jackknifed distance: The data from one experiment are excluded from both genes, and the distance is calculated. Each experiment is thus excluded in turn, and the maximum distance between the two genes (over all exclusions) is the jackknifed distance. This is a conservative estimate of distance that accounts for bias that might be introduced by single outlier experiments.) 2. Choose a gene as the seed for a new cluster. Add the gene which increases cluster diameter the least. Continue adding genes until additional genes will exceed the specified cluster diameter limit. G4 G6 G5 G8 G7 G9 G10 G2 G3 G11 G1 “Seed” gene Currently unassigned genes Current cluster G11 G12 3. Repeat step 2 for every gene, so that each gene has the chance to be the seed of a new cluster. All clusters are provisional at this point.

26. QT-Clust – 2 4. Choose the largest cluster obtained from steps 2 and 3. In case of a tie, pick one of the largest clusters at random. 5. All genes that are not in the cluster selected above are treated as currently unassigned. Repeat steps 2-4 on these unassigned genes. 6. Stop when the last cluster thus formed has fewer genes than a user-specified number. All genes that are not in a cluster at this point are treated as unassigned. G1 “Seed” gene G11 G12G7 G8 G2 “Seed” gene G11 G10 G3 G4 G1 G5 G9 G7 G8 G3 “Seed” gene G9 G4 Pick this cluster

27. Self Organizing Tree Algorithm Dopazo, J., J.M Carazo, Phylogenetic reconstruction using and unsupervised growing neural network that adopts the topology of a phylogenetic tree. J. Mol. Evol. 44:226-233, 1997. Herrero, J., A. Valencia, and J. Dopazo. A hierarchical unsupervised growing neural network for clustering gene expression patterns. Bioinformatics, 17(2):126-136, 2001. SOTA - 1

28. SOTA Characteristics Divisive clustering, allowing high level hierarchical structure to be revealed without having to completely partition the data set down to single gene vectors Data set is reduced to clusters arranged in a binary tree topology The number of resulting clusters is not fixed before clustering Neural network approach which has advantages similar to SOMs such as handling large data sets that have large amounts of ‘noise’ SOTA - 2

29. SOTA Topology Parent Node Winning Cell Sister Cell pp ww ss    migration factor (  s <  p <  w ) SOTA - 3 Centroid Vector Members

30. Adaptation Overview -each gene vector associated with the parent is compared to the centroid vector of its offspring cells. -the most similar cell’s centroid and its neighboring cells are adapted using the appropriate migration weights. SOTA - 4

31. -following the presentation of all genes to the system a measure of system diversity is used to determine if training has found an optimal position for the offspring. -if the system diversity improves (decreases) then another training epoch is started otherwise training ends and a new cycle starts with a cell division. SOTA - 5

32. The most ‘diverse’ cell is selected for division at the start of the next training cycle. SOTA - 6

33. Growth Termination Expansion stops when the most diverse cell’s diversity falls below a threshold. SOTA - 7

34. Each training cycle ends when the overall tree diversity ‘stabilizes’. This triggers a cell division and possibly a new training cycle. SOTA - 8

35. Self-organizing maps (SOMs) – 1 1. Specify the number of nodes (clusters) desired, and also specify a 2-D geometry for the nodes, e.g., rectangular or hexagonal N = Nodes G = Genes G1G6 G3 G5 G4 G2 G11 G7 G8 G10 G9 G12G13 G14 G15 G19 G17 G22 G18 G20 G16 G21 G23 G25 G24 G26G27 G29G28 N1N2 N3N4 N5N6

36. SOMs – 2 2. Choose a random gene, e.g., G9 3. Move the nodes in the direction of G9. The node closest to G9 (N2) is moved the most, and the other nodes are moved by smaller varying amounts. The further away the node is from N2, the less it is moved. G1G6 G3 G5 G4 G2 G11 G7 G8 G10 G9 G12G13 G14 G15 G19 G17 G22 G18 G20 G16 G21 G23 G25 G24 G26G27 G29G28 N1N2 N3N4 N5N6

37. SOM Neighborhood Options G11 G7 G8 G10 G9 N1N2 N3N4 N5N6 G11 G7 G8 G10 G9 N1N2 N3N4 N5N6 Bubble Neighborhood Gaussian Neighborhood radius All move, alpha is scaled. Some move, alpha is constant.

38. SOMs – 3 4. Steps 2 and 3 (i.e., choosing a random gene and moving the nodes towards it) are repeated many (usually several thousand) times. However, with each iteration, the amount that the nodes are allowed to move is decreased. 5. Finally, each node will “nestle” among a cluster of genes, and a gene will be considered to be in the cluster if its distance to the node in that cluster is less than its distance to any other node G1G6 G3 G5 G4 G2 G11 G7 G8 G10 G9 G12G13 G14 G15 G19 G17 G22 G18 G20 G16 G21 G23 G25 G24 G26G27 G29G28 N1 N2 N3 N4 N5 N6

39. Compute first principle component of expression matrix Shave off  % (default 10%) of genes with lowest values of dot product with 1 st principal component Orthogonalize expression matrix with respect to the average gene in the cluster and repeat shaving procedure Repeat until only one gene remains Results in a series of nested clusters Choose cluster of appropriate size as determined by gap statistic calculation Gene Shaving

40. Gap statistic calculation (choosing cluster size) Quality measure for clusters: Create random permutations of the expression matrix and calculate R 2 for each Large R 2 implies a tight cluster of coherent genes within variance between variance R 2 = Compare R 2 of each cluster to that of the entire expression matrix Choose the cluster whose R 2 is furthest from the average R 2 of the permuted expression matrices. between variance of mean gene across experiments within variance of each gene about the cluster average Gene Shaving The final cluster contains a set of genes that are greatly affected by the experimental conditions in a similar way.

41. Relevance Networks Set of genes whose expression profiles are predictive of one another. Genes with low entropy (least variable across experiments) are excluded from analysis. H = -  p(x)log 2 (p(x)) x=1 10 Can be used to identify negative correlations between genes

42. Relevance Networks Correlation coefficients outside the boundaries defined by the minimum and maximum thresholds are eliminated. ADEBC.28.75.15.37.40.02.51.11.63.92 ADEBC T min = 0.50 The expression pattern of each gene compared to that of every other gene. The ability of each gene to predict the expression of each other gene is assigned a correlation coefficient T max = 0.90 The remaining relationships between genes define the subnets