1. Preprocessing of cDNA microarray data Lecture 19, Statistics 246, April 1, 2004

2. Was the experiment a success? What analysis tools should be used? Are there any specific problems? Begin by looking at the data

3. Red/Green overlay images Good: low bg, lots of d.e.Bad: high bg, ghost spots, little d.e. Co-registration and overlay offers a quick visualization, revealing information on color balance, uniformity of hybridization, spot uniformity, background, and artifiacts such as dust or scratches

4. Always log, always rotate log 2 R vs log 2 GM=log 2 R/G vs A=log 2 √RG

5. Signal/Noise = log 2 (spot intensity/background intensity) Histograms

6. Boxplots of log 2 R/G Liver samples from 16 mice: 8 WT, 8 ApoAI KO.

7. Spatial plots: background from the two slides

8. Highlighting extreme log ratios Top (black) and bottom (green) 5% of log ratios

9. Boxplots and highlighting Clear example of spatial bias (here high is red, low green) Print-tip groups Log-ratios pin group #

10. Pin group (sub-array) effects Boxplots of log ratios by pin groupLowess lines through points from pin groups

11. Plate effects

12. KO #8 Probes: ~6,000 cDNAs, including 200 related to lipid metabolism. Arranged in a 4x4 array of 19x21 sub-arrays.

13. Time of printing effects Green channel intensities (log 2 G). Printing over 4.5 days. The previous slide depicts a slide from this print run. spot number

14. Normalization Why? To correct for systematic differences between samples on the same slide, or between slides, which do not represent true biological variation between samples. How do we know it is necessary? By examining self-self hybridizations, where no true differential expression is occurring. We find dye biases which vary with overall spot intensity, location on the array, plate origin, pins, scanning parameters,….

15. Self-self hybridizations False color overlayBoxplots within pin-groupsScatter (MA-)plots

16. From the NCI60 data set (Stanford web site) A series of non self-self hybridizations

17. Early Ngai lab, UC Berkeley

18. Early Goodman lab, UC Berkeley

19. From the Ernest Gallo Clinic & Research Center

20. Early PMCRI, Melbourne Australia

21. Normalization: methods a) Normalization based on a global adjustment log 2 R/G -> log 2 R/G - c = log 2 R/(kG) Choices for k or c = log 2 k are c = median or mean of log ratios for a particular gene set (e.g. housekeeping genes). Or, total intensity normalization, where k = ∑R i / ∑G i. b) Intensity-dependent normalization. Here we run a line through the middle of the MA plot, shifting the M value of the pair (A,M) by c=c(A), i.e. log 2 R/G -> log 2 R/G - c (A) = log 2 R/(k(A)G). One estimate of c(A) is made using the LOWESS function of Cleveland (1979): LOcally WEighted Scatterplot Smoothing.

22. Normalization: methods c) Within print-tip group normalization. In addition to intensity-dependent variation in log ratios, spatial bias can also be a significant source of systematic error. Most normalization methods do not correct for spatial effects produced by hybridization artifacts or print-tip or plate effects during the construction of the microarrays. It is possible to correct for both print-tip and intensity-dependent bias by performing LOWESS fits to the data within print-tip groups, i.e. log 2 R/G -> log 2 R/G - c i (A) = log 2 R/(k i (A)G), where c i (A) is the LOWESS fit to the MA-plot for the ith grid only.

23. Which spots to use for normalization? The LOWESS lines can be run through many different sets of points, and each strategy has its own implicit set of assumptions justifying its applicability. For example, we can justify the use of a global LOWESS approach by supposing that, when stratified by mRNA abundance, a) only a minority of genes are expected to be differentially expressed, or b) any differential expression is as likely to be up-regulation as down- regulation. Pin-group LOWESS requires stronger assumptions: that one of the above applies within each pin-group. The use of other sets of genes, e.g. control or housekeeping genes, involve similar assumptions.

24. Use of control spots M = log R/G = logR - logGA = ( logR + logG) /2 Positive controls (spotted in varying concentrations) Negative controls blanks Lowess curve

25. Global scale, global lowess, pin-group lowess; spatial plot after, smooth histograms of M after

26. MSP titration series ( Microarray Sample Pool) Control set to aid intensity- dependent normalization Different concentrations Spotted evenly spread across the slide Pool the whole library

27. Yellow: GAPDH, tubulin Light blue: MSP pool / titration Orange: Schadt-Wong rank invariant set Red line: lowess smooth MSP normalization compared to other methods

28. Composite normalization Before and after composite normalization -MSP lowess curve -Global lowess curve -Composite lowess curve (Other colours control spots) c i (A)=  A g(A)+(1-  A )f i (A)

29. Comparison of Normalization Schemes (courtesy of Jason Goncalves) No consensus on best segmentation or normalization method Scheme was applied to assess the common normalization methods Based on reciprocal labeling experiment data for a series of 140 replicate experiments on two different arrays each with 19,200 spots

30. DESIGN OF RECIPROCAL LABELING EXPERIMENT Replicate experiment in which we assess the same mRNA pools but invert the fluors used. The replicates are independent experiments and are scanned, quantified and normalized as usual

31. The following relationship would be observed for reciprocal microarray experiments in which the slides are free of defects and the normalization scheme performed ideally We can measure using real data sets how well each microarray normalization scheme approaches this ideal

32. Deviation metric to assess normalization schemes We now use the mean array average deviation to compare the normalization methods. Note that this comparison addresses only variance (precision) and not bias (accuracy) aspects of normalization.

33. ***

34. Scale normalization: between slides Boxplots of log ratios from 3 replicate self-self hybridizations. Left panel: before normalization Middle panel: after within print-tip group normalization Right panel: after a further between-slide scale normalization.

35. The “NCI 60” experiments (no bg) Some scale normalization seems desirable

36. Scale normalization: another data set Log-ratios Only small differences in spread apparent. No action required. `

37. Assumption: All slides have the same spread in M True log ratio is  ij where i represents different slides and j represents different spots. Observed is M ij, where M ij = a i  ij Robust estimate of a i is MAD i = median j { |y ij - median(y ij ) | } One way of taking scale into account

38. A slightly harder normalization problem Global lowess doesn’t do the trick here.

39. Print-tip-group normalization helps

40. But not completely There is still a lot of scatter in the middle in a WT vs KO comparison.

41. Effects of previous normalisation Before normalisationAfter print-tip-group normalization

42. Within print-tip-group box plots of M after print-tip-group normalization

43. Assumption: All print-tip-groups have the same spread in M True log ratio is  ij where i represents different print-tip-groups and j represents different spots. Observed is M ij, where M ij = a i  ij Robust estimate of a i is MAD i = median j { |y ij - median(y ij ) | } Taking scale into account, cont.

44. Effect of location & scale normalization Clearly care is needed in making decisions like this one.

45. A comparison of three MA-plots Unnormalized Print-tip normalizationPrint tip & scale n.

46. The same idea on another data set After print-tip location and scale normalization. Log-ratios Print-tip groups

47. Follow-up experiment On each slide, half the spots (  8) are differentially expressed, the other half are not.

48. Paired-slides: dye-swap Slide 1, M = log 2 (R/G) - c Slide 2, M’ = log 2 (R’/G’) - c’ Combine by subtracting the normalized log-ratios: [ (log 2 (R/G) - c) - (log 2 (R’/G’) - c’) ] / 2  [ log 2 (R/G) + log 2 (G’/R’) ] / 2  [ log 2 (RG’/GR’) ] / 2 provided c = c’. Assumption: the normalization functions are the same for the two slides.

49. Checking the assumption MA plot for slides 1 and 2: it isn’t always like this.

50. Result of self-normalization (M - M’)/2 vs. (A + A’)/2

51. Summary of normalization —Reduces systematic (not random) effects —Makes it possible to compare several arrays —Use logratios (MA-plots) —Lowess normalization (dye bias) —MSP titration series – composite normalization —Pin-group location normalization —Pin-group scale normalization —Between slide scale normalization —More? Use controls! —Normalization introduces more variability —Outliers (bad spots) are handled with replication

52. What is missing? Principally, a discussion of data quality issues. Most image analysis programs collect a wide range of measurements associated with each spot: morphological measures such as area and perimeter (in pixels), uniformity measures such as the SD of foreground and background intensities in each channel, and of ratios of intensities (with and without background) across the pixels in a spot; and spot brightness indicators such as the ratio of spot foreground to spot background, and the fraction of pixels in the foreground with intensity greater than background intensity (or a given multiple thereof). From these, further derived measures can be calculated, such as coefficients of variation, and so on. How should we make use of the various quality indicators? Most programs include procedures for flagging spots on the basis of one or more indicators, and users typically omit flagged spots from their primary analyses. “Data filtering” of this kind clearly improves the appearance of the data, but….can we do more? That is a longer story, for another time.

53. Acknowledgments Jean Yee Hwa Yang (UCB) Sandrine Dudoit (UCB) Natalie Thorne (WEHI) Ingrid Lönnstedt (Uppsala) Henrik Bengtsson (Lund) Jason Goncalves (Iobion) Matt Callow (LLNL) Percy Luu (UCB) John Ngai (UCB) Vivian Peng (UCB) Dave Lin (Cornell)

54. Reference: Yang et al (2002) Nucleic Acids Research 30, e15. Some web sites: Technical reports, talks, software etc. http://www.stat.berkeley.edu/users/terry/zarray/Html/ Statistical software R (“GNU’s S”) http://www.R-project.org/ Packages within R environment: -- SMA (statistics for microarray analysis) http://www.stat.berkeley.edu/users/terry/zarray/Software/smacode.html http://www.stat.berkeley.edu/users/terry/zarray/Software/smacode.html --Spot http://www.cmis.csiro.au/iap/spot.htm