Biometric Research Branch Activities in DNA Microarray Investigations Division of Cancer Diagnosis and Treatment National Cancer Institute Richard Simon, D.Sc., Chief

DNA Microarray Technology Powerful tools for understanding mechanisms and enabling predictive medicine Challenges ability of biomedical scientists to analyze and interpret data Challenges mathematical scientists with new problems for which existing analysis paradigms are often inapplicable

Difficulties Resulting from Inadequate Training and Organization for 21 st Century Biomedicine Hype Excessive skepticism and inappropriate criticism Mis-information –Amature statisticians –Professional statisticians not sufficiently involved in collaborative studies Software that encourages invalid analysis –Developed by individuals who don’t know enough about statistical analysis of high dimensional data –Funded by decision makers who are floundering about how to organize to achieve the potential of bioinformatics and computational biology

Involvement of Biometric Research Branch Developmental Diagnostics Working Group Courses in genomics –Biotechnology techniques (18 hrs lecture/20 hrs lab) –Recombinant DNA methodology I (18/20) –Advanced genome analysis (18/20) –DNA binding proteins and transcriptional regulators (18/20) –Cellular & molecular approaches to the study of cancer (18/20) –Cold Spring Harbor Course in Computational Genomics Collaborations with Jeff Trent’s Laboratory of Cancer Genetics Establishment of Molecular Statistics & Bioinformatics Section

Collaborative research Methodological research Software development –MadB –BRB-ArrayTools Training –BRB post-docs –CCR scientists

BRB Collaboration with CCR on Microarray based Investigations Joanna Shih, Ph.D. –Available for collaboration with CCR scientists on the analysis of genomic data (including micro-array analysis). Has supervised 2 post- docs working jointly with BRB & CCR George Wright, Ph.D. –Collaborates intensively with Dr. Lou Staudt’s Laboratory and is the primary statistician in the Lymphoma Leukemia Molecular Profiling Project (LLMPP).

Other BRB Staff Collaboration in Microarray Studies Dr Paul Albert, serves as primary statistician for Urologic-oncology, Neuro-oncology, Pathology, Radiation oncology, Prevention, Metabolism branches Dr. Dodd, Korn, McShane, Zhao & post-docs –No obligations to intramural program, but collaborate on selected studies, e.g. –With A Hildesheim, Rose Yang, Sophia Wang on gene disregulation in Nasal pharyngeal carcinoma and the mechanism of HPV carcinogenesis in cervical cancer –With T Ried, M Difilippantonio on whole genome expression and CGH analysis of human rectal carcinoma

Collaboration with CCR on Microarray Studies Rich Simon –Sandy Swain (human breast cancer) –Xin Wang (human hepatocellular cancer) –Jeff Green (mouse models of breast cancer) –Ed Liu, Rick Klausner, Marston Linnehan, Lanny Kirsh –Numerous consultations, eg Curt Harris, Snorri Thorgeirsson, Steve Zeichner, Pat Steeg, Howard Fine,…

Classes & Workshops by BRB on Analysis of DNA Microarray Expression Profile Data Monthly 4 hr class on statistical analysis of expression profiling data Monthly 4 hr class on BRB-ArrayTools software Full day workshops in which BRB statisticians consult with investigators on design and analysis of genomic data

Use of BRB Website for Education of Biomedical Scientists Reprints & Technical Reports Powerpoint presentations –Audio files BRB-ArrayTools software –Message board BRB-ArrayTools Data Archive Sample Size Planning for Targeted Clinical Trials

DNA Microarray Methodology Simon R, Korn E, McShane L, Radmacher M, Wright G, Zhao Y. Design and analysis of DNA microarray investigations, Springer-Verlag, 2004 Simon R, Radmacher MD, Dobbin K. Design of studies with DNA microarrays. Genetic Epidemiology 23:21-36, 2002 McShane LM, Radmacher MD, Freidlin B, Yu R, Li MC, Simon R. Methods of assessing reproducibility of clustering patterns observed in analyses of microarray data. Bioinformatics 18: , 2002 Korn EL, McShane LM, Troendle JF, Rosenwald A and Simon R. Identifying pre-post chemotherapy differences in gene expression in breast tumors: a statistical method appropriate for this aim. British Journal of Cancer 86: , 2002 Radmacher MD, McShane LM and Simon R. A paradigm for class prediction using gene expression profiles. Journal of Computational Biology 9: , Simon R, Radmacher MD, Dobbin K, McShane LM. Pitfalls in the analysis of DNA microarray data. Journal of the National Cancer Institute 95:14-18, 2003.

Wright G, Simon R. A random variance model for detection of differential gene expression in small microarray experiments. Bioinformatics 19: , 2003 Simon R. Using DNA microarrays for diagnostic and prognostic prediction, Expert Review of Molecular Diagnostics 3(5) , 2003 Simon R. Diagnostic and prognostic prediction using gene expression profiles in high dimensional microarray data, British J Cancer 89: , 2003 Korn EL, Troendle JF, McShane LM, Simon R.Controlling the number of false discoveries. Journal of Statistical Planning and Inference 124:379-08, 2004 Molinaro A, Simon R, Pfeiffer R. Prediction error estimation: A comparison of resampling methods. Bioinformatics 21:3301-7,2005. Varma S, Simon R. Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics 7:91, 2006.

Dobbin K, Shih J, Simon R. Questions and answers on design of dual-label microarrays for identifying differentially expressed genes. Journal of the National Cancer Institute 95: , 2003 Dobbin K, Shih J, Simon R. Statistical design of reverse dye microarrays. Bioinformatics 19: , 2003 Dobbin K, Simon R. Comparison of microarray designs for class comparison and class discovery, Bioinformatics 18: , 2002; 19: , 2003; 21: , 2005; 21:2803-4, 2005 Dobbin K and Simon R. Sample size determination in microarray experiments for class comparison and prognostic classification. Biostatistics 6:27-38, 2005 Dobbin K, Beer DG, Meyerson M, et al. Inter-laboratory comparability study of cancer gene expression analysis using oligonucleotide microarrays. Clinical Cancer Research 11: , 2005 Dobbin KK, Kawasaki ES, Petersen DW, and Simon RM. Characterizing dye bias in microarray experiments. Bioinformatics 21: , 2005 Dobbin K and Simon R. Experimental design of DNA microarray studies. Encyclopedia of Genetics, Genomics, Proteomics and Bioinformatics. (L Jorde, P Little, M Dunn & S Subramaniam, eds). Wiley, 2005 Dobbin K and Simon R. Sample size planning for developing classifiers using high dimensional DNA microarray data. Biostatistics (In Press).

Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10: , Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24: , Simon R. When is a genomic classifier ready for prime time? Nature Clinical Practice – Oncology 1:4-5, Simon R. An agenda for Clinical Trials: clinical trials in the genomic era. Clinical Trials 1: , Simon R. Development and Validation of Therapeutically Relevant Multi-gene Biomarker Classifiers. Journal of the National Cancer Institute 97: , Simon R. A roadmap for developing and validating therapeutically relevant genomic classifiers. Journal of Clinical Oncology 23: ,2005. Freidlin B and Simon R. Adaptive signature design. Clinical Cancer Research 11: , Simon R. Validation of pharmacogenomic biomarker classifiers for treatment selection. Disease Markers (In Press). Simon R. Guidelines for the design of clinical studies for development and validation of therapeutically relevant biomarkers and biomarker classification systems. In Biomarkers in Breast Cancer, Hayes DF and Gasparini G, pp 3- 15, Humana Press, Using Genomic Classifiers In Clinical Trials

Simon R. and Wang SJ. Use of genomic signatures in therapeutics development in oncology and other diseases, The Pharmacogenomics Journal 6:166-73, Simon R. A checklist for evaluating reports of expression profiling for treatment selection. Clinical Advances in Hematology and Oncology 4: , Trepicchio WL, Essayan D, Hall ST, Schechter G, Tezak Z, Wang SJ, Weinreich D, Simon R. Designing prospective clinical pharmacogenomic trials- Effective use of genomic biomarkers for use in clinical decision- making. The Pharmacogenomics Journal 6:89-94,2006. Dupuy A and Simon R. Critical review of published microarray studies for cancer outcome and guidelines on statistical analysis and reporting, (Submitted for publication). Using Genomic Classifiers In Clinical Trials

BRB Website 15,000 hits per month 702 hits to Technical Reports & Talks 1985 hits to BRB-ArrayTools home page

BRB-ArrayTools Statistically state-of-the-art integrated software for DNA microarray data analysis –Architecture and statistical content by R Simon –Programming by contractor User interface for use and education of biomedical scientists Publicly available for non-commercial use Active user list-serve and message board

BRB-ArrayTools June Registered users Distinct institutions 62 Countries 245 Citations Registered users –3528 in US –456 at NIH –246 at NCI

NCI Support for Development of BRB-ArrayTools

Possible Reasons for the Success of BRB Array Tools It wasn’t designed by committee It wasn’t a response to user’s dictating what should be developed. Few of our 6000 users really know what they needed, although most probably think that they do It wasn’t developed in response to management decision of what was needed It offers substance, not pro-forma software engineering It offers correct approaches to data analysis in contrast to the vast majority of commercial and free software available