1. Opportunity and Pitfalls in Cancer Prediction, Prognosis and Prevention Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov

2. Kinds of Biomarkers Prognostic biomarkers – Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment – May reflect both disease aggressiveness and effect of standard treatment – Used to determine who needs more intensive treatment Predictive biomarkers – Measured before treatment to identify who will benefit from a particular treatment Early detection biomarkers Endpoint biomarkers Measured before, during and after treatment to monitor pace of disease and treatment effect

3. Most cancer treatments benefit only a minority of patients to whom they are administered Being able to predict who requires intensive treatment and who is likely to benefit from which treatments could – save patients from unnecessary debilitating adverse effects of treatments that they don’t need or benefit from – enhance their chance of receiving a treatment that helps them – Help control medical costs – Improve the success rate of clinical drug development

4. Pusztai et al. The Oncologist 8:252-8, 2003 939 articles on “prognostic markers” or “prognostic factors” in breast cancer in past 20 years ASCO guidelines only recommend routine testing for ER, PR and HER-2 in breast cancer “With the exception of ER or progesterone receptor expression and HER-2 gene amplification, there are no clinically useful molecular predictors of response to any form of anticancer therapy.”

10. Inter-disciplinary collaboration

11. Separate silos for the generation and mining of data – Generation of state-of-the-art genomic data – Application of state-of-the-art analysis tools – Data generators know little about the tools – Analysts know little about the application field and have to use the data at hand – Generate predictive classifiers first, find use later – Find answers to questions that no-one even asked – Hype meets hype

12. Trans-disciplinary collaboration Partners working together to try to develop solutions to important specific medical problems – Involve the right people to define the key problems and the key opportunities – Define the objective first – What data do we need to develop a solution – What partners do we need to accomplish the goal – The goal is impact on patients, not publication of papers

13. Prognostic markers There is an enormous published literature on prognostic markers in cancer. Very few prognostic markers (factors) are recommended for measurement by ASCO, are approved by FDA or are reimbursed for by payers. Very few play a role in treatment decisions.

14. Prognostic Factors in Oncology Most prognostic factors are not used because they are not therapeutically relevant Most prognostic factor studies are not conducted with an intended use clearly in mind – They use a convenience sample of patients for whom tissue is available. – Generally the patients are too heterogeneous to support therapeutically relevant conclusions There is rarely a validation study that addresses medical utility – Clinical validity = marker is predictive of outcome – Medical utility = marker is actionable, enables treatment decisions that improve patient outcome

15. Most prognostic factor studies use a heterogeneous convenience sample of patients based on tissue availability. Often the patients have been previously treated with various types of chemotherapy and so it is not possible to focus the analysis on patients who would have such good prognosis with only local surgery/xrt that chemotherapy is not needed.

16. Prognostic Biomarkers in Node Negative Breast Cancer Intended use is to identify patients who are likely to be cured by surgery/radiotherapy and hormonal therapy and therefore are unlikely to benefit from adjuvant chemotherapy – Oncotype Dx recurrence score based on expression of 21 genes measured by RT-PCR on FFPE diagnostic biopsy

17. B-14 Results—Relapse-Free Survival 338 pts 149 pts 181 pts p<0.0001 Paik et al, SABCS 2003

18. Key Features of OncotypeDx Development Focus on important therapeutic decision context – Stage I breast cancer patients with ER positive tumors receiving tamoxifen as only systemic therapy Staged development and validation – Separation of data used for test development from data used for test validation – Patients in a large clinical trial with little missing data Development of robust analytically validated assay

19. Predictive Biomarkers – Predictive markers identify patients who are likely (or unlikely) to benefit from specific drugs. – Particularly important for molecularly targeted drugs – Often single gene/protein markers HER2 for anti-Her2 rx in breast cancer KRAS for anti-EGFR antibodies in colorectal cancer

20. To discover a marker or signature predictive of survival or disease-free survival benefit of treatment T relative to control C, you should ideally examine specimens from patients in a randomized clinical trial comparing T to C

25. Because of the long time between first mutation and clinical diagnosis of human solid tumors, there would seem to be great opportunity for early detection Effective detection must have long lead time and high specificity for tumors which will evolve to be life threatening

27. Most studies of early detection markers are based on a convenience sample of diagnostic tumor tissue from cancer patients and non cancer tissue from controls. These studies rarely identify markers with sufficient sensitivity, specificity and lead time characteristics for use in population screening settings.

28. Key steps in development of biomarkers Identify specific intended use of the biomarker Perform developmental study using samples appropriate for the intended use Develop an analytically validated assay Perform a focused validation study that addresses the medical utility of a specific biomarker or biomarker score/classifier

29. Key roadblocks to progress in therapeutics Lack of knowledge of the key molecular targets that drive tumor invasion and are present in all sub-clones Inability to effectively “drug” key targets like P53, Rb, Ras Where we have identified druggable key targets, clinical development is effective (e.g. bcr-abl, her2).

30. Provocative questions Find the “founder” mutations in individual tumors Find synthetic lethality targets for frequently mutated tumor supressor and other “un- druggable” genes

31. Find the founder mutations Passenger mutations Occur at rate of synonomous mutations No functional role in oncogenesis and pathogenesis Driver mutations Founder (early) mutations Progression mutations

32. The earliest driver mutations may be of special importance They exist in all sub-clones They permit the tumor to grow to a size in which subsequent mutations occur in a non-rate-limiting manner The subsequent mutations are selected for in the context of the the founder mutations – Some tumors may be “addicted” to the early mutations They may represent key molecular targets for treatment Discovering the founder mutations may shed light on the early stages of oncogenesis, intra-tumor heterogeneity and the time course of tumor development, invasion and dissemination

35. “With the appropriate informatic analyses and experimental design, the depth and breadth of sequencing available on the next-generation platforms will provide the tools to reconstruct clonal interrelationships of cancer cell populations, with relevance to identifying and tracking subpopulations of cells responsible for drug resistance, invasion, metastasis, and relapse, as well as annotating the genuinely initiating genetic lesions.”

38. Biotechnology Has Forced Biostatistics to Focus on Prediction This has led to many exciting methodological developments – P>n problems in which number of genes is much greater than the number of cases And many erroneous publications

39. Goodness of Fit vs Prediction Accuracy Fit of a model to the same data used to develop it is no evidence of prediction accuracy for independent data “Prediction is difficult; particularly the future.” – Dan Quale or Neils Bohr?

41. Prediction on Simulated Null Data Simon et al. J Nat Cancer Inst 95:14, 2003 Generation of Gene Expression Profiles 20 specimens (P i is the expression profile for specimen i) Log-ratio measurements on 6000 genes P i ~ MVN(0, I 6000 ) Can we distinguish between the first 10 specimens (Class 1) and the last 10 (Class 2)? Prediction Method Compound covariate predictor built from the log-ratios of the 10 most differentially expressed genes.

43. Cross validation is only valid if the test set is not used in any way in the development of the model. Using the complete set of samples to select genes violates this assumption and invalidates cross-validation. With proper cross-validation, the model must be developed from scratch for each leave-one- out training set. This means that feature selection must be repeated for each leave- one-out training set.

45. In order to bridge the valley of death new approaches to cancer research are needed New tools of biotechnology or data mining are not sufficient for progress

46. Progress requires Clear objectives for specific medical contexts Active, long-term trans-disciplinary collaboration Objectives should dictate the needed data, analysis tools, and academic/industrial partners Patient-oriented success metrics, not publications