Clinical Trial Designs for the Evaluation of Prognostic & Predictive Classifiers Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute

Validation = Fitness for Intended Use

Intended Uses l Prognostic biomarkers l Measured before treatment to indicate long-term outcome for patients untreated or not receiving chemotherapy l Used to determine who who doesn’t need more treatment l Predictive biomarkers l Measured before treatment to identify who will benefit from a particular treatment l Early detection biomarkers l Disease progression biomarkers

Prognostic Biomarkers in Oncology l Most gene expression signatures are developed as prognostic biomarkers. l Like numerous previously developed prognostic markers, most will never be used because they have not been demonstrated to be therapeutically relevant l Most prognostic marker studies are not conducted with an intended use clearly in mind l Most use a convenience sample of heterogeneous patients for whom tissue is available rather than patients selected for evaluating an intended use

Prognostic Markers in Oncology l There is rarely attention to analytical validation l There is rarely a separate validation study that addresses medical utility l Without a defined intended use, validation is meaningless and impossible

Prognostic Biomarkers Can Have Medical Utility Node Negative ER Positive Breast Cancer l Intended use is to identify patients who are likely to be cured by surgery/radiotherapy and hormonal therapy and therefore are unlikely to benefit from adjuvant chemotherapy l Oncotype Dx recurrence score l MammaPrint

Key steps in development and validation of biomarkers l Identify specific intended use of the biomarker l Perform developmental study using samples appropriate for the intended use l Use independent data to clinically validate the predictive accuracy of the pre-specified marker, classifier or score l Develop an analytically validated test l Perform a prospective study that addresses the medical utility of the biomarker or biomarker score/classifier

Types of Validation l Analytical validation l Accuracy in measurement of analyte l Robustness and reproducibility l Clinical validation l Correlation of score/classifier with clinical state or outcome l Medical utility l Actionable l Use results in patient benefit

Medical Utility l Benefits patient by improving treatment decisions l Depends on context of use of the biomarker l Treatment options and practice guidelines l Other prognostic factors

Clinical validity vs medical utility l A prognostic signature for patients with breast cancer may correlate with outcome, but does it identify a set of patients who have such good outcome without chemotherapy that they do not require treatment? l A prognostic signature for patients with early NSCLC may correlate with outcome, but does it identify a set of patients who have poor outcome untreated and benefit from chemotherapy?

Developmental vs Validation Studies l Developmental studies screen candidate markers to develop biomarker scores or classifiers l Train classifiers, optimize tuning parameters, set cut-off values for classification l Developmental studies often use cross-validation or split-sample validation to provide a preliminary estimate of the accuracy of the marker/classifier for predicting a clinical outcome l Developmental studies generally address clinical- validity (i.e. prediction accuracy), not medical utility

Developmental vs Validation Studies l Validation studies use a previously developed, completely specified classifiers/scores l Validation studies should use analytically validated tests and focus on medical utility, not predictive accuracy l This often requires a prospective clinical trial

Marker Strategy Design

SOC is Chemorx Marker Strategy Design

l Generally very inefficient because many patients in both randomization groups receive the same treatment l So inefficient as to be an insurmountable roadblock to validation of potentially valuable classifiers

Marker Strategy Design l Sometimes poorly informative l Not measuring marker in control group means that merits of complex marker treatment strategies cannot be dissected l Requires a marker/signature to be used for determining treatment decisions which may result in inferior outcome to the SOC

Marker Strategy Design l Data is not useful for evaluation of other markers or tests l Provides no information not provided by the test-all design

SOC is Chemorx Test-All Design

Using phase II data, develop predictor of response to new drug Develop Predictor of Response to New Drug Patient Predicted Responsive New Drug Control Patient Predicted Non-Responsive Off Study Targeted (Enrichment) Design

Evaluating the Efficiency of Targeted Design l Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10: , 2004; Correction and supplement 12:3229, 2006 l Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24: , 2005.

l Relative efficiency of targeted design depends on l proportion of patients test positive l effectiveness of new drug (compared to control) for test negative patients l When less than half of patients are test positive and the drug has little or no benefit for test negative patients, the targeted design requires dramatically fewer randomized patients than the standard design in which the marker is not used

Stratification Design for New Drug Development with Companion Diagnostic Develop Predictor of Response to New Rx Predicted Non- responsive to New Rx Predicted Responsive To New Rx Control New RXControl New RX

l Develop prospective analysis plan for evaluation of treatment effect and how it relates to biomarker l type I error should be protected for multiple comparisons l Trial sized for evaluating treatment effect overall and in subsets defined by test l Stratifying” (balancing) the randomization may be useful but is not a substitute for a prospective analysis plan.

Fallback Analysis Plan l Compare the new drug to the control overall for all patients ignoring the classifier. l If p overall ≤ 0.01 claim effectiveness for the eligible population as a whole l Otherwise perform a single subset analysis evaluating the new drug in the classifier + patients l If p subset ≤ 0.04 claim effectiveness for the classifier + patients.

In some cases a trial with optimal structure for evaluating a new biomarker will have been previously performed and will have pre-treatment tumor specimens archived l Under certain conditions, a focused analysis based on specimens from the previously conducted clinical trial can provide highly reliable evidence for the medical utility of a prognostic or predictive biomaker l In some cases, it may be the only way of obtaining high level evidence

Prospective-Retrospective Study

Guidelines Proposed by Simon, Paik, Hayes Prospective-retrospective design 1. Adequate archived tissue from an appropriately designed phase III clinical trial must be available on a sufficiently large number of patients that the appropriate biomarker analyses have adequate statistical power and that the patients included in the evaluation are clearly representative of the patients in the trial. 2. The test should be analytically and pre-analytically validated for use with archived tissue. Testing should be perform blinded to the clinical data. 3. The analysis plan for the biomarker evaluation should be completely specified in writing prior to the performance of the biomarker assays on archived tissue and should be focused on evaluation of a single completely defined classifier. 4. The results should be validated using specimens from a similar, but separate study involving archived tissues.