Download presentation

Presentation is loading. Please wait.

Published byCatherine Wildes Modified over 3 years ago

1
It is difficult to have the right single completely defined predictive biomarker identified and analytically validated by the time the pivotal trial of a new drug is ready to start accrual Changes in the way we do phase II trials Adaptive methods for the refinement and evaluation of predictive biomarkers in the pivotal trials in a non-exploratory manner Use of archived tissues in focused “prospective-retrospective” designs based on randomized pivotal trials

2
**Biomarker Adaptive Threshold Design**

Wenyu Jiang, Boris Freidlin & Richard Simon JNCI 99: , 2007

3
**Biomarker Adaptive Threshold Design**

Randomized trial of T vs C Have identified a univariate biomarker index B thought to be predictive of patients likely to benefit from T relative to C Eligibility not restricted by biomarker No threshold for biomarker determined Biomarker value scaled to range (0,1) Time-to-event data

4
**Procedure A Fallback Procedure**

Compare T vs C for all patients If results are significant at level .04 claim broad effectiveness of T Otherwise proceed as follows

5
**Procedure A Test T vs C restricted to patients with biomarker B > b**

Let S(b) be log likelihood ratio statistic Repeat for all values of b Let S* = max{S(b)} Compute null distribution of S* by permuting treatment labels If the data value of S* is significant at 0.01 level, then claim effectiveness of T for a patient subset Compute point and interval estimates of the threshold b

6
**Sample Size Planning (A)**

Standard broad eligibility trial is sized for 80% power to detect reduction in hazard D at significance level 5% Biomarker adaptive threshold design is sized for 80% power to detect same reduction in hazard D at significance level 4% for overall analysis

7
Model Hazard reduction for those who benefit Overall Power Adaptive Test Everyone benefits 33% .775 .751 50% benefit 60% .888 .932 25% .429 .604

8
**Biomarker Adaptive Threshold A**

Estimated Power of Broad Eligibility Design (n=386 events) vs Adaptive Design A (n=412 events) 80% power for 30% hazard reduction Model Broad Eligibility Design Biomarker Adaptive Threshold A 40% reduction in 50% of patients (22% overall reduction) .70 .78 60% reduction in 25% of patients (20% overall reduction) .65 .91 79% reduction in 10% of patients (14% overall reduction) .35 .93

9
**Estimation of Threshold**

10
506 prostate cancer patients were randomly allocated to one of four arms: Placebo and 0.2 mg of diethylstilbestrol (DES) were combined as control arm C 1.0 mg DES, or 5.0 mg DES were combined as E. The end-point was overall survival (death from any cause). Covariates: Age: In years Performance status (pf): Not bed-ridden at all vs other Tumor size (sz): Size of the primary tumor (cm2) Index of a combination of tumor stage and histologic grade (sg) Serum phosphatic acid phosphatase levels (ap)

11
**# patients with measured covariate**

Prostate Cancer Data Covariate # patients with measured covariate Overall Test p value Procedure A Stage 2 Procedure B AP 505 .084 .019 .041 SG 494 .110 .025 .050

12
**# patients with measured covariate**

Prostate Cancer Data Covariate # patients with measured covariate Estimated Threshold 95% CI 80% CI AP 505 36 (9,170) (25,108) SG 494 11 (10,13) (11,11)

15
Procedure B S(b)=log likelihood ratio statistic for treatment effect in subset of patients with Bb T=max{S(0)+R, max{S(b)}} Compute null distribution of T by permuting treatment labels If the data value of T is significant at 0.05 level, then reject null hypothesis that T is ineffective Compute point and interval estimates of the threshold b

16
**Sample Size Planning (B)**

Estimate power of procedure B relative to standard broad eligibility trial based on Table 1 for the row corresponding to the expected proportion of sensitive patients ( ) and the target hazard ratio for sensitive patients e.g. =25% and =.4 gives RE=.429/.641=.67 When B has power 80%, overall test has power 80*.67=53% Use formula B.2 to determine the approximate number of events needed for overall test to have power 53% for detecting =.4 limited to =25% of patients

17
**Example Sample Size Planning for Procedure B**

Design a trial to detect =0.4 (60% reduction) limited to =25% of patients Relative efficiency from Table /.641=.67 When procedure B has power 80%, standard test has power 80%*.67=53% Formula B.2 gives D’=230 events to have 53% power for overall test and thus approximate 80% power for B Overall test needs D=472 events for 80% power for detecting the diluted treatment effect

18
**Events needed to Detect Hazard Ratio With Proportional Hazards**

19
**Events (D’) Needed for Overall Test to Detect Hazard Ratio Limited to Fraction **

22
**Multiple Biomarker Design A Generalization of the Biomarker Adaptive Threshold Design**

Have identified K candidate binary classifiers B1 , …, BK thought to be predictive of patients likely to benefit from T relative to C RCT comparing new treatment T to control C Eligibility not restricted by candidate classifiers Let the B0 classifier classify all patients positive

23
**Test T vs C restricted to patients positive for Bk for k=0,1,…,K**

Let S(Bk) be a measure of treatment effect in patients positive for Bk Let S* = max{S(Bk)} , k* = argmax{S(Bk)} S* is the largest treatment effect observed k* is the marker that identifies the patients where the largest treatment effect is observed

24
**For a global test of significance**

Randomly permute the treatment labels and repeat the process of computing S* for the shuffled data Repeat this to generate the distribution of S* under the null hypothesis that there is no treatment effect for any subset of patients The statistical significance level is the area in the tail of the null distribution beyond the value of S* obtained for the un-suffled data If the data value of S* is significant at 0.05 level, then claim effectiveness of T for patients positive for marker k*

25
**Repeating the analysis for bootstrap samples of cases provides**

an estimate of the stability of k* (the indication)

26
Adaptive Signature Design An adaptive design for generating and prospectively testing a gene expression signature for sensitive patients Boris Freidlin and Richard Simon Clinical Cancer Research 11:7872-8, 2005

27
**Adaptive Signature Design End of Trial Analysis**

Compare E to C for all patients at significance level 0.04 If overall H0 is rejected, then claim effectiveness of E for eligible patients Otherwise

28
Otherwise: Using only the first half of patients accrued during the trial, develop a binary classifier that predicts the subset of patients most likely to benefit from the new treatment E compared to control C Compare E to C for patients accrued in second stage who are predicted responsive to E based on classifier Perform test at significance level 0.01 If H0 is rejected, claim effectiveness of E for subset defined by classifier

29
**Sensitive subset .01 level test 42.2**

Treatment effect restricted to subset. 10% of patients sensitive, 10 sensitivity genes, 10,000 genes, 400 patients. Test Power Overall .05 level test 46.7 Overall .04 level test 43.1 Sensitive subset .01 level test (performed only when overall .04 level test is negative) 42.2 Overall adaptive signature design 85.3

30
**Sensitive subset .01 level test 1.0**

Overall treatment effect, no subset effect. 10% of patients sensitive, 10 sensitivity genes, 10,000 genes, 400 patients. Test Power Overall .05 level test 74.2 Overall .04 level test 70.9 Sensitive subset .01 level test 1.0 Overall adaptive signature design

31
True Model

32
**Classifier Development**

Using data from stage 1 patients, fit all single gene logistic models (j=1,…,M) Select genes with interaction significant at level

33
**Classification of Stage 2 Patients**

For i’th stage 2 patient, selected gene j votes to classify patient as preferentially sensitive to T if

34
**Classification of Stage 2 Patients**

Classify i’th stage 2 patient as differentially sensitive to T relative to C if at least G selected genes vote for differential sensitivity of that patient

35
**Empirical Power Response Rate for Control Patients 25%**

Response Rate in Sensitive Subset Overall .05 Overall .04 Subset .01 Overall Adaptive 98% 49.5 45.4 75.8 85.7 95% 43.0 38.5 63.1 75.0 87% 36.7 31.7 34.5 51.6 80% 31.6 28.4 17.6 38.8 71% 26.0 22.6 6.3 26.3

36
**Chief, Biometric Research Branch, National Cancer Institute**

Adaptive Signature Design for Clinical Trial of Advanced Prostate Cancer Richard Simon, D.Sc. Chief, Biometric Research Branch, National Cancer Institute

37
Cancers of a primary site often represent a heterogeneous group of diverse molecular diseases which vary fundamentally with regard to the oncogenic mutations that cause them their responsiveness to specific drugs

38
How can we develop new drugs in a manner more consistent with modern tumor biology and obtain reliable information about what regimens work for what kinds of patients?

39
Developing a drug with a companion test increases complexity and cost of development but should improve chance of success and has substantial benefits for patients and for the economics of medical care

40
Although the randomized clinical trial remains of fundamental importance for predictive genomic medicine, some of the conventional wisdom of how to design and analyze rct’s requires re-examination The concept of doing an rct of thousands of patients to answer a single question about average treatment effect for a target population presumed homogeneous with regard to the direction of treatment efficacy in many cases no longer has an adequate scientific basis

41
**Predictive biomarkers**

Measured before treatment to identify who will benefit from a particular treatment

42
**Prospective Co-Development of Drugs and Companion Diagnostics in Ideal Settings**

Develop a completely specified classifier identifying the patients most likely to benefit from a new drug Based on biology, pre-clinical data and phase I-II studies Establish analytical validity of the classifier Design and analyze a focused clinical trial to evaluate effectiveness of the new treatment and how it relates to the classifier

43
Cancer biology is complex and it is not always possible to have the right single completely defined predictive classifier identified and analytically validated by the time the pivotal trial of a new drug is ready to start accrual Adaptive methods for the refinement and evaluation of predictive biomarkers in the pivotal trials in a non-exploratory manner Use of archived tissues in focused “prospective-retrospective” designs based on previously conducted randomized pivotal trials Simon, Paik, Hayes; JNCI 101:1-7, 2009

44
**Adaptive Signature Design**

Boris Freidlin and Richard Simon Clinical Cancer Research 11:7872-8, 2005

45
**Adaptive Signature Design End of Trial Analysis**

Compare X to C for all patients at significance level 0.01 If overall H0 is rejected, then claim effectiveness of X for eligible patients Otherwise Compare X to C in adaptively defined subset of patients using threshold of statistical significance 0.04

46
Divide the patients randomly into a training set T and a validation set V. The training set will contain one-third of the patients. Using the biomarker information, treatment and outcome for the patients in T, develop a binary classifier that identifies the subset of patients who appear most likely to benefit from the new treatment X compared to control C f(B1,B2,B3,B4) = log hazard ratio of death for X relative to C as a function of biomarker values If f(B1,B2,B3,B4)/ser <c then Classifier(B1,B2,B3,B4)=X If f(B1,B2,B3,B4)/ser >c then Classifier(B1,B2,B3,B4)=C Cutpoint c optimized

47
**Use the classifier developed in training set T to classify the patients in the validation set V.**

Let VX denote the subset of patients in V who are classified as likely to benefit from X Compare survivals of patients who received T to survivals of those who received C for patients accrued in VX If the difference in survival is significant at level 0.04, then the new treatment is more effective than the control for patients with biomarker values for which Classifier(B1,B2,B3,B4) =X.

48
This approach can also be used to identify the subset of patients who don’t benefit from X in cases where X is superior to C overall at the 0.01 level. The patients in VC= V – VX are predicted not to benefit from X. Survivals of X vs C can be examined for patients in that subset and a confidence interval for the hazard ratio calculated.

49
This design has improved statistical power for identifying treatments that benefit a subset of patients in molecularly heterogeneous diseases It has greater specificity than the standard approach which results in over-treatment of vast numbers of patients with approved drugs that do not benefit them

50
**Sample Size Planning for Advanced Prostate Cancer Trial**

Survival endpoint Final analysis when there are 700 deaths total 90% power for detecting a 25% overall reduction in hazard at two-sided 0.01 significance level (increase in median from 12 months to 9 months) Power for evaluating treatment in adaptively determined subset 157 deaths required for 80% power to detect 37% reduction in hazard at two-sided 0.04 significance level. If one-third of patients in the validation set are classifier positive, then to have 157 deaths in the subset we need 157*3=471 deaths in the validation set. Since the validation set is two-thirds of the total, we require 707 total deaths. To have 700 deaths at final analysis, 935 patients will be accrued and followed till the event rate is 75%

51
Sample Size Planning For this example, the sample size is strongly dependent on having high statistical power for detecting relatively modest treatment effect overall and in an adaptively defined subset consisting of only 33% of the patients. The number of required patients can be substantially reduced by Targeting larger treatment effects Targeting treatment benefits that apply to more than 33% of the patients Refining the simple interim analysis for futility described for this example

52
**Tumor specimen at entry as condition for eligibility**

Specimen preserved for later assay Assays will be performed prior to analysis using analytically validated tests Reproducible, robust and accurate for use with archived tissue No cut-point required Additional markers could be included prior to using specimens

53
**Interim Futility Analysis**

Interim futility analysis conducted when there are approximately 340 patients who have been followed for 6 months after randomization The analysis will use 6-month progression-free survival as intermediate endpoint. If difference between X group and C group is not significant at one-sided 0.20 level, then accrual will be terminated Power 90% for detecting 12 percentage point increase in proportion free of recurrence at 6 months from baseline of 40%

54
**Interim Futility Analysis**

Interim futility analysis does not utilize any of the 5% type I error of the study Using 6 month PFS as endpoint for interim futility analysis does not assume that PFS is a valid surrogate of survival; only that it is plausible to not expect a survival benefit if there is no PFS benefit Using PFS enables trial to be stopped earlier if there is no evidence of benefit for X The one-sided 0.20 significance level is used because the overall effect may be weak if the treatment benefits only a 33% subset of the patients.

55
**If the Markers Were Measured at Randomization**

Analytically validated tests would be required by the start of accrual The interim analysis could involve marker-defined subsets of patients Restricting accrual based on interim evaluation of marker specific treatment effects could substantially reduce sample size but would introduce additional issues not addressed in the current design

56
**Key Features Trial-wise type I error limited to 0.05**

Chance of any false positive conclusion of treatment benefit limited to 0.05 Randomized treatment assignment Regulatory endpoint Sample size sufficient for evaluating treatment effect in 33% subset Biomarkers measured using analytically validated tests Analysis algorithm pre-defined, and specific analysis plan defined prior to any assaying of tumors or data analysis

57
**This approach is as sound statistically as the conventional one treatment fits all design**

It provides strong evidence for evaluating the new treatment overall and within the classifier positive subset and for evaluating the classifier In settings where a single conventional “average effect” trials would be the basis for drug approval, this design should be the basis for approval either overall or for the identified subset.

58
This approach is more science based and consistent with tumor biology than the standard approach of treating thousands of patients with a heterogeneous disease to answer one question of whether the average treatment effect is zero and then treating everyone in a one treatment fits all manner.

59
**Cross-Validated Adaptive Signature Design**

Wenyu Jiang, Boris Freidlin, Richard Simon Clin Ca Res 16:691-8, 2010

60
**Cross-Validated Adaptive Signature Design End of Trial Analysis**

Compare T to C for all patients at significance level overall If overall H0 is rejected, then claim effectiveness of T for eligible patients Otherwise

61
**Otherwise Partition the full data set into K parts**

Form a training set by omitting one of the K parts. The omitted part is the test set Using the training set, develop a predictive classifier of the subset of patients who benefit preferentially from the new treatment T compared to control C using the methods developed for the ASD Classify the patients in the test set as either sensitive or not sensitive to T relative to C Repeat this procedure K times, leaving out a different part each time After this is completed, all patients in the full dataset are classified as sensitive or insensitive

62
**Perform test at significance level 0.05 - overall **

Compare T to C for sensitive patients by computing a test statistic S e.g. the difference in response proportions or log-rank statistic (for survival) Generate the null distribution of S by permuting the treatment labels and repeating the entire K-fold cross-validation procedure Perform test at significance level overall If H0 is rejected, claim effectiveness of E for subset defined by classifier The sensitive subset is determined by developing a classifier using the full dataset

63
80% Response to T in Sensitive Patients 25% Response to C otherwise 25% Response to C 10% Patients Sensitive ASD CV-ASD Overall 0.05 Test 0.223 0.240 Overall 0.04 Test 0.198 0.209 Sensitive Subset 0.01 Test 0.205 0.661 Overall Power 0.351 0.714

64
70% Response to T in Sensitive Patients 25% Response to T Otherwise 25% Response to C 20% Patients Sensitive ASD CV-ASD Overall 0.05 Test 0.486 0.503 Overall 0.04 Test 0.452 0.471 Sensitive Subset 0.01 Test 0.207 0.588 Overall Power 0.525 0.731

65
70% Response to T in Sensitive Patients 25% Response to T Otherwise 25% Response to C 30% Patients Sensitive ASD CV-ASD Overall 0.05 Test 0.830 0.838 Overall 0.04 Test 0.794 0.808 Sensitive Subset 0.01 Test 0.306 0.723 Overall Power 0.825 0.918

66
**35% Response to T 25% Response to C No Subset Effect**

ASD CV-ASD Overall 0.05 Test 0.586 0.594 Overall 0.04 Test 0.546 0.554 Sensitive Subset 0.01 Test 0.009 Overall Power

67
**25% Response to T 25% Response to C No Subset Effect**

ASD CV-ASD Overall 0.05 Test 0.047 0.056 Overall 0.04 Test 0.04 0.048 Sensitive Subset 0.01 Test 0.001 Overall Power 0.041

68
**Predictive Analysis of Clinical Trials**

Using cross-validation we can evaluate our methods for analysis of clinical trials, including complex subset analysis algorithms, in terms of their effect on improving patient outcome via informing therapeutic decision making R. Simon Clinical trials for predictive medicine, Clinical Trials 2010:1-9

69
Define an algorithm for predicting optimal treatment as function of covariate vector x using training dataset D For patients with covariate vector x, the algorithm predicts preferred treatment R(x | D) = T Or R(x | D) = C

70
At the conclusion of the trial randomly partition the patients into 10 equally sized sets P1 , … , P10 Let D-i denote the full dataset minus data for patients in Pi Using 10-fold complete cross-validation, omit patients in Pi Apply the defined algorithm to analyze the trial using only data in D-i For each patient j in Pi record the treatment recommendations based on fitting the algorithm to data D-I i.e. Rj=T or Rj=C.

71
**Repeat the above for all 10 loops of the cross-validation**

When all 10 loops are completed, all patients have been classified as what their optimal treatment is predicted to be

72
**Test of Significance for Effectiveness of T vs C Using the New Algorithm**

Let A denote the set of patients for whom treatment T is predicted optimal with the new algorithm i.e. Rj=T Compare outcomes for patients in A who actually received T to those in A who actually received C Let z = standardized log-rank statistic Compute statistical significance of z by randomly permuting treatment labels and repeating the entire procedure Do this 1000 or more times to generate the permutation null distribution of treatment effect for the patients predicted to be the best candidates for T

73
The significance test based on comparing T vs C for the adaptively defined subset is the basis for demonstrating that T is more effective than C for some patients. This test may be more powerful than the standard overall test in cases where the proportion of patients who benefit from T is limited. Although there is less certainty about which patients actually benefit, prediction accuracy may be substantially greater than for the standard single null hypothesis test based method have greater specificity for identifying the right patients

74
506 prostate cancer patients were randomly allocated to one of four arms: Placebo and 0.2 mg of diethylstilbestrol (DES) were combined as control arm C 1.0 mg DES, or 5.0 mg DES were combined as E. The end-point was overall survival (death from any cause). Covariates: Age: In years Performance status (pf): Not bed-ridden at all vs other Tumor size (sz): Size of the primary tumor (cm2) Index of a combination of tumor stage and histologic grade (sg) Serum phosphatic acid phosphatase levels (ap)

75
After removing records with missing observations in any of the covariates, 485 observations remained. A proportional hazards regression model was developed using patients in both E and C groups. Main effect of treatment, main effect of covariates and treatment by covariate interactions were considered. log[HR(z,x)]=a z + b’x + z c’x z = 0,1 treatment indicator (z=0 for control) x = vector of covariates log[HR(1,x)] – log[HR(0,x)] = a + c’x Define classifier C(X) = 1 if a + c’x < c = 0 otherwise c was fixed to be the median of the a + c’x values in the training set.

76
**Figure 1: Overall analysis. The value of the log-rank statistic is 2**

Figure 1: Overall analysis. The value of the log-rank statistic is 2.9 and the corresponding p-value is The new treatment thus shows no benefit overall at the 0.05 level.

77
Figure 2: Cross-validated survival curves for patients predicted to benefit from the new treatment. log-rank statistic = 10.0, permutation p-value is .002

78
Figure 3: Survival curves for cases predicted not to benefit from the new treatment. The value of the log-rank statistic is 0.54.

79
**Proportional Hazards Model Fitted to Full Dataset**

coef p-value Treatment age pf(Normal.Activity) sz sg ap Treatment*age Treatment*pf(Normal.Activity) Treatment*sz Treatment*sg Treatment*ap

80
By applying the analysis algorithm to the full RCT dataset D, recommendations are developed for how future patients should be treated; i.e. R(x|D) for all x vectors. The stability of the recommendations can be evaluated based on the distribution of R(x|D(b)) for non-parametric bootstrap samples D(b) from the full dataset D.

86
**Characteristics of Patients for Whom Classifications are Stable**

< 20% of classifiers favor X >80% of classifiers favor X Median IQR Age 76 4 68 12 Size 7 8 17 Stage 9 2 11 Acid Phos 0.6 0.4 1.45 9.42 Perf Status 144 good 31 poor 166 good 2 poor

87
**Standard Analysis Algorithm**

Test the overall H0 If you reject H0 then treat all future patients with T, otherwise treat all future patients with C

88
**Expected K-Year DFS Using Standard Analysis**

If the overall null hypothesis is not rejected Expected K-Year DFS is the observed K-year DFS in the control group If the overall null hypothesis is rejected Expected K-Year DFS is the observed K-year DFS in T group

89
**Expected K-Year DFS Using New Algorithm**

Let S(T) = observed K-year DFS for patients j for whom Rj=T and who received treatment T mT such patients Let S(C) = observed K-year DFS for patients j for whom Rj=C and who received treatment C mC such patients Expected K-Year DFS using new algorithm {mT S(T) + mC S(C)}/{mT + mC} Confidence limits for this estimate can be obtained by bootstrapping the complete cross-validation procedure

90
Hence, alternative methods for analyzing RCT’s can be evaluated in an unbiased manner with regard to their value to patients using the actual RCT data

Similar presentations

OK

Prospective Subset Analysis in Therapeutic Vaccine Studies Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute

Prospective Subset Analysis in Therapeutic Vaccine Studies Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute

© 2018 SlidePlayer.com Inc.

All rights reserved.

To ensure the functioning of the site, we use **cookies**. We share information about your activities on the site with our partners and Google partners: social networks and companies engaged in advertising and web analytics. For more information, see the Privacy Policy and Google Privacy & Terms.
Your consent to our cookies if you continue to use this website.

Ads by Google

Ppt on cardiac arrest Ppt on agriculture download Ppt on balanced diet for adolescent Ppt on law against child marriage in afghanistan Ppt on central limit theorem definition Ppt on solar power technology Make appt online Ppt on causes of land pollution Ppt on course management system Ppt on ready to serve beverages direct