1 Efficacy Results NDA (MTP-PE) Laura Lu Statistical Reviewer Office of Biostatistics FDA/CDER

2 Main Issues for DFS 1.The Applicant’s pooled analysis not appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care. 2.Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset. 3.Conduct of interim analyses complicate the interpretation of DFS results.

3 Main Issues for OS 1.The primary endpoint (DFS) of the study was not met. 2.Patient follow-up for OS was inadequate to perform a meaningful analysis.

4 INT-0133 Efficacy Datasets IDM 2003 Dataset – used by COG for analyses published in JCO IDM 2006 Dataset FDA Dataset

5 FDA considered data captured on case report forms to be “primary source data”. Where discrepancies identified between CRFs and IDM 2003 dataset, CRF information used to determine dates of death and relapse. FDA did not modify IDM 2006 dataset because CRFs for additional follow-up not submitted. Therefore, FDA could not verify the accuracy of this information.

6 FDA Dataset Excludes Seven (7) ineligible patients contained in IDM Dataset 2003 Includes Nine (9) additional events identified in review of CRFs Additional follow-up documented on CRFs from one institution Modified based on FDA review of CRFs Change in length of disease-free survival for 66 pts Change in length of overall survival for 68 pts

7 Main Issues for DFS 1.The Applicant’s pooled analysis not appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care. 2.Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset. 3.Conduct of interim analyses complicate the interpretation of DFS results.

8 Study Design Randomized to four study arms One arm (Regimen AΘMTP-PE ) represents the control arm/standard of care Three experimental arms, each containing at least one experimental agent Evaluation of the efficacy of any experimental regimen needs to be considered relative to the control regimen.

9 Study Analysis Issues Study designers considered use of pooled analysis (A  /B  MTP-PE vs. A Θ /B Θ MTP-PE ) for the effect of MTP-PE. CCG/POG discussed the risk of this study design and analysis: “We hope that interactions between MTP-PE and the alternative chemotherapy arms will be similar. In this case it will be possible to analyze the proposed study by a factorial design. If the interactions are different, it will be necessary to consider the study as if it were a four arm analysis.”

10 Pooled Analysis Applicant's Method Compare Regimen A  MTP-PE vs. Regimen AΘMTP-PE Regimen B  MTP-PE vs. Regimen BΘMTP-PE separately Pool the results in the two comparisons (stratified log-rank test).

11 Treatment by Regimen Interaction Hazard ratio for A  MTP-PE vs. AΘMTP-PE = 0.99 Hazard ratio for B  MTP-PE vs. BΘMTP-PE = 0.62 The p-value for the treatment of MTP-PE by regimen interaction is

12 Kaplan-Meier Curves for DFS by Regimen Based on Primary Data

13 Kaplan-Meier Curves for DFS by Regimen Based on Primary Data

14 Regimen B vs. Regimen A Hazard ratio of Regimen BΘMTP- PE vs. Regimen AΘMTP-PE is Therefore, Regimen BΘMTP- PE performed worse than Regimen AΘMTP-PE.

15 Sensitivity Analysis Conducted to evaluate impact of inferior performance of Regimen B Θ MTP-PE on pooled outcome. In this analysis, results for Regimen A Θ MTP-PE (control arm) are substituted for the results in Regimen B Θ MTP-PE (inferior experimental chemo arm) in the pooled analysis.

16 Hazard Ratio P-value Results by substituting AΘMTP-PE for BΘMTP-PE on FDA Dataset Results of Sensitivity Analysis

17 Analysis Driven by Comparison to Inferior Regimen This sensitivity analysis shows that the small p-value of the pooled analysis is driven by a comparison to an experimental regimen that performed worse than the control regimen.

18 Invalid Analytic Method Based on Evidence of an treatment by regimen interaction AND Evidence of pooled analysis driven by a comparison where experimental regimen (BΘMTP-PE) is inferior to control regimen (AΘMTP-PE) A pooled analysis is not appropriate.

19 DFS Results by Regimen Relative to Regimen A Based on FDA Dataset VariableNumber of Patients Number of Events Hazard Ratio relative to A P-value Regimen AΘMTP-PE Regimen A  MTP-PE Regimen BΘMTP-PE Regimen B  MTP-PE

20 Main Issues for DFS 1.The Applicant’s pooled analysis not appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care. 2.Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset. 3.Conduct of interim analyses complicate the interpretation of DFS results.

21 DFS Results by Applicant’s Pooled Method Based on FDA Dataset Hazard ratio for A  /B  MTP-PE vs. A Θ /B Θ MTP-PE = 0.78, p-value = 0.065

22 Main Issues for DFS 1.The Applicant’s pooled analysis not appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care. 2.Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset. 3.Conduct of interim analyses complicate the interpretation of DFS results.

23 Type I Error Rate It is impossible to determine the actual Type I error rate due to the conduct/timing of the interim and final analyses.

24 Interim Analysis Conduct According to the final protocol amendment on 6/16/1997, one interim analysis performed with no detailed information for conduct or alpha spending. Two additional interim analyses on EFS conducted; timing not based on specific number of events.

25 Final Analysis Conduct A final analysis was not conducted according to the protocol (approx 167 DFS events) IDM provided results for an analysis performed after 228 events If the timing of the final analysis is influenced by the results of the interim analyses, the type I error rate will be impacted. It thus unclear what alpha should be used for the IDM analysis that included available data as of 4/9/2003.

26 Final Analysis Conduct If a pooled analysis is performed based on 167th DFS event using IDM 2003 dataset, the results are not statistically significant (nominal p-value = 0.11).

27 Main issues for OS 1.The primary endpoint of the study (DFS) was not met. 2.Follow-up on OS was inadequate to perform a meaningful analysis.

28 Survival Analysis Issues When the primary endpoint was not met, all alpha was spent. Any further analysis after the study failed to win on the primary endpoint increases the type I error rate. So literally, the difference in other endpoints should not be considered statistically significant.

29 Survival Analysis Issues Analysis plan for OS was not pre-specified. ‘Post-hoc’ analyses make it difficult to interpret the results for OS, since, by continuing to conduct tests for a treatment effect on different endpoints and/or the same endpoint a so-called “statistically significant result” (p < 0.05) can eventually be obtained even when there is no treatment effect. When an endpoint is selected based on the study results, the results for that endpoint are biased. Overall Survival analyses are exploratory

30 Main issues for OS 1.The primary endpoint of the study (DFS) was not met. 2.Follow-up on OS was inadequate to perform a meaningful analysis.

31 Adequacy of Follow-up for OS Analyses As of the 2003 data cut-off for OS –22% of patients (148/678) had died per IDM’s 2003 dataset –Among 530 remaining patients who were alive as of last contact, there were: 8% with last contact on/before 12/31/ % with last contact on/before 12/31/ % with last contact on/before 12/31/2000

32 Adequacy of Follow-up for OS Analyses More than 50% of the 530 patients alive at the last contact were lost to follow-up two years prior to data cutoff in 2003 In a well conducted trial for registration with OS as a primary endpoint, FDA expects substantially less than 5% of patients lost to follow-up at data cutoff.

33 Adequacy of Follow-up for Survival after Disease Occurrence Among those patients lost to follow-up for survival, –26 had active disease (recurrent osteosarcoma or AML) at the last follow- up. –These patients probably died.