1 Pulminiq™ Cyclosporine Inhalation Solution Pulmonary Drug Advisory Committee Meeting June 6, 2005 Statistical Evaluation Statistical Evaluation Jyoti Zalkikar, Ph.D. Office of Biostatistics Center for Drug Evaluation and Research Food and Drug Administration

2 ACS001- Study Conduct Small (56 patients in total), Phase II, Single Center (UPMC) Study Small (56 patients in total), Phase II, Single Center (UPMC) Study Not designed as a confirmatory study – planned sample size was 136 patients. Not designed as a confirmatory study – planned sample size was 136 patients. Investigators did not use Case Report Forms during the conduct of the study. Investigators did not use Case Report Forms during the conduct of the study. These were generated retrospectively by the applicant. These were generated retrospectively by the applicant. No prospective Statistical Data Analysis Plan or formal stopping rules. No prospective Statistical Data Analysis Plan or formal stopping rules.

3 ACS001- Study Conduct Study Began in Study Began in First 10 patients – received CyIS (Cyclosporine Inhalation solution) as part of the open label pilot phase First 10 patients – received CyIS (Cyclosporine Inhalation solution) as part of the open label pilot phase Next 58 patients were randomized to either CyIS with propylene glycol as vehicle or propylene glycol alone. Next 58 patients were randomized to either CyIS with propylene glycol as vehicle or propylene glycol alone. Two of these 58 patients did not receive any dose of the study medication and were excluded from all analyses. Two of these 58 patients did not receive any dose of the study medication and were excluded from all analyses. The enrollment was stopped in August 2001 at 68 subjects. The enrollment was stopped in August 2001 at 68 subjects. Study was “Vehicle” Controlled. Study was “Vehicle” Controlled. All subjects received systemic immuno-suppresion All subjects received systemic immuno-suppresion

4 ACS001- Study Design ACS001- Study Design Patients were to be enrolled from 7-42 days, after transplant. Patients were to be enrolled from 7-42 days, after transplant. 3 violations – 47 days, 53 days and 77 days 3 violations – 47 days, 53 days and 77 days Randomization was stratified by enrollment window (7-21 days and days) and CMV (D+/R-) mismatch status. Randomization was stratified by enrollment window (7-21 days and days) and CMV (D+/R-) mismatch status. 2 violations - incorrect strata 2 violations - incorrect strata Patients were to be on treatment daily for the first 10 days and 3 times a week thereafter for a total of 2 years. Patients were to be on treatment daily for the first 10 days and 3 times a week thereafter for a total of 2 years. 30 patients discontinued treatment early 30 patients discontinued treatment early All Patients were followed for at least 33 months. All Patients were followed for at least 33 months.

5 ACS001 - Study Design Evaluation of Survival and Chronic Rejection were NOT the primary objectives. Evaluation of Survival and Chronic Rejection were NOT the primary objectives. Pre-specified Primary Objective – Superiority over propylene glycol in terms of Rate of Acute Rejection (AR). Pre-specified Primary Objective – Superiority over propylene glycol in terms of Rate of Acute Rejection (AR). Study failed to achieve the primary objective. Study failed to achieve the primary objective. Mean number of AR higher in the CyIS arm (1.35) compared to the propylene glycol (1.20) arm. Mean number of AR higher in the CyIS arm (1.35) compared to the propylene glycol (1.20) arm.

6 Survival Distributions Survival Probability Month CyIS96%88%88%84%77% Placebo83%77%60%56%45% Log-rank p-value=0.0165

7 Challenge for the Review Team Given that the study had failed on the primary endpoint, determine if the observed survival difference attributable to Cyclosporine Inhalation Solution. Review found Several Major Weaknesses

8 Baseline Imbalances Though randomized, due to small sample size, the study failed to benefit from randomization – several baseline characteristics that clinicians consider to influence patient survival are not balanced between the two groups. Though randomized, due to small sample size, the study failed to benefit from randomization – several baseline characteristics that clinicians consider to influence patient survival are not balanced between the two groups. Therefore, groups are not comparable for evaluating survival or chronic rejection. Therefore, groups are not comparable for evaluating survival or chronic rejection.

9 Observed Important Baseline Imbalances CyIS (26) PG (30) Single transplant * 15 (58%) 24 (80%) Single transplant * 15 (58%) 24 (80%) [Double transplant][11 (42%) 6 (20%) ] [Double transplant][11 (42%) 6 (20%) ] Not Emphysema * 17 (65%) 11 (37%) Not Emphysema * 17 (65%) 11 (37%) Prior AR Grade 2+ 8 (31%) 13 (43%) Prior AR Grade 2+ 8 (31%) 13 (43%) Donor with inotropic 13 (50%) 25 (83%) support * Donor with inotropic 13 (50%) 25 (83%) support * Time in ICU >10 days * 1 (4%) 6 (20%) Time in ICU >10 days * 1 (4%) 6 (20%) * statistically significant at 10% level

10 Nature of Imbalance – All patients had at least one risk factor

Proportional Hazards Model Statistical Adjustments “one factor at a time” (Applicant’s briefing document and presentation) OR analyses with some (not all) of the factors included are NOT appropriate as the two groups are still not comparable due to imbalance with respect to the other factors. Statistical Adjustments “one factor at a time” (Applicant’s briefing document and presentation) OR analyses with some (not all) of the factors included are NOT appropriate as the two groups are still not comparable due to imbalance with respect to the other factors. The sample size of the study is too small for simultaneous adjustment for all the factors of importance that show imbalance. The sample size of the study is too small for simultaneous adjustment for all the factors of importance that show imbalance.

12 The questionable validity of any further inferential statistical analyses performed on the data from this study lends to caution while interpreting the results.

13 Blinding ? ABCD code as part of the patient number may have revealed the treatment assignment to the investigators. ABCD code as part of the patient number may have revealed the treatment assignment to the investigators. This together with the retrospective nature of the study makes it vulnerable to the introduction of bias, as inadvertent as it may be.

14 Detection of the presence and magnitude of bias is difficult, but its possibility lends to caution when interpreting the study results.

15 An Illustration of the effect of small perturbations in the assignment of patients

16 Survival Distributions Survival Probability Month CyIS96%88%88%84%77% PG83%77%60%56%45% Log-rank p-value=0.017

17 Small Sample Sensitivity Analysis Survival Probability Month CyIS86%79%79%75%69% PG93%85%67%62%50% Log-rank p-value=0.271

18 Due to Small Sample Size, the study results are highly sensitive to small perturbations in the assignment of patients.

19 Discontinuations - Reasons Reason for Discontinuation CyIS (N=26) PG (N=30) Withdrawal of Consent 6 0 AE Unsatisfied with their response (LOE) 1 2 Protocol Deviations 2 5 Completed Treatment 13 13

20 Number of Doses CyIS (n=26) PG (n=30) 1 to <10 doses 3 (12%) 3 (10%) 10 to < 25 doses 3 (12%) 2 (7%) 25 to < 42 doses - 1 (3%) 42 to < 78 doses 3 (12%) 2 (7%) 78 to <150 doses - 3 (10%) 150 to < 222 doses 3 (12%) 4 (13%) 4 (13%) 222 to < 294 doses 7 (27%) 6 (20%) 6 (20%) ≥ 294 doses 7 (27%) 9 (30%) 9 (30%)

21 Sensitivity analyses All 11 patients excluded All 11 patients excluded All 6 CyIS patients treated as PG patients All 6 CyIS patients treated as PG patients Define Treatment Failure as Death or Discontinuation within 25 doses due to AE or Withdrawal of Consent Define Treatment Failure as Death or Discontinuation within 25 doses due to AE or Withdrawal of Consent

22 Survival Distributions Patients with less than 25 doses excluded Log-rank p-value=0.202 Hazard Ratio =.51 95% CI = (.16, 1.4)

23 Sensitivity analyses All 11 patients excluded All 11 patients excluded All 6 CyIS patients treated as PG patients All 6 CyIS patients treated as PG patients Define Treatment Failure as Death or Discontinuation within 25 doses due to AE or Withdrawal of Consent Define Treatment Failure as Death or Discontinuation within 25 doses due to AE or Withdrawal of Consent

24 Survival Distributions Patients with less than 25 doses are treated as PG patients Log-rank p-value=0.194 Hazard Ratio =.52 95% CI = (.17, 1.34)

25 Sensitivity analyses All 11 patients excluded All 11 patients excluded All 6 CyIS patients treated as PG patients All 6 CyIS patients treated as PG patients Define Treatment Failure as Death or Discontinuation within 25 doses due to AE or Withdrawal of Consent Define Treatment Failure as Death or Discontinuation within 25 doses due to AE or Withdrawal of Consent

26 Distributions of Time to Treatment Failure* Survival Probability Month CyIS73%65%65%61%55% PG77%70%53%53%43% Log-rank p-value=0.556 * Treatment Failure = Death or Discontinuation Within 25 Doses due to Adverse Event or Withdrawal of Consent

27 The data from patients who discontinued early and are not expected to have any influence of the treatment they are assigned to, have a big impact on the study results and that lends to caution when interpreting the study results.

28 Lung Function – FEV1 Number in bracket is sample size MonthCyIS (26) PG (30) MonthCyIS (26) PG (30) Mean(FEV1_Pre-enroll) (15)1.371 (17) Mean(FEV1_3m) (26)1.787 (26) Mean(FEV1_6m) (25)1.912 (27) Mean(FEV1_12m) (24)1.867 (24) Mean(FEV1_18m) (22)1.789 (22) Mean(FEV1_24m) (20)1.643 (19) _______________________________________________ Information on Pre-enrollment FEV1 is grossly incomplete – 43% missing – Difference in pre-enrollment FEV1 between CyIS and Propylene Glycol is statistically significant.

29 Lung Function – FEV

30 Time to BOS* Log-rank p-value=0.214 No at Risk CyIS PG * Patients who died without diagnosis of BOS, as defined by the Applicant, were censored at the time of last follow-up for BOS.

31 Additional Data 10 patients in Study ACS001 in CyIS open label phase – Limited information 10 patients in Study ACS001 in CyIS open label phase – Limited information Baseline Characteristics CyIS- OLCyIS-R PG Baseline Characteristics CyIS- OLCyIS-R PG Donor Age > Patient Age 20% 15% 20% CMV D+/R- Mismatch 40% 19% 23% Gender Mismatch 50% 27% 33% Single Transplant 70% 58% 80% Double Transplant 30% 42% 20% Emphysema 70% 35% 63% Emphysema 70% 35% 63% Time in ICU > 10 days 0% 4% 23%

32 Survival Distributions

33 Summary Primary Endpoint of Acute Rejection - No Treatment Effect Primary Endpoint of Acute Rejection - No Treatment Effect Risk Factor Imbalance - Caution Risk Factor Imbalance - Caution Blinding Concerns - Caution Blinding Concerns - Caution Sensitivity to small perturbations in assignment - High Sensitivity to small perturbations in assignment - High Discontinuations - Caution Discontinuations - Caution Lung Function - FEV1 and BOS - No Treatment Effect Lung Function - FEV1 and BOS - No Treatment Effect Additional Data - Not supportive Additional Data - Not supportive

34 The Question Still Remains… Is the Observed Survival Difference, between the two arms in the randomized portion of the trial, due to Cyclosporine Inhalation Solution or due to other factors?