1. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 1 Statistical Considerations for Topical Microbicide Phase 2 and 3 Trial Designs: A Regulatory Perspective Rafia Bhore, Ph.D. Greg Soon, Ph.D. Division of Antiviral Drug Products Food and Drug Administration

2. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 2 Outline Phase 2/3 Clinical Trial Design: An Example Two Arms or Three Arms ? p-value (Single Trial vs. Two Trials) Criteria for a “Win”/Success Power Considerations Sample Size Estimates Other Considerations

3. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 3 Phase 2/3 Clinical Trial Design An Example

4. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 4 To establish safety and efficacy of investigational microbicide in preventing HIV infection Test Group Control Group 1 Control Group 2 Microbicide “Placebo” + Condom Condom Condom-only (“no-treatment”)

5. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 5 Two Arms or Three Arms?

6. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 6 “Placebo” Arm “Placebo” provides a means to blinding investigators and participants Maximizes the likelihood of obtaining an unbiased estimate of efficacy Can we assume “Placebo” is inert? –Antimicrobial activity of “Placebo” not known/not proven. –Protective effect or harmful effect?

7. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 7 Condom-only (“No-Treatment”) Arm Established standard for prevention of HIV Provide comparison of “real world” effectiveness in preventing transmission –Data on sexual behaviors associated with use or non-use of microbicide products Single component of other arms containing gel + condom

8. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 8 P-value (Single vs. Two Trials) (Significance Level = Probability of Type 1 error: false positive)

9. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 9 Level of Evidence Number ofOne-sidedTwo-sidedLevel of Trials  / 2  Evidence 10.0250.05One Trial Single “large”0.025^1.50.008One-and-half Trial 20.025 each0.05 eachTwo Trials Single “LARGE”0.001Two Trials 20.025^20.00125Two Trials =0.000625 P-value <

10. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 10 P-value (Two trials) Two trials (Regulatory Standards) –Trial 1p-value < 0.05 (two-sided) –Trial 2p-value < 0.05 (two-sided) Run in parallel or staggered in time? –If staggered, how much gap in time?

11. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 11 P-value (One trial) Need to show as strong and robust evidence as two separate trials May not be repeatable –ethical concerns ? One trial: p-value < 0.001 (two-sided) (because 2x[0.025^2]=0.00125) same as p-value < 0.0005 (one-sided)

12. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 12 Replicating a Study Result Probability of observing a statistically significant result (e.g. p < 0.05) upon repetition of a clinical trial when the effect size observed in the first trial is assumed to be the true effect ObservedProbability of a significant p-value result (Power) in future 0.0550% 0.0173% 0.00191% Reference: Goodman (1992), Statistics in Medicine, 875-879

13. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 13 Overall Level of Evidence p-value < 0.001 considered convincing p-value >= 0.01 would be inadequate p-value between 0.001 and 0.01 possibly adequate, provided that –results are consistent across various subgroups –other supportive evidence is strong

14. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 14 Criteria for a “Win”/Success

15. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 15 Definition of a “Win” HIV infection rate in Microbicide < “Placebo” p-value < 0.001 (two-sided) AND Microbicide < Condom-only p-value < 0.001 (two-sided ) Overall  = 0.001

16. 16 Why win versus “Placebo” arm? If the HIV infection rate in –Microbicide + Condom  “Placebo”+Cond. –Microbicide + Condom < Condom –then is “Placebo” as effective as Microbicide? (does not prove efficacy of microbicide)

17. 17 Why win versus Condom-only? If the HIV infection rate in –Microbicide + Condom < “Placebo” + Cond. –Microbicide + Condom  Condom –then the use of microbicide in conjunction with condom does not provide any additional protection than condom alone

18. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 18 Definition of a “Win” HIV infection rate in Microbicide < “Placebo” p-value < 0.001 (two-sided) AND Microbicide < Condom-only p-value < 0.001 (two-sided )

19. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 19 Sample Size Estimates

20. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 20 Factors Affecting Sample Size HIV Seroincidence Rates in Control –0.5, 6, 7, 9 per 100 person-years Effect Size (33%, 50%, … ) Length of Follow-up –12 / 24 months for each participant –Last patient completes 12 / 24 months Statistical Power (90%, 80%, …)

21. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 21 Sample Size Estimates (Duration of study=24 months, Power=90%,  =.001) Control Rate Microbicide Rate Effect Size Sample Size (N) 6%4%33%12,520 7%4.67%33%10,797 9%6%33% 8,501 6%3%50% 4,993 7%3.5%50% 4,304 9%4.5%50% 3,385 Reference: Lachin,J. and M. Foulkes (Biometrics 1986)

22. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 22 Power Considerations (Power = 1-Probability of Type II error [false negative])

23. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 23 Example Test two sets of hypotheses H 0 : Microbicide+condom = “Placebo”+condom H A : Microbicide+condom < “Placebo”+condom H 0 : Microbicide+condom = Condom-only H A : Microbicide+condom < Condom-only Calculate sample-size –Power for each test is 90% at 33% reduction in HIV infection rate from condom-only arm

24. 24 Power Curve: Varying Rates of Risk Reduction from Placebo

25. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 25 Other Considerations

26. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 26 Follow-up Continue study until last subject enrolled completes at least 12 months on study Pro-active in following participants –actively pursue and identify reasons for dropouts (safety issues?) –continue follow-up after study drug discontinuation

27. 27 Condom + Microbicide Use Monitoring Collect data on use of condom and other barrier/drug use Efficacy evidence closely tied with compliance “Frequently” collect information on number of sexual acts

28. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 28 Allocation Ratio Microbicide : Placebo : Condom-only –Standard practice is to randomize 1:1:1 –Alternatively randomize x: y: z e.g., 3:2:2 or 1.5:1:1 –More participants in microbicide arm than control groups. Same number in both control groups –Alternative allocation ratios will optimize overall power to detect a difference and increase safety data on microbicide arm.

29. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 29 Summary Three-arm design will ensure that the first generation of microbicides is appropriately studied Single trial to show same level of evidence as two separate trials Sample size depends on assumptions Length of follow-up important in observing HIV endpoints.

30. August 20, 2003FDA Antiviral Drugs Advisory Committee Meeting 30 Acknowledgments Dr. Teresa Wu, Medical Officer Dr. Debra Birnkrant, Director Division of Antiviral Drug Products, FDA