Presentation is loading. Please wait.

Presentation is loading. Please wait.

Statistical Challenges and Adaptive Design Strategies in Evaluation of New Vaccines Ivan S. F. Chan, Ph.D. Late Development Statistics Merck Research Laboratories.

Similar presentations

Presentation on theme: "Statistical Challenges and Adaptive Design Strategies in Evaluation of New Vaccines Ivan S. F. Chan, Ph.D. Late Development Statistics Merck Research Laboratories."— Presentation transcript:

1 Statistical Challenges and Adaptive Design Strategies in Evaluation of New Vaccines Ivan S. F. Chan, Ph.D. Late Development Statistics Merck Research Laboratories IMS Workshop on Design and Analysis of Clinical Trials National University of Singapore, Singapore October 24-28, 2011

2 2 Collaborators Jonathan Hartzel Xiaoming Li Josh Chen Yanli Zhao Jenny Sun

3 3 Outline Introduction to Vaccines Clinical Development of Vaccines –Herpes Zoster Vaccine Example Opportunities for Adaptive Design Strategy Issues and Challenges in Adaptive Designs –Illustrated with Examples

4 4 The Ten Greatest Public Health Achievements of the 20 th Century Vaccination Motor-vehicle safety Safer workplaces Control of infectious diseases Decline in deaths from coronary heart disease and stroke Safer and healthier foods Healthier mothers and babies Family planning Fluoridation of drinking water Recognition of tobacco use as a health hazard MMWR (1999);48:1141

5 5 What Are Vaccines? Biological products Typically for prophylaxis, not treatment Use antigen or attenuated live virus to trigger immune responses for disease protection Administered as a single dose or series with a potential booster dose Highly complex immunologic milieu –Array of humoral and cellular immune responses

6 6 Examples of Vaccines Pediatric vaccines –Polio –Measles, mumps, rubella (MMR) –Chickenpox (Varivax  ) –Hepatitis B –Diphtheria, tetanus, pertussis –Rotavirus (infant gastroenteritis, RotaTeq  ) –Invasive pneumococcal disease (Prevnar  ) Adolescents and Adult vaccines –HPV (cervical cancer, Gardasil  ) –Meningitis (Menactra  ) –Influenza –Invasive pneumococcal disease (Pneumovax 23  ) –Herpes zoster (shingles, Zostavax  )

7 7 Benefits of Vaccines Direct benefit –Efficacy in clinical trials –Risk benefit at individual level Indirect benefit –Herd immunity by reducing exposure and transmission –Public health implications

8 8 Types of Immunity Humoral (antibody-mediated) immunity –B lymphocytes, –Plasma cells –Immunoglobulins (Ig) IgG, IgM, IgA, IgD and IgE Cell-mediated immunity (CMI) –T lymphocytes –Cytokine/Interleukins

9 9 Functions of Immunoglobulins Serve as antibodies Neutralize viruses and bacterial toxins –IgG accounts for ~80% of total immunoglobulin pool Bind antigen Prevent or clear first infection

10 10 Functions of T-cells (CMI) T lymphocytes (helper cells) stimulate B cells to produce antibodies T suppressor (regulatory) cells play an inhibitory role and control the level and quality of the immune response (CD4) Cytotoxic T-cells recognize and destroy infected cells (CD8)

11 Brief Overview of Clinical Development of Vaccines

12 12 Source: Based on PhRMA analysis, updated for data per Tufts Center for the Study of Drug Development (CSDD) database. Discovery and Development of a Successful Drug/Vaccine 3,000 - 10,000 10 - 20 5 - 10 2 2 1 1 POST-MARKETING SURVEILLANCE PRECLINICAL TEST (ANIMALS) DEVELOPMENT 2 - 5 QUANTITY OF SUBSTANCES YEARS I I SYNTHESIS, EXAMINATION & SCREENING 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 11 12 13 14 15 MARKET LAUNCH RESEARCH CLINICAL TEST (HUMANS) PHASES IV III II BASIC

13 13 Evaluation of New Vaccines - Safety Assess local (injection-site) and systemic adverse experiences Need a large database, particularly because of giving vaccines to healthy subjects Choice of safety parameters depend on type of disease, population, and route of administration Need large-scale post licensure study for additional safety monitoring

14 14 Evaluation of New Vaccines - Efficacy Measure the relative reduction (RR) of disease incidence after vaccination compared with placebos VE = 1 – RR = 1 – P V /P C Require a high level of evidence and precision –Success typically requires showing efficacy greater than a non-zero (e.g. 20% - 50%) lower bound –May need a very large study Need long-term data to assess duration of efficacy –Historical controls may be used if concurrent controls are not available –When is a booster dose needed?

15 15 Impact of VE Lower Bound Requirement on Sample Size Rapid increase of sample size when VE lower bound increases Example assumes –5/1000 incidence –90% power –60% true VE –One-sided 2.5% test –1:1 randomization VE Lower Bound Total Sample Size 016,300.1020,800.2028,500.3043,900

16 16 Evaluation of New Vaccines - Immunogenicity Important in understanding the biology Humoral immunity –Antibody responses –priming, first defense Cell-mediated immunity –T-cell responses – prevent virus reactivation, kill infected cells Identify immune markers that correlates with disease protection

17 17 Variability/Stability of Vaccines Vaccines are biological products that have more variability in than chemical compound –Need to demonstrate consistency of manufacturing Many vaccines contains attenuated live viruses and will lose potency over time –E.g., chickenpox vaccine, zoster vaccine Need to establish a range of potency for manufacturing and product shelf-life –Study the safety at the high potency –Establish efficacy at near-expiry potencies

18 An Example: Clinical Development of ZOSTAVAX ® A live-virus vaccine to prevent herpes zoster (shingles)

19 19 Herpes Zoster Is a Consequence of Varicella- Zoster Virus (VZV) Reactivation Copyright ©2005 by Merck & Co., Inc., All rights reserved. Spinal cord Dorsal root ganglion Site of VZV replication Image courtesy of Courtesy of JW Gnann.

20 20 Ophthalmic Zoster Ophthalmic Zoster Courtesy of MN Oxman UCSD/San Diego VAMC.

21 21 Phase I Study for Dose Ranging Assess the immune responses of 8 dose levels –Potencies = 0 (placebo), 2000, 8000, 17000, 19000, 34000, and 67000 PFUs –Evaluate both antibody and T-cell responses –N ~40 per group Results suggested potencies above 17000 PFUs elicit immune responses –Some plateau between 34,000 and 67,000 PFUs –No safety concern

22 22 Phase II Study for Dose Selection Assess the immune responses of 2 dose levels –Potencies = 0 (placebo), 34000, and 50000 PFUs –Evaluate T-cell responses –N =398 total (1:3:3 ratio) Results showed similar immune responses of two selected potencies –1.9 fold higher than placebo (p<0.001) –Confirmed the plateau observed in phase I study

23 23 Phase III Study for Efficacy and Safety: The Shingles Prevention Study (SPS) (Oxman et al., NEJM 2005) N = 38,546 subjects ≥60 years of age randomized 1:1 to receive ZOSTAVAX ® or placebo Single dose of vaccine with potency ranging from 18,700 to 60,000 PFU (median 24,600 PFU) –To bracket end-expiry potency Average of 3.1 years of HZ surveillance and ≥6- month follow-up of HZ pain after HZ rash onset Conducted by Dept. of Veteran Affairs (VA) in collaboration with the National Institutes of Health (NIH) and Merck & Co., Inc.

24 24 Key Efficacy Endpoints of SPS HZ incidence HZ pain burden of illness (BOI) –Composite of incidence, severity, and duration of pain Postherpetic neuralgia (PHN) –Clinically significant pain persisting for or present after 90 days of HZ rash onset Success requires 95% CI lower bound for vaccine efficacy >25%

25 25 ZOSTAVAX ® Efficacy: HZ Incidence Estimate of the Cumulative Incidence of HZ Over Time by Vaccination Group

26 26 ZOSTAVAX ® Efficacy 51.3% 66.5% 61.1% 25%=prespecified lower bound success criterion

27 27 Phase IV/Market Expansion Studies After ZOSTAVAX ® Approval in 2006 Bridging between frozen and refrigerated formulation of vaccines –Allow vaccine to be distributed in markets without freezer capacity in physician’s office Concomitant use with flu vaccine –Desirable for elderly population High risk populations (HIV, immunocompromised adults) Another efficacy trial (~22,000 subjects) in 50-59 year olds

28 Opportunities for Adaptive Design Strategy

29 29 Adaptive Design Strategy New paradigm of clinical development: Learn and Confirm Build adaptive features in clinical trials to provide a “window” of opportunity to adjust the trial based on interim data Accelerate the clinical development by reducing the time gaps between studies –E.g., seamless phase II/III trials Adapt by design and not post hoc as a remedy

30 30 Features of Adaptive Design Strategy Optimize dose-response curve Interim stopping for futility or efficacy Drop the “losers” (dose selection) Adjust sample size Change of populations (enrichment) Seamless phase II/III trial Adaptation in individual study and whole development program

31 31 Adaptation Benefits More efficient clinical development –More accurate dose selection for last phase development, increase the probability of final success –Midcourse correction for trials that are off target –Fewer patients enrolled into ineffective treatment arms Reduce costs by stopping unsuccessful trials early –Shift resources to more promising candidates Reduce development time by combining phases –Reduce white space between phases –Reduce overall time of clinical development

32 32 Opportunities for Adaptive Designs in Vaccine Development Availability of immune markers –Potentially predictive of protection –Useful for assessing dose responses at phases I & II Large and long-duration efficacy trial –Rare disease incidence –High bar for success

33 33 Example 1: Adaptive Phase II/III Study for a new Vaccine (Dose selection using Immune Marker) 300 patients randomized to each arm Control Phase II Part A ( select 1 Dose ) Phase III Part B (Pivotal Efficacy) Additional 6000 patients added to each arm Low dose Med. dose High dose Interim decision point Selected dose Control F/up Dose selection decision based on safety and immune responses Non-selected dose groups Stop

34 34 Phase II/III Dose-Selection Trial Dose selection based on immune responses at phase II Efficacy outcome followed at both phases for the selected dose and control –Analysis combines data from phases II and III Overall type I error depends on –the number of doses at phase II –correlation between immune marker and disease outcome –Sample size at different phases Type I error can be controlled using closed testing procedure and/or use of exact test for efficacy comparison

35 35 Empirical Type I Error Rate for 4-Arm Dose-Selection Phase II/III Trial: Continuous Immune Marker, Binary Disease Outcome, Exact Test for Efficacy Comparison at 0.025 Level Empirical Type I Error Rate for 4-Arm Dose-Selection Phase II/III Trial: Continuous Immune Marker, Binary Disease Outcome, Exact Test for Efficacy Comparison at 0.025 Level Sample Size Sample Size Correlation (marker and disease) Disease Incidence Rate (p) 0.2 0.2 0.1 0.1 0.01 0.01 0.004 0.004 N 1 =300 ρ =0.8 0.0170.0230.0250.022 N 2 =6000 ρ =0.5 0.0160.0220.0230.021 ρ =0.1 0.0140.0190.0230.019 Based on 100,000 simulation runs

36 36 Example 2: Seamless Phase II/III Trial Using Group Sequential Design 2-Stage design comparing vaccine vs placebo for prevention of a rare infection Stage 1 (phase II): Enroll subjects to evaluate proof-of-concept (POC) of the new vaccine –If successful, continue to Stage 2 Stage 2 (phase III): Increase enrollment to confirm the efficacy of the vaccine –Build interim analysis strategy for early stopping due to either futility or overwhelming efficacy

37 37 Fixed-Endpoint Design of Vaccine Efficacy Trial Total # of Subjects (Cases) Required to Conclude Vaccine Efficacy >  (~90% power, 2% Placebo Infection Rate, one-sided  = 0.025) VE Lower Bound (  ) True VE 50%60%70%80% 0 6668 (95) 4364 (58) 2998 (37) 2106 (24) 0.10 9336 (133) 5640 (75) 3646 (45) 2458 (28) 0.20 14810 (211) 7746 (103) 4618 (57) 2984 (34) 0.25 19932 (284) 9550 (127) 5346 (66) 3336 (38)  = success criterion of VE lower bound VE = Vaccine Efficacy

38 38 Group Sequential Design (POC and Phase III): Example Testing Strategy – 25% LB Analysi s Total Cases Observed Vaccine Cases Futility/ Failure Observed Vaccine Cases for Success Efficacy = 0% Efficacy = 25% Efficacy = 70% Cumulativ e Probabilit y of Failure (%) Cumulative Probability of Success (%) Cumulative Probability of Failure (%) Cumulativ e Probabilit y of Success (%) Cumulativ e Probabilit y of Failure (%) Cumulati ve Probabilit y of Success (%) 120  10 58.8133.470.77 235  15 666685.85<0.0159.380.121.2727.12 352  19  15 98.310.0286.470.632.6570.04 Final69  22  21 99.910.0997.622.386.5193.49 POC demonstration at 2 nd Interim Analysis (Observed VE >25%)

39 39 Benefit of Seamless Phase II/III Trial Using Group Sequential Design Interim monitoring using “well accepted” statistical method Allow early stopping of trial if POC is not shown Reduce “white-space” between phase II and phase III Reduce overall sample size by leveraging data from both phases for efficacy analysis

40 40 Example 3: 2-Stage Adaptive Design with Sample Size Re-estimation Study Objective: –Vaccine prevention of a rare form of disease Main Challenge: High uncertainty in the incidence rate of disease –Rare incidence –Natural history not well studied previously –Feasibility and cost

41 41 Rationale for Adaptive Design Approach Feasibility issue: Uncertainty around incidence rate (IR) of disease Uncertainty around incidence rate (IR) of disease Establish incidence rate early in study Establish incidence rate early in study Inform final sample size requirements for efficacy demonstration, while maintaining scientific rigor Inform final sample size requirements for efficacy demonstration, while maintaining scientific rigor Options to address Large efficacy study based on most conservative IR Stand-alone epidemiology study first Adaptive study with option for sample size re-estimation Adaptive design with sample size re-estimation

42 42 Features of the 2-Stage Design Evaluate feasibility early (Stage I) based on incidence rate –Start with N=2000 with an option to increase up to 7500 Adapt sample size (total case count) requirements for efficacy demonstration, if necessary, while maintaining scientific rigor Build an interim analysis using conditional rejection probability approach to allow for –Stopping due to futility –Stopping due to overwhelming efficacy (Stage II) –Sample size increase (Stage II)

43 43 Project timeline to accrue targeted cases Observed Case Split in Stage I Acceptable Adapt Targeted Cases (if needed) Unacceptable No additional subjects required Add subjects to meet Timeline Use: placebo incidence, existing cases, number of subjects still on study Bad efficacy Stop (futility) Not clear Stop (efficacy) Good efficacy Enroll Stage I Follow for casesCase accrual OK? Stop (not feasible) Too slow Yes

44 44 Potential Case Accrual Adaptation

45 45 Conditional Rejection Probability

46 46 Adaptive Study Design - Stage I 17 cases required to have 90% power Interim analysis at 11 cases once feasibility demonstrated Total Cases Cases in Vaccine Group (Out of 11 cases) Observed Vaccine Efficacy Decision 11 0100% Stop for efficacy 1-4 ≥ 40% Continue to Stage II, with potential for adaptation ≥ 5 < 40% Stop for futility

47 47 Adaptive Study Design - Stage II Project number of cases in Stage II to achieve 80% conditional power Vaccine Cases in Stage I Additional Cases Targeted for Stage II Total Cases in Vaccine Group Decision 1 or 2 (ε 0 =0.6563/ 0.3438)6 ≤ 4 (out of 17 cases) H 0 rejected > 4 (out of 17 cases) H 0 not rejected 3 (ε 0 =0.1094) 12 ≤ 6 (out of 23 cases) H 0 rejected > 6 (out of 23 cases) H 0 not rejected 4 (ε 0 =0.0156) 24 Interim analysis at 12/24 cases ≤ 5 (out of 23 cases) Stop for efficacy ≥ 9 (out of 23 cases) Stop for futility 6 or 7 or 8 (out of 23 cases) Continue for additional 12 cases Final analysis at 24/24 cases ≤ 10 (out of 35 cases) H 0 rejected ≥ 11 (out of 35 cases) H 0 not rejected

48 48 Extensive simulations were conducted to evaluate the characteristics (study power, study duration, average enrollment, etc.) under different scenarios: –Incidence rate –Vaccine efficacy –Initial enrollment and maximum enrollment –Preferred timeline to the interim analysis and final analysis –Feasibility assessment –Interim analysis strategy –Lost-to-follow-up rate –Randomization ratio Evaluation of Design Characteristics

49 49 Design Characteristics: Interim Analysis at 11 Cases (2000 initial enrollment, 8000 maximum enrollment) VE Incidence Rate (/100 pyrs) Power(%) (1-sided 2.5% level) Probability of the Sample Size is Increased Expected Total Number of Subjects Average Study Duration(Months) Expected Total Number of Cases Percentage of stopping at (percentage of stopping for efficacy) Stage I Stage II Interim Stage II 0% 0.21.97%22306810.6 90% (1%) 9% (1%) 1% (0%) 0.51.914%23104314.5 74% (0%) 23% (2%) 3% (0%) 0.81.64%20503414.3 77% (0%) 21% (1%) 2% (0%) 80% 0.2454%2140828.5 82% (28%) 18% (17%) 0% (0%) 0.36510%22808012.4 54% (21%) 46% (43%) 0% (0%) 0.47914%23407615.4 32% (15%) 67% (63%) 1% (1%) 0.58814%23206917.0 22% (14%) 74% (70%) 3% (3%) 0.8917%21505118.3 15% (12%) 81% (76%) 4% (4%) 90% 0.2631%2040837.8 88% (52%) 12% (12%) 0% (0%) 0.3773%20907710.7 68% (45%) 32% (32%) 0% (0%) 0.4905%21107313.3 50% (40%) 50% (50%) 0% (0%) 0.5954%20906614.7 39% (35%) 60% (60%) 0% (0%) 0.8992%20404915.6 34% (34%) 66% (65%) 0% (0%)

50 50 Decision Making Numerous dynamic discussions on study design were held within statistics, cross functional areas, and with senior management –Primarily focus on scenario planning based on simulations Based on these simulation results, broad agreement on the design strategy and the selection of key design parameters was quickly achieved

51 51 Summary Adaptive trial designs are attractive and increasingly popular in an effort to accelerate clinical development Good opportunities to use adaptive designs in vaccine efficacy trials Statistical guidance and scenario planning is very important Trial simulations are important in understanding the design characteristics

52 Back-up

53 53 Antibody Responses to Vaccination Antibody increases steeply to a plateau and then decline Primary responses may have a longer lag phase and reach a lower plateau than booster responses

54 54 Temporal Antibody Responses Following Primary Immunization WHO Immunological Basis for Immunization Series, Module 1, General Immunology.

55 55 Phases of Clinical Trials Phase I –Healthy subjects –PK/PD of drugs –Modeling and simulations –Dose ranging for safety and immunogenicity of vaccines –Biomarker/assay development

56 56 Phases of Clinical Trials Phase II –Target population –Dose ranging and dose selection for safety and efficacy (or immunogenicity for vaccines) Minimum effective dose Optimal dose –Proof of concept (POC) study of efficacy –Hypothesis generating

57 57 Phases of Clinical Trials Phase III –Confirmatory trial of efficacy and safety –Demonstration of consistency of the manufacturing process for vaccines –Large scale in size –Last stage before submission for licensure

58 58 Phases of Clinical Trials Phase IV –Post-marketing studies to collect additional data on safety, efficacy or immunogenicity –Supports marketing or regulatory commitments –Expansion to different populations

59 59 Immune Responses to a new Vaccine Placebo 5 ug 30 ug 90 ug

Download ppt "Statistical Challenges and Adaptive Design Strategies in Evaluation of New Vaccines Ivan S. F. Chan, Ph.D. Late Development Statistics Merck Research Laboratories."

Similar presentations

Ads by Google