1. Statistical Challenges and Adaptive Design Strategies in Evaluation of New Vaccines
Ivan S. F. Chan, Ph.D.
Late Development Statistics
Merck Research Laboratories
IMS Workshop on Design and Analysis of Clinical Trials
National University of Singapore, Singapore
October 24-28, 2011

2. Collaborators Jonathan Hartzel Xiaoming Li Josh Chen Yanli Zhao
Jenny Sun

3. Outline Introduction to Vaccines Clinical Development of Vaccines
Herpes Zoster Vaccine Example
Opportunities for Adaptive Design Strategy
Issues and Challenges in Adaptive Designs
Illustrated with Examples

4. The Ten Greatest Public Health Achievements of the 20th Century
Vaccination
Motor-vehicle safety
Safer workplaces
Control of infectious diseases
Decline in deaths from coronary heart disease and stroke
Safer and healthier foods
Healthier mothers and babies
Family planning
Fluoridation of drinking water
Recognition of tobacco use as a health hazard
MMWR (1999);48:1141

5. What Are Vaccines? Biological products
Typically for prophylaxis, not treatment
Use antigen or attenuated live virus to trigger immune responses for disease protection
Administered as a single dose or series with a potential booster dose
Highly complex immunologic milieu
Array of humoral and cellular immune responses

6. Examples of Vaccines Pediatric vaccines Adolescents and Adult vaccines
Polio
Measles, mumps, rubella (MMR)
Chickenpox (Varivax)
Hepatitis B
Diphtheria, tetanus, pertussis
Rotavirus (infant gastroenteritis, RotaTeq)
Invasive pneumococcal disease (Prevnar)
Adolescents and Adult vaccines
HPV (cervical cancer, Gardasil)
Meningitis (Menactra)
Influenza
Invasive pneumococcal disease (Pneumovax 23)
Herpes zoster (shingles, Zostavax)

7. Benefits of Vaccines Direct benefit Indirect benefit
Efficacy in clinical trials
Risk benefit at individual level
Indirect benefit
Herd immunity by reducing exposure and transmission
Public health implications

8. Types of Immunity Humoral (antibody-mediated) immunity
B lymphocytes,
Plasma cells
Immunoglobulins (Ig)
IgG, IgM, IgA, IgD and IgE
Cell-mediated immunity (CMI)
T lymphocytes
Cytokine/Interleukins

9. Functions of Immunoglobulins
Serve as antibodies
Neutralize viruses and bacterial toxins
IgG accounts for ~80% of total immunoglobulin pool
Bind antigen
Prevent or clear first infection

10. Functions of T-cells (CMI)
T lymphocytes (helper cells) stimulate B cells to produce antibodies
T suppressor (regulatory) cells play an inhibitory role and control the level and quality of the immune response (CD4)
Cytotoxic T-cells recognize and destroy infected cells (CD8)

11. Brief Overview of Clinical Development of Vaccines

12. Discovery and Development of a Successful Drug/Vaccine15 15
PHASES
MARKET LAUNCH 14 14 IV
POST-MARKETING SURVEILLANCE
POST-MARKETING SURVEILLANCE 13 13 1 1 12 12
III 11 11 2 2 10 10
DEVELOPMENT
2 - 5
2 - 5
CLINICAL TEST (HUMANS) II 9 9
YEARS 8 8 I
5 - 10
5 - 10 I 7 7 6 6
PRECLINICAL TEST (ANIMALS)
PRECLINICAL TEST (ANIMALS) 5 5 4 4 3 3
BASIC
SYNTHESIS,
EXAMINATION &
SCREENING 2 2
RESEARCH
3, ,000
3, ,000 1 1
QUANTITY OF SUBSTANCES
Source: Based on PhRMA analysis, updated for data per Tufts Center for the Study of Drug Development (CSDD) database.

13. Evaluation of New Vaccines - Safety
Assess local (injection-site) and systemic adverse experiences
Need a large database, particularly because of giving vaccines to healthy subjects
Choice of safety parameters depend on type of disease, population, and route of administration
Need large-scale post licensure study for additional safety monitoring

14. Evaluation of New Vaccines - Efficacy
Measure the relative reduction (RR) of disease incidence after vaccination compared with placebos VE = 1 – RR = 1 – PV/PC
Require a high level of evidence and precision
Success typically requires showing efficacy greater than a non-zero (e.g. 20% - 50%) lower bound
May need a very large study
Need long-term data to assess duration of efficacy
Historical controls may be used if concurrent controls are not available
When is a booster dose needed?

15. Impact of VE Lower Bound Requirement on Sample Size
Rapid increase of sample size when VE lower bound increases
Example assumes
5/1000 incidence
90% power
60% true VE
One-sided 2.5% test
1:1 randomization
VE Lower Bound
Total Sample Size
16,300
.10
20,800
.20
28,500
.30
43,900

16. Evaluation of New Vaccines - Immunogenicity
Important in understanding the biology
Humoral immunity
Antibody responses
priming, first defense
Cell-mediated immunity
T-cell responses
– prevent virus reactivation, kill infected cells
Identify immune markers that correlates with disease protection

17. Variability/Stability of Vaccines
Vaccines are biological products that have more variability in than chemical compound
Need to demonstrate consistency of manufacturing
Many vaccines contains attenuated live viruses and will lose potency over time
E.g., chickenpox vaccine, zoster vaccine
Need to establish a range of potency for manufacturing and product shelf-life
Study the safety at the high potency
Establish efficacy at near-expiry potencies

18. An Example: Clinical Development of ZOSTAVAX®
A live-virus vaccine to prevent herpes zoster (shingles)

19. Site of VZV replication
Herpes Zoster Is a Consequence of Varicella-Zoster Virus (VZV) Reactivation
Spinal cord
Dorsal
root ganglion
Site of VZV replication
Image courtesy of Courtesy of JW Gnann.
Copyright ©2005 by Merck & Co., Inc., All rights reserved.

20. Ophthalmic Zoster
Courtesy of MN Oxman UCSD/San Diego VAMC.

21. Phase I Study for Dose Ranging
Assess the immune responses of 8 dose levels
Potencies = 0 (placebo), 2000, 8000, 17000, 19000, 34000, and PFUs
Evaluate both antibody and T-cell responses
N ~40 per group
Results suggested potencies above PFUs elicit immune responses
Some plateau between 34,000 and 67,000 PFUs
No safety concern

22. Phase II Study for Dose Selection
Assess the immune responses of 2 dose levels
Potencies = 0 (placebo), 34000, and PFUs
Evaluate T-cell responses
N =398 total (1:3:3 ratio)
Results showed similar immune responses of two selected potencies
1.9 fold higher than placebo (p<0.001)
Confirmed the plateau observed in phase I study

23. Phase III Study for Efficacy and Safety: The Shingles Prevention Study (SPS) (Oxman et al., NEJM 2005)
N = 38,546 subjects ≥60 years of age randomized 1:1 to receive ZOSTAVAX® or placebo
Single dose of vaccine with potency ranging from 18,700 to 60,000 PFU (median 24,600 PFU)
To bracket end-expiry potency
Average of 3.1 years of HZ surveillance and ≥6-month follow-up of HZ pain after HZ rash onset
Conducted by Dept. of Veteran Affairs (VA) in collaboration with the National Institutes of Health (NIH) and Merck & Co., Inc.

24. Key Efficacy Endpoints of SPS
HZ incidence
HZ pain burden of illness (BOI)
Composite of incidence, severity, and duration of pain
Postherpetic neuralgia (PHN)
Clinically significant pain persisting for or present after 90 days of HZ rash onset
Success requires 95% CI lower bound for vaccine efficacy >25%

25. ZOSTAVAX® Efficacy: HZ Incidence
Estimate of the Cumulative Incidence of HZ Over Time by Vaccination Group

26. 25%=prespecified lower bound success criterion
ZOSTAVAX® Efficacy
51.3%
66.5%
61.1%
25%=prespecified lower bound success criterion

27. Phase IV/Market Expansion Studies After ZOSTAVAX® Approval in 2006
Bridging between frozen and refrigerated formulation of vaccines
Allow vaccine to be distributed in markets without freezer capacity in physician’s office
Concomitant use with flu vaccine
Desirable for elderly population
High risk populations (HIV, immunocompromised adults)
Another efficacy trial (~22,000 subjects) in year olds

28. Opportunities for Adaptive Design Strategy

29. Adaptive Design Strategy
New paradigm of clinical development:
Learn and Confirm
Build adaptive features in clinical trials to provide a “window” of opportunity to adjust the trial based on interim data
Accelerate the clinical development by reducing the time gaps between studies
E.g., seamless phase II/III trials
Adapt by design and not post hoc as a remedy

30. Features of Adaptive Design Strategy
Optimize dose-response curve
Interim stopping for futility or efficacy
Drop the “losers” (dose selection)
Adjust sample size
Change of populations (enrichment)
Seamless phase II/III trial
Adaptation in individual study and whole development program

31. Adaptation Benefits More efficient clinical development
More accurate dose selection for last phase development, increase the probability of final success
Midcourse correction for trials that are off target
Fewer patients enrolled into ineffective treatment arms
Reduce costs by stopping unsuccessful trials early
Shift resources to more promising candidates
Reduce development time by combining phases
Reduce white space between phases
Reduce overall time of clinical development

32. Opportunities for Adaptive Designs in Vaccine Development
Availability of immune markers
Potentially predictive of protection
Useful for assessing dose responses at phases I & II
Large and long-duration efficacy trial
Rare disease incidence
High bar for success

33. Part B (Pivotal Efficacy) Additional 6000 patients added
Example 1: Adaptive Phase II/III Study for a new Vaccine (Dose selection using Immune Marker)
Phase II
Part A ( select 1 Dose )
Phase III
Part B (Pivotal Efficacy)
Interim decision point
Additional 6000 patients added
to each arm
Low dose
Selected dose
300 patients
randomized to each arm
Med. dose
F/up
High dose
Control
Control
Non-selected dose groups
Stop
Dose selection decision based on safety and immune responses

34. Phase II/III Dose-Selection Trial
Dose selection based on immune responses at phase II
Efficacy outcome followed at both phases for the selected dose and control
Analysis combines data from phases II and III
Overall type I error depends on
the number of doses at phase II
correlation between immune marker and disease outcome
Sample size at different phases
Type I error can be controlled using closed testing procedure and/or use of exact test for efficacy comparison

35. Disease Incidence Rate (p)
Empirical Type I Error Rate for 4-Arm Dose-Selection Phase II/III Trial: Continuous Immune Marker, Binary Disease Outcome, Exact Test for Efficacy Comparison at Level
 Sample Size
Correlation
(marker and
disease)
Disease Incidence Rate (p)
0.2
0.1
0.01
0.004
N1=300
ρ =0.8
0.017
0.023
0.025
0.022
N2=6000
ρ =0.5
0.016
0.021
ρ =0.1
0.014
0.019
Based on 100,000 simulation runs

36. Example 2: Seamless Phase II/III Trial Using Group Sequential Design
2-Stage design comparing vaccine vs placebo for prevention of a rare infection
Stage 1 (phase II): Enroll subjects to evaluate proof-of-concept (POC) of the new vaccine
If successful, continue to Stage 2
Stage 2 (phase III): Increase enrollment to confirm the efficacy of the vaccine
Build interim analysis strategy for early stopping due to either futility or overwhelming efficacy

37. Fixed-Endpoint Design of Vaccine Efficacy Trial
Total # of Subjects (Cases) Required to Conclude Vaccine Efficacy >  (~90% power, 2% Placebo Infection Rate, one-sided  = 0.025)
VE Lower Bound ()
True VE
50%
60%
70%
80%
6668 (95)
4364 (58)
2998 (37)
2106 (24)
0.10
9336 (133)
5640 (75)
3646 (45)
2458 (28)
0.20
14810 (211)
7746 (103)
4618 (57)
2984 (34)
0.25
19932 (284)
9550 (127)
5346 (66)
3336 (38)
 = success criterion of VE lower bound
VE = Vaccine Efficacy

38. Group Sequential Design (POC and Phase III): Example Testing Strategy – 25% LB
Analysis
Total Cases
Observed Vaccine Cases Futility/ Failure
Observed Vaccine Cases for Success
Efficacy = 0%
Efficacy = 25%
Efficacy = 70%
Cumulative Probability of Failure (%)
Cumulative Probability of Success (%) 1 20
10
58.81
33.47
0.77 2 35
15 6
85.85
<0.01
59.38
0.12
1.27
27.12 3 52
19
15
98.31
0.02
86.47
0.63
2.65
70.04
Final 69
22
21
99.91
0.09
97.62
2.38
6.51
93.49
POC demonstration at 2nd Interim Analysis (Observed VE >25%)

39. Benefit of Seamless Phase II/III Trial Using Group Sequential Design
Interim monitoring using “well accepted” statistical method
Allow early stopping of trial if POC is not shown
Reduce “white-space” between phase II and phase III
Reduce overall sample size by leveraging data from both phases for efficacy analysis

40. Example 3: 2-Stage Adaptive Design with Sample Size Re-estimation
Study Objective:
Vaccine prevention of a rare form of disease
Main Challenge: High uncertainty in the incidence rate of disease
Rare incidence
Natural history not well studied previously
Feasibility and cost

41. Rationale for Adaptive Design Approach
Feasibility issue:
Uncertainty around incidence rate (IR) of disease
Establish incidence rate early in study
Inform final sample size requirements for efficacy demonstration, while maintaining scientific rigor
Options to address
Large efficacy study based on most conservative IR
Adaptive study with option for sample size re-estimation
Stand-alone epidemiology study first
Adaptive design with sample size re-estimation

42. Features of the 2-Stage Design
Evaluate feasibility early (Stage I) based on incidence rate
Start with N=2000 with an option to increase up to 7500
Adapt sample size (total case count) requirements for efficacy demonstration, if necessary, while maintaining scientific rigor
Build an interim analysis using conditional rejection probability approach to allow for
Stopping due to futility
Stopping due to overwhelming efficacy (Stage II)
Sample size increase (Stage II)

43. Stop Stop Stop (not feasible) (efficacy) (futility) Enroll Stage I
Follow for cases
Case accrual OK?
Too slow
Yes
Stop
(efficacy)
Observed Case Split in Stage I
Stop
(futility)
Good
efficacy
Bad efficacy
Not clear
Adapt Targeted Cases (if needed)
Use: placebo incidence, existing cases, number of subjects still on study
Project timeline to accrue targeted cases
Acceptable
Unacceptable
No additional subjects required
Add subjects to meet Timeline

44. Potential Case Accrual Adaptation

45. Conditional Rejection Probability

46. Adaptive Study Design - Stage I
17 cases required to have 90% power
Interim analysis at 11 cases once feasibility demonstrated
Total Cases
Cases in Vaccine Group
(Out of 11 cases)
Observed Vaccine Efficacy
Decision 11
100%
Stop for efficacy
1-4
≥ 40%
Continue to Stage II, with potential for adaptation
≥ 5
< 40%
Stop for futility

47. Adaptive Study Design - Stage II
Project number of cases in Stage II to achieve 80% conditional power
Vaccine Cases in Stage I
Additional Cases Targeted for Stage II
Total Cases in Vaccine Group
Decision
1 or 2
(ε0=0.6563/
0.3438) 6
≤ 4 (out of 17 cases)
H0 rejected
> 4 (out of 17 cases)
H0 not rejected 3
(ε0=0.1094) 12
≤ 6 (out of 23 cases)
> 6 (out of 23 cases) 4
(ε0=0.0156) 24
Interim analysis at 12/24 cases
≤ 5 (out of 23 cases)
Stop for efficacy
≥ 9 (out of 23 cases)
Stop for futility
6 or 7 or 8 (out of 23 cases)
Continue for additional 12 cases
Final analysis at 24/24 cases
≤ 10 (out of 35 cases)
≥ 11 (out of 35 cases)

48. Evaluation of Design Characteristics
Extensive simulations were conducted to evaluate the characteristics (study power, study duration, average enrollment, etc.) under different scenarios:
Incidence rate
Vaccine efficacy
Initial enrollment and maximum enrollment
Preferred timeline to the interim analysis and final analysis
Feasibility assessment
Interim analysis strategy
Lost-to-follow-up rate
Randomization ratio

49. Design Characteristics: Interim Analysis at 11 Cases (2000 initial enrollment, 8000 maximum enrollment) VE
Incidence Rate
(/100 pyrs)
Power
(%)
(1-sided 2.5% level)
Probability of the Sample Size is Increased
Expected Total Number of
Subjects
Average Study
Duration
(Months)
Expected Total Number of Cases
Percentage of stopping at
(percentage of stopping for efficacy)
Stage I
Stage II Interim
Stage II 0%
0.2
1.9 7%
2230 68
10.6
90% (1%)
9% (1%)
1% (0%)
0.5
14%
2310 43
14.5
74% (0%)
23% (2%)
3% (0%)
0.8
1.6 4%
2050 34
14.3
77% (0%)
21% (1%)
2% (0%)
80% 45
2140 82
8.5
82% (28%)
18% (17%)
0% (0%)
0.3 65
10%
2280 80
12.4
54% (21%)
46% (43%)
0.4 79
2340 76
15.4
32% (15%)
67% (63%)
1% (1%) 88
2320 69
17.0
22% (14%)
74% (70%)
3% (3%) 91
2150 51
18.3
15% (12%)
81% (76%)
4% (4%)
90% 63 1%
2040 83
7.8
88% (52%)
12% (12%) 77 3%
2090
10.7
68% (45%)
32% (32%) 90 5%
2110 73
13.3
50% (40%)
50% (50%) 95 66
14.7
39% (35%)
60% (60%) 99 2% 49
15.6
34% (34%)
66% (65%)

50. Decision Making
Numerous dynamic discussions on study design were held within statistics, cross functional areas, and with senior management
Primarily focus on scenario planning based on simulations
Based on these simulation results, broad agreement on the design strategy and the selection of key design parameters was quickly achieved

51. Summary
Adaptive trial designs are attractive and increasingly popular in an effort to accelerate clinical development
Good opportunities to use adaptive designs in vaccine efficacy trials
Statistical guidance and scenario planning is very important
Trial simulations are important in understanding the design characteristics

52. Back-up

53. Antibody Responses to Vaccination
Antibody increases steeply to a plateau and then decline
Primary responses may have a longer lag phase and reach a lower plateau than booster responses

54. Temporal Antibody Responses Following Primary Immunization
WHO Immunological Basis for Immunization Series, Module 1, General Immunology.

55. Phases of Clinical Trials
Phase I
Healthy subjects
PK/PD of drugs
Modeling and simulations
Dose ranging for safety and immunogenicity of vaccines
Biomarker/assay development

56. Phases of Clinical Trials
Phase II
Target population
Dose ranging and dose selection for safety and efficacy (or immunogenicity for vaccines)
Minimum effective dose
Optimal dose
Proof of concept (POC) study of efficacy
Hypothesis generating

57. Phases of Clinical Trials
Phase III
Confirmatory trial of efficacy and safety
Demonstration of consistency of the manufacturing process for vaccines
Large scale in size
Last stage before submission for licensure

58. Phases of Clinical Trials
Phase IV
Post-marketing studies to collect additional data on safety, efficacy or immunogenicity
Supports marketing or regulatory commitments
Expansion to different populations

59. Immune Responses to a new Vaccine
90 ug
30 ug
5 ug
Placebo