Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Peter Gilbert Vaccine Infectious Disease Institute Fred Hutchinson Cancer Research Center and University of Washington December 14, 2009 Evaluating an.

Similar presentations


Presentation on theme: "1 Peter Gilbert Vaccine Infectious Disease Institute Fred Hutchinson Cancer Research Center and University of Washington December 14, 2009 Evaluating an."— Presentation transcript:

1 1 Peter Gilbert Vaccine Infectious Disease Institute Fred Hutchinson Cancer Research Center and University of Washington December 14, 2009 Evaluating an Immunological Surrogate of Protection

2 2 Prototype Preventive Vaccine Efficacy Trial ● Primary Objective ► Assess VE: Vaccine Efficacy to prevent pathogen- specific disease ● Secondary Objective ► Assess vaccine-induced immune responses as “correlates of protection” Randomize Vaccine Measure immune response Follow for clinical endpoint (pathogen-specific disease) Receive inoculations Placebo

3 3 ● Finding an immune correlate is a central goal of vaccine research ► One of the 14 ‘Grand Challenges of Global Health’ of the NIH & Gates Foundation (for HIV, TB, Malaria) ● Immune correlates useful for: ► Affording insight into mechanisms of vaccine-protection ► Guiding iterative development of vaccines between basic and clinical research ► Shortening trials and reducing costs ► Guiding regulatory decisions ► Guiding immunization policy ► Bridging efficacy of a vaccine observed in an efficacy trial to a new setting Importance of an Immune Correlate

4 4 Considerable Uncertainty About Immune Correlates for Most Licensed Vaccines Knowledge Level about Immune Correlates for Licensed Vaccines None/LowIntermediateHigh 1. Acellular Pertussis 2. BCG Live 3. Hepatitis A 4. Japanese Encephalitis Invactivated 5. Poliovirus Inactivated 6. Rotavirus 7. Rubella Live 8. Typhoid Live 9. Yellow Fever 1. Anthrax 2. Hepatitis B Recombinant 3. Influenza Live 4. Measles Live 5. Mumps Live 6. MMR 7. Pneumococcal Polyvalent 8. Smallpox 9. Varicella Live 1. Diphtheria & Tetanus Toxoids 2. Haemophilus b Conjugate 3. Meningococcal Polysaccharide Diphtheria 4. Rabies 5. Tetanus & Diphtheria Toxoids

5 5 ● Normative for the scientific literature to not define what is meant ● Confusion in the meaning of immune correlate (or “correlate of protection”, “correlate of protective immunity”) ● This talk will: ► Clarify 3 distinct concepts of “immune correlate,” with nomenclature and definitions ► Summarize a general framework for evaluating the 3 levels of immune correlates ► Suggest how lab investigations and clinical trials can be designed to improve the development of immune correlates But ─ What Exactly is an Immune Correlate?

6 6 Take Home Points ● Consider adopting nomenclature and definitions proposed in Qin, Gilbert, McElrath, Corey, Self (2007, JID) ► “correlate of risk”, “specific surrogate of protection”, “general surrogate of protection” ► Jerald Sadoff and Janet Wittes “encourage researchers in vaccinology to adopt this useful heirarchy” (2007, JID) Opportunity to improve immune correlates assessment by augmenting standard efficacy trial designs Opportunity to improve immune correlates assessment by augmenting standard efficacy trial designs Baseline Immunogenicity Predictors: Measure participant characteristics that predict the immune responses of interest Baseline Immunogenicity Predictors: Measure participant characteristics that predict the immune responses of interest Closeout Placebo Vaccination: Vaccinate placebo recipients at the end of follow-up and measure their immune responses to vaccine Closeout Placebo Vaccination: Vaccinate placebo recipients at the end of follow-up and measure their immune responses to vaccine

7 7 Three Tiers of Immune Correlates Evaluation* NomenclatureDefinition Framework for empirical assessment Tier 1 Correlate of Risk An immune response measurement that correlates with the clinical endpoint measuring vaccine efficacy Vaccine efficacy trials/ epidemiological studies Tier 2 Specific Surrogate of Protection A correlate of risk such that vaccine effects on it reliably predict vaccine efficacy, for the same setting as the efficacy trial Single large efficacy trial or multiple similar trials Tier 3 General Surrogate of Protection A specific SoP that reliably predicts vaccine efficacy in different settings (e.g., across vaccine lots, vaccine formulations, human populations, viral populations) Multiple diverse efficacy and/or post-licensure trials *Proposed in Qin, Gilbert, McElrath, Corey, Self (2007, JID)

8 8 Outline Introduction to Three Tiers of Surrogate Endpoint Evaluation Tier 1: Correlate of Risk (CoR) Tier 2: Specific Surrogate of Protection (Specific SoP)   Statistical Surrogate (Prentice, 1989)   Principal Surrogate (Frangakis and Rubin, 2002) Tier 3: General Surrogate of Protection (General SoP) Summary and Conclusions

9 9 ● Definition: A correlate of risk (CoR) is an immunologic measurement that predicts the rate of the clinical endpoint in some population ● Example: Individuals with higher antibody titers have a lower rate of pathogen-specific disease ► In an observational study ► In the vaccine group of a Phase III trial ► In the placebo group of a Phase III trial Tier 1: “Correlate of Risk” (CoR)

10 10 Example 1. U.S. VaxGen Phase III Trial: CD4 Blocking Level a CoR for HIV Infection in Vaccine Group* In vaccine group assess CD4 blocking level as a predictor of HIV infection (Cox model) Antibody data measured on 239 infected/ 163 uninfected p =.026 *Gilbert et al., 2005,JID

11 11 Example Influenza Vaccine Field Trial (Salk, Menke, and Francis, 1945) ● N = 1,776 men in an Army Service Unit at the Univ. of Michigan ● Evaluated a trivalent vaccine containing Weiss Strain A and PR8 Strain A ● Men assigned vaccine or placebo based on alphabetical ordering of names ► Inoculations completed in 7 days ● Strain-specific Ab titers to Weiss Strain A and to PR8 Strain A measured 2 weeks post-inoculation ► Every 10 th vaccinee and 5 th placebo ● Clinical endpoint = Hospitalization due to respiratory illness with a specific influenza strain found in throat culture ● Follow-up: Oct to Jan

12 12 Example Influenza Vaccine Field Trial Distributions of Weiss Strain A Log Ab Titer

13 13 ● CoR Estimation ■ Inverse correlation between Weiss strain A Ab titers and clinical risk in the placebo group and in the vaccine group Example 2. Ab Titer a CoR for Weiss Strain A-Specific Hospitalization (Similarly for PR8 Strain A)

14 14 Key Consideration for Evaluating a CoR (1) ● ● Two-phase designs are typically used ► ► Phase I: Stratify all vaccinees by infection status and covariate strata S ► ► Phase II: Take a random sample of vaccinees within each cell, and measure the immune response of interest ● ● There are many ways to do the sampling and the analysis ► ► Most published case-control studies use an approach that ignores valuable information in the data ► ► There are approaches that leverage all the information (and are not overly complicated)- important to implement these Infected Not infected Covariate level

15 15 Key Consideration for Evaluating a CoR (1) ● ● Optimizing the two-phase design ► ► Recommend to use an approximately optimal analysis method* that leverages information in subject characteristics predictive of the immune response and of the clinical endpoint ■ ■ Provides tighter confidence intervals and greater power than methods in standard use ► ► This analysis method can be used in combination with intelligent/efficient sampling design ■ ■ Sample all cases ■ ■ Over-sample controls with unusual immune responses ■ ■ Under-sample controls for which, based on their characteristics, the anticipated variability of the immune response is low *Breslow et al. (2009, American Journal of Epidemiology)

16 16 Planning a Two-Phase CoR Study for the RV144 Thailand Trial Estimated VE = 31%; 95% CI 1% to 52%; p=0.04 ● Collaborative effort underway to evaluate a suite of immunological measurements as potential CoRs of HIV infection rate in the vaccine arm

17 17 Key Consideration for Evaluating a CoR (2) ● ● Understand the components of variability of the immunological measurement ► ► Biological variability between and within subject; technical measurement error of the assay ► ► Protection-irrelevant variability in the immunological measurement attenuates statistical power

18 18 Power to Assess a CoR with 5 Controls per Case in RV144 (195 Controls, 39 Cases)

19 19 Outline Introduction: Three Tiers of Surrogate Endpoint Evaluation Tier 1: Correlate of Risk (CoR) Tier 2: Specific Surrogate of Protection (Specific SoP)   Statistical Surrogate (Prentice, 1989)   Principal Surrogate (Frangakis and Rubin, 2002) Tier 3: General Surrogate of Protection (Principal SoP) Summary and Conclusions

20 20 ● Definition: A specific surrogate of protection (specific SoP) is an immunological correlate of risk such that ● Definition: A specific surrogate of protection (specific SoP) is an immunological correlate of risk such that vaccine effects on it reliably predict vaccine efficacy, for the same setting as the efficacy trial ► E.g., groups with no difference in immune response vaccine vs placebo have VE = 0; and groups with the greatest difference have the largest VE > 0 ► 2 detailed definitions of a specific SoP- next slides ► A specific SoP can be used to reliably predict VE for identical or similar settings as the vaccine trial Tier 2: Specific SoP

21 21 ● The specific SoP definition is consistent with the general definition of a valid surrogate endpoint International Conference on Harmonization's statement in document E8: “A validated surrogate endpoint is an endpoint which allows prediction of a clinically important outcome.” ► This concept of “reliable predictor” is distinct from a “mechanistic surrogate,” where the goal is insight rather than reliable prediction Tier 2: Specific SoP

22 22 ● An immune response can be a strong CoR but fail totally to be a specific SoP ● VaxGen Trial: Example of a CoR worthless for predicting VE ■ CD4 blocking level a “mere marker” of infection risk ■ Vaccine recipients with lowest (highest) antibody titers had immune systems less (more) able to naturally ward of infection ■ Follow-up studies that validate this explanation: ♦ Vaccine recipient sera did not neutralize 28 primary HIV-1 isolates sampled from VaxGen infected subjects (Montefiori et al., submitted) ♦ No observed “sieve effect” (no HIV sequence differences between infected vaccine and infected placebo) A CoR May Not Be a Specific SoP

23 23 Surrogate True Clinical Endpoint Outcome Vaccination Infection/Disease Assurance about surrogate validity requires a comprehensive understanding of: The biological processes leading to the clinical endpoint The effects of vaccination on the biomarker and the clinical endpoint A Valid Surrogate Fully Mediates the Vaccine Effect on the Clinical Endpoint

24 24 Infection/Disease Biomarker True Clinical Endpoint Outcome The biomarker is not in the causal pathway of the disease process Reason for Surrogate Failure (1)

25 25 HIV Exposure Antibody HIV LevelsAcquisition The immune response is not in the causal pathway of HIV acquisition (Abs only neutralize specific lab strains)  AIDSVAX VE 95% CI 6% (-24%, 17%)  Biomarkers MN neutralization titers, binding antibodies, ADCVI antibodies Illustration: VaxGen Trial

26 26 True Clinical Outcome Biomarker Endpoint Infection/Disease Intervention The surrogate is not in the pathway of the intervention's effect, or is insensitive to its effect Reason for Surrogate Failure (2)

27 27 (Sweden I Trial with DT control: 10,000 subjects)  Other relevant immune responses not captured by the assay (incomplete measurement of Ab responses) VaccineVE 95% CI SKB 58% (51%, 66%) Aventis Pasteur 85% (81%, 89%)  Biomarkers Filamentous Haemagglutinin (FHA) and Pertussis Toxoid (PT) antibody responses were superior with the SKB vaccine Illustration: Acellular Pertussis Vaccine (Tom Fleming)

28 28 FHA & PT Confirmed Antibodies Pertussis AP Vaccine Infection/Disease Other immune responses, including those resulting from additional antigens in the vaccines: ~ Pertactin ~ Fimbriae (types 2 and 3) Durability of effect Illustration: Acellular Pertussis Vaccine (Tom Fleming)

29 29 Reason for Surrogate Failure: Protection-Irrelevant Assay Variability ● ● Suppose S* is a valid surrogate endpoint, but the assay measures a noisy version of S*, S ● ● S may be unreliable as a surrogate endpoint ● ● Key Point: ► ► Limitations of the immunological assay an important cause for surrogate failure ► ► Defining assay components of variability a key part of developing SoPs

30 30 Validating a Surrogate of Protection a Hard Problem “But the parts of the world are all so related and linked together that I think it is impossible to know one without the other and without the whole.” −Blaise Pascal [1670, The Pensees, Lafuma Edition, Number 199]

31 31 Surrogate Failure Due to Measurement Error ● ● Measurement error in the assay used to define a biomarker endpoint can erode its surrogate validity ● ● Example: Genital VL vs Plasma VL as surrogate for secondary transmission ► ► Mechanistically, genital VL may be a better surrogate than plasma VL ► ► However, the seminal VL assay [qt-HIV assay (Gen-Probe)] has much greater variability than the plasma VL assay ► ► Plasma VL is a stronger predictor of transmission than seminal VL (Butler, Little, et al., 2 nd International Workshop on HIV Transmission, 2007) ● ● Defining assay components of variability a key part of developing SoPs

32 32 Two Definitions of a Specific SoP 1.Statistical SoP Prentice (1989, Statistics in Medicine) definition of a surrogate endpoint Prentice (1989, Statistics in Medicine) definition of a surrogate endpoint Based on observed associations Based on observed associations 2.Principal SoP Builds on Frangakis and Rubin’s (2002, Biometrics) definition of a surrogate endpoint Builds on Frangakis and Rubin’s (2002, Biometrics) definition of a surrogate endpoint Based on potential outcomes framework for causal inference Based on potential outcomes framework for causal inference

33 33 Prentice (1989, Stats Med) Definition and Operational Criteria for a Statistical SoP Definition: A statistical SoP is an immunologic measurement satisfying: Definition: A statistical SoP is an immunologic measurement satisfying: 1.Vaccination impacts the immunological marker 2.The immunological marker is a CoR in each of the vaccine and placebo groups 3.The relationship between the immunological marker and the clinical endpoint rate is the same in the vaccine and placebo groups  I.e., after accounting for the marker, vaccine/placebo assignment contains no information about clinical risk  Interpretation: All of the vaccine effect on the clinical endpoint is mediated through the marker

34 34 ● 98% retention of study participants ● Incidence of Hospitalization with Weiss Strain A ► Placebos:75 of 888(8.45%) ► Vaccinees: 20 of 888(2.25%) Estimated VE = (1 – 2.25/8.45)×100% = 73% Example Influenza Vaccine Field Trial (Salk, Menke, and Francis, 1945)

35 35 Evaluate Weiss Strain A Ab Titer as a Specific SoP ● Check Criteria 1 and 2 ► Log Weiss Strain A Ab titers significantly greater in vaccine group than placebo group ► Log Weiss Strain A Ab titer strongly inversely correlated with case rate in each study group [shown earlier] ► Criteria 1 and 2 hold ● Check Criterion 3 ► Assess consistency between CoR models in the vaccine and placebo groups

36 36 Demonstration of Criterion 3 for a Specific SoP Logistic Regression Models: Estimated Coefficients (SE) Placebo Group OnlyPlacebo and Vaccine Groups Model 1Model 2Model 3Model 4 Intercept 1.80(0.54)-2.38(0.12)1.62(0.45)1.80(0.54) log(Titer) -1.30(0.14)--0.98(0.12)-1.03(0.14) Trt (0.25)0.33(0.32)-0.43(1.28) Trt x log Titer (0.25) As suggested by Model 3, vaccine assignment does not affect influenza risk after controlling for log Ab titer  Criterion 3 holds Weiss Strain A

37 37 Criterion 3 Holds: Relationship Between Log Ab Titer and Influenza Risk the Same in the Vaccine and Placebo Groups Weiss Strain A

38 38 Further Validation of a Statistical SoP: Estimated VE  Predicted VE ● Direct estimate of VE (ignoring Ab titer) ► Estimated VE = 73% ● Predicted VE ► Based on the CoR model for the placebo group and the distribution of Ab titers in the vaccinated ► Predicted VE = 82% ● Conclusion: Log Weiss strain A Ab titer satisfies the Prentice criteria for a statistical SoP

39 39 What about PR8 Strain A Ab titers? Evaluate them as a CoR and a SoP for hospitalization with PR8 Strain A-specific influenza infection

40 40 ● Incidence of Hospitalization with PR8 Strain A ► Placebos:73 of 888(8.22%) ► Vaccinees: 20 of 888(2.25%) Estimated VE = (1 – 2.25/8.22)×100% = 73% (incidentally the same Estimated VE as for Weiss Strain A) VE for PR8 Strain A

41 41 Criterion 3 Fails: Relationship Between Log Ab Titer and Influenza Risk Different in the Vaccine and Placebo Groups

42 42 Logistic Regression Models: Estimated Coefficients (SE) Control Group Control and Vaccine Groups Model 1 Model 2 Model 3 Model 4 Intercept (0.59) (0.12) (0.53) (0.59) log(Titer) (0.15) (0.14) (0.15) Tmt (0.26) (0.34) (1.79) Tmt*log(Titer) (0.34) PR8 Strain A Evidence that vaccination impacts PR8 Strain A hospitalization after accounting for Ab titer

43 43 Estimated and Predicted VE: PR8 Strain A ● Direct estimate of VE (ignoring Ab titer) ► Estimated VE = 73% ● Predicted VE ► Based on the CoR model for the placebo group and the distribution of Ab titers in the vaccinated ► Predicted VE = 33% ● ● Full mediation condition for a statistical surrogate: ► ► Log(Ab titer) does not satisfy criterion ► ► Only ~½ of overall protective effect is predicted from effect on Ab titer

44 44 Why Are PR8 Strain A Titers an Imperfect SoP? ● ● Protection from PR8 Strain A only partly described by PR8 Ab titer ► ► A possible explanation is that antibodies are protective, but the measurements reflect something else besides protective responses (e.g., measurement error)

45 45 Why Are PR8 Strain A Titers an Imperfect SoP? ● ● Protection from PR8 Strain A only partly described by PR8 Ab titer ► ► Another possible explanation is that there are other protective immune responses that were not measured ■ ■ E.g., cell-mediated immune responses ► ► Another is that PR8 Strain A has different protective determinants than Weiss Strain A ► ► Yet another is that PR8 Ab titer is a valid SoP, but there was residual confounding in the regression assessment (next)

46 46 Challenge with the Statistical SoP Approach (1) ● For efficacy trials of pathogen-naïve participants, the immune response of interest will be “non-response”/zero for (almost) all placebo recipients ● No variation of the marker in the placebo arm implies: ► CoR model in placebo group cannot be evaluated (Criterion 2) ► Conceptually difficult to check “full mediation” (Criterion 3)

47 47 Challenge with the Statistical SoP Approach (2) ● Checking “full mediation” entails checking, for each immune response level s, equal clinical risk between the groups {Vaccinees w/ marker level s} vs {Placebos w/ marker level s} ● However, S is measured after randomization ► Therefore this comparison is subject to confounding, potentially misleading about the biomarker’s value as a SoP

48 48 Illustration of Post-randomization Selection Bias ● Binary immunologic measurement (positive or negative) ● Consider an unmeasured covariate reflecting strength of immune system (strong or weak) 2000 Placebo N = Vaccine 1000 Weak 1000 Strong 10% w/ pos response 10% w/ pos response 90% w/ pos response 90% w/ pos response 0% w/ pos response 0% w/ pos response 0% w/ pos response 0% w/ pos response

49 49 Criterion for a Statistical SoP: Compares Apples and Oranges Neg response Weak Placebo Placebo Vaccine Vaccine {Vaccinees w/ neg response} vs {Placebos w/ neg response} compares clinical endpoint rates between the groups compares clinical endpoint rates between the groups Strong Compares a group with 90% weak immune systems to one with 50% weak immune systems: Incomparable

50 50 Assumptions Needed for the Statistical SoP Approach to be Valid ● Assumes all subject characteristics predictive of both the biomarker and the clinical endpoint are included in the regression model ► Unverifiable assumption, similar to “no unmeasured confounders” ► Need biological understanding to make the assumption plausible ● Assumes all subject characteristics predictive of the clinical endpoint prior to and after the measurement of the biomarker are included in the regression model ► Must include all clinical risk factors in the regression model

51 51 Alternative Approach to Evaluating a Specific SoP ● These limitations motivate research into an alternative approach to surrogate endpoint evaluation ► Principal surrogate framework which leverages augmented data collection from vaccine efficacy trials

52 52 Outline Introduction: Three Tiers of Surrogate Endpoint Evaluation Tier 1: Correlate of Risk (CoR) Tier 2: Specific Surrogate of Protection (Specific SoP)   Statistical Surrogate (Prentice, 1989)   Principal Surrogate (Frangakis and Rubin, 2002) Tier 3: General Surrogate of Protection (Principal SoP) Summary and Conclusions

53 53 Motivation for Principal SoP Definition: CD4 Blocking Level a CoR in the Vaccine Group [VaxGen Trial] Antibody data measured on 239 infected/ 163 uninfected p =.026

54 54 Motivation for Principal SoP Definition: Re-Analysis with Placebo Group as Reference

55 55 How Assess Immune Response as a Principal SoP?GROUP Quartile 1 Quartile 2 Quartile 3 Quartile 4 Placebo ? ? ? ?.11 Vaccine VE 1 –.18/? 1 –.10/? /? /? Infection Rate by Quartile of Immune Response to Vaccine

56 56 Hypothetical Example 1: The CoR is Not a Principal SoP (VE flat)GROUP Quartile 1 Quartile 2 Quartile 3 Quartile 4 Placebo Vaccine VE Infection Rate by Quartile of Immune Response to Vaccine

57 57 Hypothetical Example 2: The CoR is a Principal SoP (VE Increases with Ab Titer)GROUP Quartile 1 Quartile 2 Quartile 3 Quartile 4 Placebo Vaccine VE Infection Rate by Quartile of Immune Response to Vaccine

58 58 Definition of a Principal SoP Consider the case where all placebo recipients have S = 0 Consider the case where all placebo recipients have S = 0 Define Define VE(s) = 1 – ● Interpretation: Percent reduction in clinical risk for groups of vaccinees with Ab titer s compared to if they had not been vaccinated ● Definition: A Principal SoP is an immunologic measurement satisfying 1.VE(negative response s = 0) = 0[Necessity] 2.VE(high enough response s) > 0[Sufficiency] Risk of infection for Vaccinees with Ab titer s to Vaccine Risk of infection for Placebos with Ab titer s to Vaccine

59 59 The Principal SoP Framework Provides a Way to Compare “Surrogate Value” of Different Markers to Predict VE Immunological measurement percentile v VE(s) s

60 60 Challenge to Evaluating a Principal SoP: The Immune Responses to Vaccine are Missing for Placebos ● Accurately filling in the unknown immune responses is needed to evaluate a principal SoP ● Approaches to filling in the missing data (suggested by Dean Follmann, 2006, Biometrics): ► Baseline immunogenicity predictors (BIP) ► Close-out placebo vaccination (CPV) ● Literature on statistical methods for evaluating a principal SoP ► Follmann (2006, Biometrics)* Gilbert and Hudgens (2008, Biometrics)* ► Joffe and Greene (2008, Biometrics)Qin, Gilbert, Follmann, Li (2008, AOAS)* ► Gilbert, Qin, Self (2009a, b, Stats Med)* Gallop, Small, et al. (2009, Stats Med) ► Li, Taylor, and Elliott (2009, Biometrics)Wolfson and Gilbert (in press, Biometrics)* ► Hudgens and Gilbert (in press, Biometrics)* *Method uses BIP and/or CPV

61 61 Schematic of Closeout Placebo Vaccination (CPV) & Baseline Immunogenicity Predictor (BIP) Trial Designs* X X - - S=S(1) ScSc *Proposed by Follmann (2006, Biometrics)    1  1 +   1  1 +  S(1) CPV Approach BIP Approach Vx

62 62 Closeout Placebo Vaccination (CPV) Approach ● At the end of the trial, inoculate uninfected placebo recipients with HIV vaccine ● Measure the immune response on the same schedule as it was measured for vaccinees ● Assume the measurement is what we would have seen, had we inoculated during the trial

63 63 Baseline Immunogenicity Predictor (BIP) Approach Evaluate correlation XS of X and S in vaccine group S(1) Predict S(1) from vaccine group model and X in controls X X Control

64 64 Application of CPV & BIP to Thai Trial RV144 ● ● In discussions with Thai trial leadership to implement CPV approach ► ► Vaccinate ~400 uninfected placebo recipients ● ● Will enable application of CPV and BIP methods to evaluate various antibody measurements and T cell measurements as specific SoPs

65 65 Illustration with 1943 Influenza Trial: BIP Approach ● S = log Ab titer to Weiss strain A 2 weeks post-vaccination ● X = Same measurement at baseline ● X and S are correlated (natural immune response predicts vaccine-induced immune response) ● Would like to evaluate S as a specific SoP using X as the BIP

66 66 Illustration with 1943 Influenza Trial: BIP Approach ● S = log Ab titer to Weiss strain A 2 weeks post-vaccination ● X = Same measurement at baseline ● X and S are correlated (natural immune response predicts vaccine-induced immune response) ● Would like to evaluate S as a specific SoP using X as the BIP ● Unfortunately, we do not have the data on X ► Consider a “poorer analysis” using the fact that S is measured for subjects in both the vaccine and control groups

67 67 Illustration with 1943 Influenza Trial: BIP Approach ● S = log Ab titer to Weiss strain A 2 weeks post-vaccination Inverse ranking approach to filling in S for placebos ► Assume any two placebos with log Ab titers s1 Plac s2 ► This assumption allows construction of a complete construction of a complete dataset of S’s for all subjects dataset of S’s for all subjects for whom the Ab titer was for whom the Ab titer was measured measured Ab titer observed in placebos Imputed S (Ab titer to vaccine) or 128 (coin flip) Assume Inverse Ranking* *Supported by studies including Gorse et al. (2004, JID)

68 68 Estimation of VE(s) ● For each s = {32, 128, 256, 512, 1024, 2048, 4096, 8192} (the observed values for vaccinees) estimate VE(s) by (the observed values for vaccinees) estimate VE(s) by Est. VE(s) = 1 - ● Will also estimate the case rates by fitted values from logistic regression models Case Rate for Vaccinees with Ab titer s Case Rate for Placebos with imputed Ab titer s

69 69 Influenza Trial: Estimated VE(s) Under Inverse Rank- Preserving Assumption

70 70 Sensitivity Analysis: Estimated VE(s) Under Rank-Preservation

71 71 Ab Titers Have Excellent Principal Surrogate Value

72 72 Sensitivity Analysis: Equipercentile Assumption

73 73 Example of a Good Baseline Predictor: Antibody Levels to Hepatitis A and B Vaccination (n=75, Czeschinski et al., 2000) Czeschinski et al (2000) Vaccine 18: r =.85 No cross- Reactivity

74 74 Outline Introduction: Three Tiers of Surrogate Endpoint Evaluation Tier 1: Correlate of Risk (CoR) Tier 2: Specific Surrogate of Protection (Specific SoP)   Statistical Surrogate (Prentice, 1989)   Principal Surrogate (Frangakis and Rubin, 2002) Tier 3: General Surrogate of Protection (Principal SoP) Summary and Conclusions

75 75 Challenge in Evaluating a Surrogate from a Single Trial ● ● There is less precision for validating a surrogate than there is for directly assessing the vaccine effect on the clinical endpoint! ● ● Suggests the necessity of: ► ► Bringing in extra information such as with the baseline immunogenicity predictors and closeout placebo vaccination approach ► ► Meta-analysis of multiple studies

76 76 Tier 3: General SoP ● Definition: ● Definition: An immunologic measurement that is a specific SoP in an efficacy trial(s) and is predictive of VE in different settings (e.g., across vaccine lots, vaccine formulations, human populations, viral populations) ● Approach to Evaluation: Meta-Analysis ► N pairs of immunologic and clinical endpoint assessments among vaccinees and placebos ► Pairs chosen to reflect specific target of prediction ■ Example: N vaccine efficacy trials for N annual flu seasons; Goal is to predict VE for next year’s flu season ► Evaluation: Study the relationship between the estimated VE and the estimated vaccine effect on the immune response

77 77 Meta-Analysis Example: VaxGen Trials of rgp120 Immunogens with Different HIV Strains in the Vaccine and Different Study Populations 2 Real Trials U.S. trial of MN/GNE8 gp120 in MSM Thai trial of MN/A244 gp120 in IDU × 95% prediction interval for VE New vaccine

78 78 Meta-Analysis Hypothetical Example: 10 Efficacy Trials of rgp120 Immunogens with Different HIV Strains in the Vaccine and Different Study Populations 2 Real Trials + 8 new hypothetical trials Demonstrates that CD4 blocking level is not useful as a SoP for HIV infection 95% prediction interval for VE × New vaccine

79 79 HIV-1 RNA and CD4 as CoRs for AIDS [HIV Surrogate Marker Collaboration Group, 2000, AIDS Res Hum Retroviruses]

80 80 Surrogate Evaluation from Multiple Trials: Meta-Analysis (N = 25)* Treatment Effect on AIDs vs Treatment Effect on VL Treatment Effect on AIDs vs Treatment Effect on CD4 *HIV Surrogate Marker Collaborative Group, 2000, AIDS Res Hum Retroviruses

81 81 Simulated Meta-Analysis Based on 29 Influenza Vaccine Trials (Villari et al., 2004, Vaccine) - - Selected all placebo- controlled influenza vaccine trials of PIV vaccines with ≥ 5 placebo cases N = 29 studies spanning different flu seasons over years Objective: Predict VE for next year’s flu season Clinical endpoint = Clinically confirmed influenza infection Marker endpoint = log Ab titer to the dominant strain circulating in the trial region Est. VE in trial Est. VE in trial Estimated VE vs Average difference in log Ab titer (Vaccine - Placebo) Confidence Elipses

82 82 Simulated Meta-Analysis Based on 29 Influenza Vaccine Trials (Villari et al., 2004, Vaccine) - -

83 83 Predicting VE in a New Trial ● ● Building on Daniels and Hughes (1997, Stats Med), Gail et al. (2000, Biostatistics) developed methods for predicting VE with a bootstrap confidence interval in a new trial from ► ► A series of N trials with estimated vaccine effects on the biomarker and on the clinical endpoint ► ► A new trial with data on the biomarker only ● ● Summary of Gail et al. conclusions: ► ► The strength of correlation of vaccine effects has a large effect on the precision for predicting VE ► ► Need at least N=10 studies that have reasonably precise estimates of VE ■ ■ Even with this, prediction of VE is quite imprecise ► ► Challenge: Do the N studies constitute an appropriate basis for extrapolating results to the setting of the new trial?

84 84 Summary on Tier 3 Evaluation ● Meta-analysis is part of the statistical validation of SoPs ► Directly assesses how well vaccine effects on the surrogate predict vaccine effects on the clinical endpoint ► The specific principal SoP framework is similar conceptually to meta-analysis (more in sequel talks) ● Of course, large resource challenges to conducting several diverse efficacy trials

85 85 Summary and Conclusions Tier 1: Correlate of Risk (CoR) 1. 1.Recommend to power efficacy trials to detect CoRs, taking into account factors such as protection-irrelevant variability in the immunological measurement that can attenuate power Use a relatively efficient method such as Breslow et al. (2009, AJE) 2. 2.A CoR may not be a SoP [“a correlate does not a surrogate make” (Tom Fleming)] 3. 3.Identifying a CoR generates the hypothesis that it is also a specific SoP, and possibly also a general SoP for certain kinds of predictive bridges

86 86 Summary and Conclusions Tier 2: Specific Surrogate of Protection 1. 1.The statistical SoP approach has 2 challenges: – –Difficult to handle immunological measurements with no responses in placebos – –For validity assumes that all clinical risk factors and all simultaneous predictors of the immune response and clinical risk are measured and are correctly accounted for 2. 2.The principal SoP approach may be more promising when can find a reasonable way to fill in placebos’ immune responses to vaccine. Therefore evaluating a principal SoP presents an opportunity for innovative trial design and data collection A specific SoP can be used for predicting VE for the same or similar setting as the efficacy trial. It may mislead for predicting VE in a new setting.

87 87 Summary and Conclusions Tier 3: General Surrogate of Protection 1. 1.Predictions based on a general SoP apply to a particular type of predictive bridge 2. 2.Meta-analysis can be used to empirically evaluate whether a specific SoP is a general SoP 3. 3.Predicting VE for a new setting based on meta-analysis usually can only be done with low precision, and it may be difficult to know that the N selected trials form a reliable basis for prediction (Gail et al., 2000, Biostatistics) 4. 4.Mechanistic/biological knowledge can form the basis for making the leap from a specific SoP to a general SoP

88 88 Take Home Points ● Consider adopting nomenclature and definitions proposed in Qin, Gilbert, McElrath, Corey, Self (2007, JID) ► “correlate of risk”, “specific surrogate of protection”, “general surrogate of protection” ► Jerald Sadoff and Janet Wittes “encourage researchers in vaccinology to adopt this useful heirarchy” (2007, JID) Opportunity to improve immune correlates assessment by augmenting standard efficacy trial designs Opportunity to improve immune correlates assessment by augmenting standard efficacy trial designs Baseline Immunogenicity Predictors: Measure participant characteristics that predict the immune responses of interest Baseline Immunogenicity Predictors: Measure participant characteristics that predict the immune responses of interest Closeout Placebo Vaccination: Vaccinate placebo recipients at the end of follow-up and measure their immune responses to vaccine Closeout Placebo Vaccination: Vaccinate placebo recipients at the end of follow-up and measure their immune responses to vaccine

89 89 Thanks! ● ● Ira Longini for the invitation to speak and Betz Halloran for the invitation to teach on this topic at the University of Washington Summer Institute in Statistics and Modeling in Infectious Diseases [2009 & planned for 2010] ● ● Collaborators on statistical methods for assessing immune correlates ► ► Steve Self, Michael Hudgens, Julian Wolfson, Li Qin, Ying Huang, Dean Follmann ● ● HVTN collaborators on the evaluation of immune correlates ► ► Core and lab led by Larry Corey and Julie McElrath ● ● NIH for funding this research (R01 grant and HVTN funding)

90 90 “Anything worth doing takes more than one generation, therefore there must be hope.” − Reinold Niebuhr

91 91 “But the parts of the world are all so related and linked together that I think it is impossible to know one without the other and without the whole.” −Blaise Pascal [1670, The Pensees, Lafuma Edition, Number 199]

92 92 “But the parts of the world are all so related and linked together that I think it is impossible to know one without the other and without the whole.” −Blaise Pascal [1670, The Pensees, Lafuma Edition, Number 199]

93 93 EXTRA SLIDES

94 94 Summary and Conclusions TermRemarks CoR Correlate of Risk Level 1 1.Efficacy trials should be powered to detect CoRs, taking into account error in the immunologic measurement that can attenuate power 2.A CoR may not be useful for predicting vaccine efficacy 3.Identifying a CoR generates the hypothesis that it is also a specific SoP, and possibly also a general SoP for certain kinds of predictive bridges Specific SoP Specific Surrogate of Protection SoP S (Statistical) SoP P (Principal) Level 2 1.The SoP S has 2 problems: - Difficult to handle immunological measurements with no responses in placebos - Inferences susceptible to post-randomization selection bias 2.The SoP P may be more promising when can find a reasonable way to fill in placebos’ immune responses to vaccine. Therefore evaluating a SoP P presents an opportunity for innovative trial design and data collection. 3.A specific SoP can be used for predicting VE for the same setting as the efficacy trial, but may mislead for predicting VE in a new setting. Mechanistic/biological knowledge can form the basis for making this leap from a specific to a general SoP. General SoP Genearl Surrogate of Protection Level 3 1.Meta-analysis is data-intensive and is required to directly empirically evaluate whether a CoP is a SoP 2.A SoP is specific to the type of predictive bridge 3.Predicting VE for a new setting without waiting for clinical endpoint data usually can only be done with low precision, and it may be difficult to know that the analyzed set of trials forms a reliable basis for prediction (Gail et al., 2000)

95 95 Immunologic Measurements at Three Levels of Immune Correlates TermDefinitionFramework for empirical assessment Data analytic methods CoR Correlate of Risk Level 1 An immunologic measurement that correlates with the rate or level of a study endpoint measuring vaccine efficacy in a defined population. Vaccine efficacy trials or analytic epidemiological studies Regression models CoP Correlate of Protection CoP S Correlate of Protection Statistical CoP C Correlate of Protection Causal Level 2 An immunologic measurement that correlates with the protective effects of the vaccine within a single study, satisfying: 1.The immunologic measurement is a CoR in each of the vaccinated and unvaccinated groups; and 2.The relationship between the immune response and clinical risk is the same in the vaccinated and unvaccinated groups. 1. Groups of subjects with no causal vaccine effect on the immune response have no vaccine efficacy; and 2. Groups of subjects with greater causal vaccine effects on the immune response have greater vaccine efficacy. Single large vaccine efficacy trial Prentice (1989) framework Causal framework Frangakis and Rubin (2002) Follmann (2006) Gilbert and Hudgens (2006) SoP Surrogate of Protection Level 3 An immunologic measurement that is a CoP and is predictive of vaccine efficacy in different settings (e.g., across vaccine lots, vaccine formulations, human populations, viral populations). Multiple efficacy and/or post- licensure trials Meta-analysis Daniels and Hughes (1997) Gail et al. (2000)


Download ppt "1 Peter Gilbert Vaccine Infectious Disease Institute Fred Hutchinson Cancer Research Center and University of Washington December 14, 2009 Evaluating an."

Similar presentations


Ads by Google