1. Augmented designs to assess immune responses in vaccine efficacy trials Talk adapted from Dean Follmann’s slides NIAID Biostat 578A Lecture 12

2. Vax004 First Phase III trial of an HIV vaccine 5403 HIV- volunteers randomized Measured antibody response to vaccine two weeks after each vaccination in vaccine group Looked at rate of infection as a function of antibody response

3. Vax004: Relative Risk of Infection Within Vaccine Group GROUP WeakModestGoodBest Vaccine1.00.43*.34*.29* * p<.05 IMMUNE RESPONSE QUARTILE Result: Antibody level is a Correlate of Risk (COR)

4. Vax004: Relative Risk of Infection Both Groups GROUP WeakModestGoodBest Placebo ? ? ? ?1.00 Vaccine1.67*.98.87.74 * p<.05 IMMUNE RESPONSE QUARTILE ? is X(1), the unknown potential immune response if assigned vaccine [more on this later]

5. What does this mean? Two possible explanations –Mere Association: The vaccine is useless, but those individuals who could mount a strong immune response are better able to remain uninfected –Causation: The vaccine tends to cause infections in those who have poor immune response, but may prevent infections in those with good immune responses

6. Suppose X = immune response induced by the HIV vaccine Phase III study with infection rates 10% placebo, 8% vaccine group Correlation of X with infection rate in vaccinees same as seen in Vax004 After celebration over, tinker with vaccine to improve the immune response X to the HIV vaccine –Could this tinkering be a waste of time?

7. Goal Replace the ?s with numbers. Want to know if immune response is merely associated with infection rate or is causative We’ll discuss two different and complementary approaches –Baseline Predictor (BP) –Closeout Placebo Vaccination (CPV) These can be used together or separately

8. Baseline Predictor (BP) At baseline, measure something(s), W, that is correlated with X –E.g., inoculate everyone with rabies vaccine, and shortly afterwards, W = immune response to rabies vaccine –E.g., W = antibody levels to a non-HIV virus such as cytomegalovirus or Adenovirus Randomization ensures (X,W) same in both groups Use a placebo subjects’ W to impute X

10. Closeout Placebo Vaccination At the end of the trial, inoculate placebo uninfecteds with HIV vaccine Measure immune response X C on the same schedule as was measured for vaccinees Pretend X C is what we would have seen, had we inoculated at baseline, i.e., X C = X 0

11. GroupOutcomeWeakModestGoodBestTotal VaccineUninfected 70 85 90 95340 Infected 30 15 10 5 60 Total100 400 PlaceboUninfected ? ? ? ?340 Infected ? ? ? ? 60 Total ? ? ? ?400 Immune Response Quartiles Observable Data in Standard Trial Design

12. GroupOutcomeWeakModestGoodBestTotal VaccineUninfected 70 85 90 95340 Infected 30 15 10 5 60 Total100 400 PlaceboUninfected ? ? ? ?340 Infected ? ? ? ? 60 Total~100 400 Immune Response Quartiles Standard Trial Design Exploiting Randomization

13. GroupOutcomeWeakModestGoodBestTotal VaccineUninfected 70 85 90 95340 Infected 30 15 10 5 60 Total100 400 PlaceboUninfected 71 84 92 93340 Infected~29 ~16~8~7 60 Total~100 400 Immune Response Quartiles Trial with CPV (Association Hypothesis True)

14. Schematic of BP& CPV

15. Assumptions No noncompliance No missing data Infections start after X 0 is measured If not vaccinated, a subject has X = 0 [i.e., X 0i (0) = 0] Time constancy of immune response: X Ci = X 0i

16. What is X? X = X(1) is the potential HIV specific immune response to HIV vaccination Vaccine: What a subject did produce in response to the vaccine. (realized: X = X(1)) Placebo: What a patient would produce in response to the vaccine. (unrealized: X(1) not observed- missing covariate)

17. Probit Regression Assume the probability of infection varies smoothly with X –Placebo Group: –Vaccine Group:

18. Causal Interpretation of Parameters A causal VE(x) parameter is some contrast in Pr{Y(1)=1|X(1)=x,X(0)=0} & Pr{Y(0)=1|X(1)=x,X(0)=0} –These probabilities can be written as Pr(Y(z)=1|X(1)=x), because all X(0)’s are zero Set  1 +  3 x =  -1 (Pr{Y(1)=1|X=x}) –  -1 (Pr{Y(0)=1|X=x}) –  3 = 0 reflects the association hypothesis –  2 = 0 and  3 < 0 reflects the causation hypothesis –  2 < 0 and  3 < 0 reflects a mixture of the two hypotheses

19. Maximum likelihood estimation (using both BP and CPV) We need X for P(Y=1 |X,Z) –Vaccine group, use X 0 –Placebo Uninfecteds use X C –Placebo Infecteds: Integrate P(Y=1 | X, 0) with respect to the distribution of X|W –If (X,W) is assumed bivariate normal, then –The parameters  (w) and  * are the mean and standard deviation of the X|W=w normal distribution

20. Likelihood (using both BP and CPV) Vaccine contribution  i in V p v (x 0i ) y(i) (1 - p v (x 0i ) ) 1 - y(i) Placebo contribution  i in P(U) (1 - p p (x Ci ) ) 1 - y(i)  i in P(I) p * (w i ) y(i) w i at baseline, x 0i shortly after randomization, x Ci at closeout

21. Maximum Likelihood Likelihood maximized using R (standard maximizers) Bootstrap used to estimate standard errors of parameters for Wald Tests

22. Simulation N=1000 per group Infection rates 10%/8% placebo/vaccine; 180 total infections Causation –Risk gradient in Vaccine group, none in placebo Association –Similar risk gradient in both groups X,W correlation 0,.25,.50,.75, 1

23. Placebo Vaccine

24. Placebo Vaccine

25. Results Measure performance by simulation sample variance of coefficient estimate, relative to the case where X is known for all subjects Association Scenario, correlation r =.5 Design Variance of  2 estimate Relative Variance CPV.057515.5 BP.0199 5.4 CPV+BP.0145 3.9 X known.0037 1.0

27. Performance depends on r If r >.50, little need for CPV If r =.25, both CPV and BP are helpful If r = 0, BP useless If r = 1, CPV useless

28. Statistical Power BP alone r =.5 N=2000/5000 180/450 total infections 22  3 Scenario.86/1.00.03/.05Association.04/.05.78/.99Causation.57/.95.35/.65Both

29. Is an improved vaccine good enough? Suppose Vaccine A had 20% VE Small studies of Vaccine A* showed the immune response is increased by  Will this be enough to launch a new Phase III trial? Using our statistical model, we can estimate the VE for A*, say VE*. Is it worth spending $100 million? Go/No go decision based on VE*, not 

30. Summary BP and CPV can be added onto standard vaccine efficacy trials to replace the “?”s in the placebo group Vaccine development focuses on cultivating the best immune response. But –Immune response may be partly causative –Different responses may be more/less causative Important to consider augmented designs to properly assess role of immune response Could incorporate BP into phase 1 or 2 trials to assess correlation

31. Summary This work is an example of a method for assessing validity of a principal surrogate endpoint [Frangakis and Rubin’s definition of a surrogate endpoint] Many open problems –More flexible modeling (semiparametric)/relax assumptions/diagnostics –Extend from binary endpoint to failure time endpoint –Fitting into a case-cohort design framework –Handling multiple immune responses Repeated measures Multiple immune responses- which combination best predicts vaccine efficacy? –Other trial designs (e.g., cluster randomized)