1. Demonstrating that an HIV Vaccine Lowers the Risk and/or Severity of HIV infection D.Mehrotra 1*, X.Li 1, P.Gilbert 2 1 Merck Research Laboratories, Blue Bell, PA 2 Fred Hutchinson Cancer Research Center & Univ. of Washington, Seattle, WA Adapted from Mehrotra’s talk at ENAR, Austin, TX March 21, 2005 Biostat 578A Lecture 6

2. 2 Outline HIV vaccine POC efficacy trial BOI vs. Simes’ method Adjusting for selection bias BOI vs. adjusted Simes’ method Concluding remarks

3. 3 HIV Vaccine POC Efficacy Trial Design - Randomized, double-blind, multinational trial - MRKAd5 gag/pol/nef versus placebo (1:1) - 1500 subjects at high risk of becoming HIV+ - Continue until 50 events (HIV infections) accrue Co-Primary Endpoints - HIV infection status (infected/uninfected) -Viral load set-point (mean of log 10 HIV RNA at 2 and 3 months after HIV+ diagnosis) Note: lowering of viral load set-point will presumably prevent or delay the onset of AIDS

4. 4 POC Efficacy Trial (continued) Null Hypothesis: Vaccine is same as Placebo VE = 0 and  = 0 Alternative Hypothesis: Vaccine is better than Placebo VE > 0 and/or  > 0 VE =  =true difference in mean viral load set-point among infected subjects [placebo – vaccine] Proof-of-concept (POC) is established if the composite null hypothesis is rejected at  = 5%

5. 5 POC Efficacy Trial: Data Set-Up

6. 6 Establishing POC: BOI vs. Simes’ Method

7. 7 Power (%) to Reject the Composite Null Hypothesis (assuming  = 1 log 10 copies/ml)

8. 8 Adjusting for Selection Bias Simulations led us to choose Simes’ over BOI for the POC efficacy trial. However … Test for viral load component in Simes’ method: -Is restricted to subjects that are selected based on a post-randomization outcome (HIV infection)  can suffer from selection bias. -Assesses mixture of (i) causal effect of vaccine and (ii) effect of variables correlated with VL that are unevenly distributed among the infected subgroups. References Rubin (1978), Rosenbaum (1984), Robins and Greenland (1992), Frangakis and Rubin (2002)

9. 9 Adjusting for Selection Bias (continued) Proposed approach -Adjust the viral load test for plausible levels of selection bias such that rejection of the null hypothesis becomes harder. -If the adjusted test is significant, then we have robust evidence of a causal vaccine effect. Hudgens, Hoering, Self (Statistics in Medicine, 2003) Gilbert, Bosch, Hudgens (Biometrics, 2003) [GBH] Mehrotra, Li, Gilbert (Biometrics, 2006) Adjustment is derived via the principal stratification framework of causal inference (Frangakis and Rubin, Biometrics, 2002)

10. 10 Adjusting for Selection Bias (continued) Subjects infected under placebo {S i (p) =1} partition into the protected and always-infected principal strata  To assess a causal vaccine effect: we need to compare Y i (v) (= Y i (v, alw.inf.) ) with Y i (p, alw.inf.), but the placebo VLs are a mixture of Y i (p, prot.) and Y i (p, alw. inf.)  How to identify the distribution of Y i (p, alw.inf.) ? Principal StratumPotential infection outcome under Z Potential VL outcome under Z given S i (z) = 1 ProtectedS i (v) = 0, S i (p) = 1undefinedY i (p,prot.) Always infectedS i (v) = 1, S i (p) = 1Y i (v) Y i (p,alw.inf.) Z = assigned treatment

11. 11 Adjusting for Selection Bias (continued) f p (y) = (VE)f p (prot) (y) + (1-VE)f p (alw.inf) (y)  f p (alw.inf) (y) = [w(y)/(1-VE)]f p (y) where w(y) = Pr{S i (v) =1|Y i (p) =y, S i (p) =1} is the unknown probability that a placebo infectee with VL set-point y would have been infected if given vaccine. VE and f p (y) can be estimated from the data, but not w(y). Solution: assume a “known” model for w(y).

12. 12 Adjusting for Selection Bias (continued) GBH (2003) assume a logistic model for w(y): w i,  = w(y i | ,  ) = exp(  +  y i )/{1+exp(  +  y i )}, i  n p where  is a fixed (pre-set) parameter: (i)  = 0  w i,  = 1 – VE for all i (ii)  < 0  for a 1-unit decrease in Y i (p), the odds of being in the always infected stratum increase multiplicatively by exp(-  ) (iii)  is a constant satisfying F p (  |  ) = 1 For a given , f p (alw.inf) can now be estimated.

13. 13 VL Distributions for the Protected and Always Infected Principal Strata Implied by the Logistic Model for w i (y)  = 0 (e -  = 1)  = -  (e -  =  )  = -2 (e -  = 7.4)  = 0: vaccine does not selectively protect subjects  same distribution for Y i (p, prot.) and Y i (p, prot.)  < 0: vaccine selectively protects subjects with higher VLs  selection bias leads to biased estimation of the causal effect that makes the vaccine look poorer than it is.

14. 14 Adjusting for Selection Bias (continued) Adjust the viral load test in Simes’ method: 1)Fix the selection bias parameter   0. 2)Adjust (reduce) all the VLs of placebo infectees: is non-parametric m.l.e. of 3)Let = Wilcoxon rank sum statistic comparing with

15. 15 Adjusting for Selection Bias (continued) When VE = 0, T  is the Wilcoxon rank sum statistic used for the unadjusted VL test. The distribution of T  is intractable, so the p-value for the adjusted VL test ( = p 2,  ) is obtained using a non- parametric bootstrap. Adjusted Simes’ method: for the specified , reject the composite null hypothesis if Robust evidence of a causal vaccine effect on either the infection or VL endpoint: reject the composite null hypothesis using the adjusted Simes’ method for all plausible values of .

16. 16 BOI vs. Simes’ Method: Power (%) Assuming  = 1 log 10 copies/ml

17. 17 Concluding Remarks The selection bias-adjusted Simes’ method is more powerful than the BOI method, unless VE is “large” (unlikely for a CMI-based HIV vaccine). 50 events will provide at least 80% power to establish POC provided: VE  60% or   0.75 c/ml: unadjusted Simes’ method VE  60% or   1.0 c/ml: adjusted Simes’ method. An  -spending interim analysis after 30 events is proposed (details omitted here). Estimated time between 30 and 50 events is 9-15 months.

18. 18 REFERENCES 1. Chang MN, Guess HA, Heyse JF (1994). Reduction in the burden of illness: a new efficacy measure for prevention trials. Statistics in Medicine, 13, 1807-1814. 2. Fisher RA (1932). Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh and London. 3. Frangakis CE, Rubin DB (2002). Principal Stratification in Causal Inference. Biometrics, 58, 21-29. 4. Gilbert PB, Bosch RJ, Hudgens MG (2003). Sensitivity analysis for the assessment of causal vaccine effects on viral load in HIV vaccine clinical trials. Biometrics, 59, 531- 541. 5. Hudgens MG, Hoering A, Self SG (2003). On the analysis of viral load endpoints in HIV vaccine trials. Statistics in Medicine, 22, 2281-2298. 6. Mehrotra DV, Li X, Gilbert PB. Dual-endpoint evaluation of vaccine efficacy: Application to a proof-of-concept clinical trial of a cell mediated immunity-based HIV vaccine. Biometrics, in press. 7. Robins JM, Greenland S (1992). Identifiability and exchangeability of direct and indirect effects. Epidemiology, 3, 143-155. 8. Rosenbaum PR (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. The Journal of the Royal Statistical Society, Series A, 147, 656-666. 9. Rubin DB (1978). Bayesian inference for causal effects: the role of randomization. The Annals of Statistics, 6, 34-58.

19. 19 APPENDIX Arguments against a selection-bias adjustment: -POC (not phase III) trial: not essential to precisely characterize the vaccine effect. -VE (and hence selection bias) anticipated to be small. -If vaccine prevents infection only for less virulent strains, then selection bias is more likely to make placebo look better than vaccine when comparing VLs, so the unadjusted test is already conservative from a causal inference perspective! Arguments for a selection-bias adjustment: -Will we really proceed to phase III without robust evidence of a causal vaccine effect? -To satisfy statisticians who are wary of any non- randomized comparison.

20. 20 Hypothetical Example

21. 21 Hypothetical Example (continued)

22. 22 Hypothetical Example Revisited Assigning weights to the VLs in the Placebo Group

23. 23 Hypothetical Example Revisited Robust evidence of a causal vaccine effect on VL 