1. Adjusted Likelihoods for Synthesizing Empirical Evidence from Studies that Differ in Quality and Design: Effects of Environmental Tobacco Smoke Kerrie Mengersen, Newcastle Aust Robert Wolpert, Duke USA

2. Is ETS associated with lung cancer? 1981: First studies published (Hirayama, Garfinkel) 1986: First meta-analyses (Wald, NRC) 1992: EPA Review of 29 studies NHMRC Review (Australia) 1994: OSHA Review 2002: Boffetta review of 51 studies

3. Individual studies Exposed Unexposed Case Control N

4. A common measure: log odds ratio Cohort Studies Variety of ways to extract information about  amid nuisance parameters. Bayesian approach? Eg, Jeffrey’s prior Be(1/2,1/2) on p c|e,p c|e Then  LOR ~ N(  2 )

5. Info about  Select conditional prior distribution  (dp|  ) for p=(p c|e,p c|e ) and summarise evidence for  alone by L CHS (  )=  L CHS (p)  (dp|  ) Change of variables to LOR; independent Jeffrey’s prior Be(½,½) for p c|e and p c|e : L CHS (  )   -1 (e  -1) e n ce   2 F 1 (n e +1,n c +1;n+2;1-e  ) Then  LOR has posterior mean and variance   log(n ce n ce /n ce n ce );  2 =1/n ce +1/n ce +1/n ce +1/n ce And if the contingency table is ‘not too extreme’,  LOR ~ Gaussian (approx).

6. A common measure: log odds ratio Case Control Studies Only have indirect evidence about p c|e, p c|e If we know p c and p e in target population, then p c|e =p e|c p c / p e and similarly for p c|e Also if the outcome is ‘rare’, the relative risk and the odds ratio are approximately equal, so which does not depend on collateral quantities

8. Exchangeable combination of evidence Simple pooling: same cond’l exposure probs Cohort studies Case control studies  LOR = 0.70 ± 0.12  LOR = 0.17 ± 0.05 Fixed effects: same  i  for each study i  I  LOR = 0.21 ± 0.05 (RR=1.23) Random effects: same overall   LOR = 0.28 ± 0.20 (RR=1.32) Boffetta (2002): Overall RR 1.25 (1.15-1.37)

9. Exchangeable hierarchical models Model variation in study-specific parameters p i explicitly Model variation in study-specific parameters p i explicitly Exchangeable implies cond’ly independent random vectors given the hyperparameter  Exchangeable implies cond’ly independent random vectors given the hyperparameter  1. Write  as a function of  2. Then the joint posterior for ,  and all p i can be factored as  (d  )  (d  |  ) 3. The marginal likelihood is thus L EHM =  [  i  L i (p i )  (dp i |  ) ]  (d  |  )

10. Reduces to RE for  =( , ,  ) and normal logistic conditional distributions  (dp i |  ) RE for  =( , ,  ) and normal logistic conditional distributions  (dp i |  ) FE for  =  and beta conditional distributions  (dp i c|e |  ),  (dp i e|c |  ) FE for  =  and beta conditional distributions  (dp i c|e |  ),  (dp i e|c |  ) Simple pooling for  =(p ce,p ce,p ce,p ce ) with unit point masses  (dp i |  ) at p i = . Simple pooling for  =(p ce,p ce,p ce,p ce ) with unit point masses  (dp i |  ) at p i = . Exchangeable hierarchical model likelihood

11. Studies are exchangeable within known groups but the G groups may differ systematically among themselves  ~  (d  );  ~  (d  |  );  g ~  (d  g |  );  i ~  (d  i |  g ) Normal priors:  ~N(0,  2 ),  g ~N( ,  g 2 ),  i ~N(  g,  i 2 ) Partially exchangeable hierarchical model

12. 1992 EPA results Country Gp All Studies Tiers 1-2 Greece 2.01 (1.42,2.84) 1.92 (1.13,3.23) Hong Kong 1.48 (1.21,1.81) 1.61 (1.25,2.07) Japan 1.41 (1.18,1.69) 1.39 (1.16,1.66) USA 1.19 (1.04,1.35) 1.23 (1.04,1.42) W. Europe 1.17 (0.84,1.62) 1.17 (0.85,1.64) China 0.95 (0.81,1.12)

13. Synthesizing heterogeneous evidence 1. Threshold exclusion 2. Weighted likelihood functions 3. Block mixtures 4. Mixtures 5. Hierarchical models 6. Systematic adjustment of likelihoods

14. Adjusting the likelihood Parametric adjustment: L i Adj (  ) = L i (  ( ,  i ) eg, Shift in binomial probability parameter  by setting  ( ,  i )=  +  i Parametric adjustment: L i Adj (  ) = L i (  ( ,  i ) eg, Shift in binomial probability parameter  by setting  ( ,  i )=  +  i Uncertain adjustment:  i is uncertain so has (informative) prior  i  (d  i |  ), so L i Adj (  ) =  L i (  ( ,  i )  i  (d  i |  ) Uncertain adjustment:  i is uncertain so has (informative) prior  i  (d  i |  ), so L i Adj (  ) =  L i (  ( ,  i )  i  (d  i |  )

15. Adjustment in the ETS studies True fraction p i jkl of ith population with case status i, exposure status j, elibility status k [eg, p i ces is true fraction of cases, exposed, smokers] True fraction p i jkl of ith population with case status i, exposure status j, elibility status k [eg, p i ces is true fraction of cases, exposed, smokers] True classification probabilities  i =(  i ce,  i ce,  i ce,  i ce ) True classification probabilities  i =(  i ce,  i ce,  i ce,  i ce ) Apparent classification probabilities (q i ce,q i ce,q i ce,q i ce ) Apparent classification probabilities (q i ce,q i ce,q i ce,q i ce )

16. Eligibility violation Change of notation: c,C; e,E; s,S Suppress superscript i for each study Change of notation: c,C; e,E; s,S Suppress superscript i for each study q ceS =  S|ces p ces +  ceS|ceS p ceS + q ceS =  S|ces p ces +  ceS|ceS p ceS +  e|cES p cES +  c|CeS p CeS  e|cES p cES +  c|CeS p CeS (Similarly for q cES, q CeS, q CES ) (Similarly for q cES, q CeS, q CES ) So we want (p ces, p cEs, p Ces, p CEs ) So we want (p ces, p cEs, p Ces, p CEs )

17. Gathering the evidence EPA Review gives us p s for each study EPA Review gives us p s for each study Get p e|s from p e|S and K=p es p ES / p eS p Es (  3, Lee) Get p e|s from p e|S and K=p es p ES / p eS p Es (  3, Lee) Take p c|es = p c|Es = R s p c|S (R s is RR lung cancer among active smokers) Write p c =p c|s p s + p c|s p S = p c|S (p s R s +p S ) Thus p c|s = p c R s /p s R s +1-p s ) and p c|s = p c /(p s R s +1-p s ) Take p c|es = p c|Es = R s p c|S (R s is RR lung cancer among active smokers) Write p c =p c|s p s + p c|s p S = p c|S (p s R s +p S ) Thus p c|s = p c R s /p s R s +1-p s ) and p c|s = p c /(p s R s +1-p s ) This gives us (p i s,p i e|s,p i c|es,p i c|Es ) so we can find all four required probabilities for each study This gives us (p i s,p i e|s,p i c|es,p i c|Es ) so we can find all four required probabilities for each study

18. Still on eligibility violation: gathering the evidence We still need p `S’|s Lee, EPA: ~ 5% of eversmokers deny smoking. Mixed evidence of dependene on case status; we ignore this. Little evidence of dependence on exposure status; we ignore this. We still need p `S’|s Lee, EPA: ~ 5% of eversmokers deny smoking. Mixed evidence of dependene on case status; we ignore this. Little evidence of dependence on exposure status; we ignore this. EPA assigns ‘penalty points’ A i ranging from –0.5 (bonus) to +1.0 for each study’s control of this bias. EPA assigns ‘penalty points’ A i ranging from –0.5 (bonus) to +1.0 for each study’s control of this bias. So we take  i S|ces =  i S|cEs =  i S|Ces =  i S|CEs = 0.05 2 A i So we take  i S|ces =  i S|cEs =  i S|Ces =  i S|CEs = 0.05 2 A i

19. Misclassification of exposure We want p `e’|E and p `E’|e We want p `e’|E and p `E’|e Some studies report the other probs eg p E|`e’ so use Bayes theorem to invert Some studies report the other probs eg p E|`e’ so use Bayes theorem to invert Friedman’83: 47% of currently nonsmoking wives have <1 hr/day exposure at home. 40-50% women with nonsmoking spouses have significant ETS exposure outside the home. Lee’92: ‘not a major issue’ Jarvis et al’01: good surrogate Friedman’83: 47% of currently nonsmoking wives have <1 hr/day exposure at home. 40-50% women with nonsmoking spouses have significant ETS exposure outside the home. Lee’92: ‘not a major issue’ Jarvis et al’01: good surrogate We take p E|`e’ = 0.25, p e|`e’ =0.10 We take p E|`e’ = 0.25, p e|`e’ =0.10

20. Misclassification of lung cancer We need p `c’|c, p `C’|c We need p `c’|c, p `C’|c No evidence that these differ w.r.t. ETS exposure among nonsmokers No evidence that these differ w.r.t. ETS exposure among nonsmokers Lee: 30-40% lung cancers seen at autopsy are missed clinically Lee: 30-40% lung cancers seen at autopsy are missed clinically Thus we take p `c’|c =0.35. Thus we take p `c’|c =0.35. EPA: ‘penalty points’ B i from –0.5 to +2.5 for each study’s control of this bias. EPA: ‘penalty points’ B i from –0.5 to +2.5 for each study’s control of this bias. So we take So we take  i E|ceS =  i E|CeS =.19 2 B i - 2.5  i e|CES =  i e|CES =.14 2 B i- 2.5  i E|ceS =  i E|CeS =.19 2 B i - 2.5  i e|CES =  i e|CES =.14 2 B i- 2.5

21. Misclassification of exposure (cont.) Lee: 22.5% report exposure: p `e’ = 0.225 EPA: rates from 15% to 87% in studies Lee: 22.5% report exposure: p `e’ = 0.225 EPA: rates from 15% to 87% in studies We take p `e’ = 0.36 We take p `e’ = 0.36 Thus we calculate p `E’|e = 0.19, p `e’|E =0.14 Thus we calculate p `E’|e = 0.19, p `e’|E =0.14 Reduce by fraction C i of histologically verified cases, given by the EPA Reduce by fraction C i of histologically verified cases, given by the EPA So we take So we take  i c|CeS =  i c|CES = 0  i C|ceS =  i C|cES =.35 (1-C i )  i c|CeS =  i c|CES = 0  i C|ceS =  i C|cES =.35 (1-C i )

22. n ce, n cE, n Ce, n CE Observed data q ce, q cE, q Ce, q CE Apparent classification probabilities p c|e, p c|E, p C|e, p C|E Adjusted conditional ‘true’ probabilities  ii Study-specific LOR

23. Hierarchical prior distribution  i can be recovered from  i c =  i ce +  i cE,  i e =  i ce +  i Ce and  i LOR so we construct a joint distribution for  from that of  c,  e, . log(  c /  e ) ~ N(  g, 0.5) log(  g ) ~ N(  25/105, 0.5) low precision, little prior opinion on nonsmokers’ cancer rates log(  c /  e ) ~ N(  g, 0.5) log(  g ) ~ N(  25/105, 0.5) low precision, little prior opinion on nonsmokers’ cancer rates log(  i e /  i E )~N(  e =log(.36/.64)=-0.57,0.84 2 ) log(  i e /  i E )~N(  e =log(.36/.64)=-0.57,0.84 2 ) p `e’  0.36; reported apparent exposure rates 15%-87%; calculate variance so that P(0.1<qie<0.75)  0.90 p `e’  0.36; reported apparent exposure rates 15%-87%; calculate variance so that P(0.1<qie<0.75)  0.90

24. Results: Partially exchangeable model

25. Shrinkage effect: exchangeable model

26. Comparing meta-analysis models

27. Effect of adjustment

29. A tale of two studies Tier 4 case control study (Chan, Hong Kong) - evidence about  e but not  c Tier 2 cohort study (Hirayama, Japan) - evidence about  c but not  e

30. A tale of two studies

31. Conclusions Flexible meta-analysis method that directly adjusts the likelihoods Flexible meta-analysis method that directly adjusts the likelihoods Requires specific, explicit account of factors for which adjustment is made Requires specific, explicit account of factors for which adjustment is made Allows quantification and introspection about the impact of quality issues Allows quantification and introspection about the impact of quality issues Allows detailed interpretation Allows detailed interpretation