1. Adjusting for non-ignorable non-response: Application to Gulf War Study. Angela Wood 1, Ian White 1 and Matthew Hotopf 2 1 MRC Biostatistics Unit, Cambridge, UK. 2 GKT School of Medicine & Institute of Psychiatry, London, UK

2. Non-ignorable non-response in surveys Recruits

3. Non-ignorable non-response in surveys Responders Non- responders

4. Non-ignorable non-response in surveys Non-ignorable non-response: non-response relates to unrecorded characteristics of interest. Bias is likely to occur if non-responders are ignored. Responders Non- responders

5. Useful information Reasons for non-response. Proxy outcomes. Intensive follow-up on sample. Number of failed contact attempts.

6. The problem Questionnaires sent to N participants n 1 responded N-n 1 did not respond

7. The problem Questionnaires sent to N participants n 1 responded N-n 1 did not respond Questionnaires sent to non-responders N-(n 1 +n 2 ) did not respond n 2 responded Mailing Wave 1 Mailing Wave 2

8. The problem Questionnaires sent to N participants n 1 responded N-n 1 did not respond Questionnaires sent to non-responders N-(n 1 +n 2 ) did not respond n 3 responded n 2 responded Questionnaires sent to non-responders N-(n 1 +n 2 +n 3 ) not responded Mailing Wave 1 Mailing Wave 2 Mailing Wave 3

9. The problem Questionnaires sent to N participants n 1 responded N-n 1 did not respond Questionnaires sent to non-responders N-(n 1 +n 2 ) did not respond n 3 responded n 2 responded Questionnaires sent to non-responders N-(n 1 +n 2 +n 3 ) not responded Mailing Wave 1 Mailing Wave 2 Mailing Wave 3 Data are only observed for responders

10. Example: Fatigue Mailing wave N=4822 Non-caseCase 11054 (51.5%) 991 (48.5%) 2381 (56.4%) 295 (43.6%) 3251 (56.8%) 191 (43.2%) Non-responders1659

11. Notation i=1,…,N participants. m waves. n 1, n 2, …,n m responders at waves 1, 2,…, m respectively. N-(n 1 + n 2 + … + n m ) non-responders. Outcome of interest Y i for individual i. Confounders X i for individual i. Y i and X i are only known for responders.

12. Response Model p i1 = P(i responds at 1 st attempt); p i2 = P(i responds at 2 nd attempt | i not responded at 1 st attempt); p i3 = P(i responds at 3 th attempt | i not responded at 1 st or 2 nd attempt): logit(p ij ) = j + Y i T (i=1,…, N; j=1,…,3).

13. Response Model p i1 = P(i responds at 1 st attempt); p i2 = P(i responds at 2 nd attempt | i not responded at 1 st attempt); p i3 = P(i responds at 3 th attempt | i not responded at 1 st or 2 nd attempt): logit(p ij ) = j + Y i T (i=1,…, N; j=1,…,3). The effect of outcome on the probability of response is the same at all waves – strong assumption.

14. How does it work? Mailing wave Y=1Y=0 1200400 2100300 Non- responders 120 ?

15. How does it work? Mailing wave Y=1Y=0 1200400 2100300 Non- responders 120 60 OR = 0.33

16. How does it work? Mailing wave Y=1Y=0 1200400 2100300 Non- responders 120 60 OR = 0.33 OR = 1.13

17. How does it work? Mailing wave Y=1Y=0 1200400 2100300 Non- responders 120 20 100 OR = 1.67

18. Estimation procedure Modified conditional likelihood method (Alho 1990, Biometrika) –Conditional likelihood, product over responders: ij P(i responds at wave j | Y i ). –Use additional estimating equations which include information about number of non-responders.

19. Weighted Outcome model Unconditional response probabilities i1 p i1 i2 p i2 (1-p i1 ) i3 p i3 (1-p i1 )(1-p i2 )(1-p i3 ) The probability of responding = ( i1 + i2 + i3 ) Use inverse response probabilities ( i1 + i2 + i3 ) -1 to weight the observed data Y i. Easily extends to multivariate case.

20. Incorporating uncertainty in the weights (1) Bootstrapping (2) Multiple weights –Generate K sets of weights from K non-parametric bootstrap samples. –Perform a weighted analysis for each set of weights. –Pool the results together, rather like the multiple imputation technique. –The sets of weights only need to be derived once and then can conveniently be used in any subsequent analyses.

21. Application: Gulf War Survey Various symptoms in military personnel in the Persian Gulf War 1990-91 have caused international speculation and concern. Cross-sectional postal survey on UK servicemen. 3 Mailing attempts

22. Application: Gulf War Survey Compare various health problems between –Gulf Cohort: Persian Gulf War veterans –Bosnia Cohort: Servicemen deployed to the Bosnia conflict –Era Cohort: Those serving during the Gulf war but not deployed there. Outcome of interest: fatigue Confounders: –age, marital status, rank, education, employment, whether still serving or discharged, smoking, alcohol.

23. Response waves Gulf N=4822 Wave 1responded2099 (43.5%) Not responded2723 Wave 2Responded701 (14.5%) Not responded2022 Wave 3Responded483 (10.0%) Non responded1539 (31.9%)

24. Response waves Gulf N=4822Bosnia N=2983Era N=3905 Wave 1responded2099 (43.5%)995 (33.4%)1417 (36.3%) Not responded272319882488 Wave 2Responded701 (14.5%)431 (14.4%)552 (14.1%) Not responded202215571936 Wave 3Responded483 (10.0%)389 (13.0%)436 (11.1%) Non responded1539 (31.9%)1168 (39.2%)1500 (38.4%)

25. Fatigue case Mailing wave Gulf CohortBosnia CohortEra Cohort Non-caseCaseNon-caseCaseNon-caseCase 11054 (51.5%) 991 (48.5%) 706 (72.7%) 265 (27.3%) 1092 (78.7%) 295 (21.3%) 2381 (56.4%) 295 (43.6%) 309 (75.2%) 102 (24.8%) 442 (81.3%) 102 (18.7%) 3251 (56.8%) 191 (43.2%) 278 (77.4%) 81 (22.6%) 326 (78.9%) 87 (21.1%) Non- responders 165912421561

26. Univariate Models Response model for EACH cohort logit(p ij ) = j + fatigue* Outcome model compares fatigue across cohorts using inverse response probability weights.

27. Estimated Fatigue cases in Gulf cohort Mailing wave Gulf Cohort fatigue non-case fatigue case 11054 (51.5%) 1056 991 (48.5%) 986 2381 (56.4%) 374 295 (43.6%) 306 3251 (56.8%) 254 191 (43.2%) 185 Non-responders1659 1158 (69.9%) 502 (30.1%) Weights = 1.7 (non-case), 1.3 (case), chi-squared = 0.77

28. Results Estimated percentage of fatigue case (se) OR (95% CI) GulfBosniaeraG vs BG vs E Responders only46.725.720.62.5 (2.2-2.9) 3.4 (3.0-3.8) Adjusting for non-responders Without adjusting for uncertainty in the weights 41.021.119.52.6 (2.2-3.0) 2.9 (2.5-3.2) Adjusting for uncertainty in weights using 1000 bootstrap samples 41.0 (2.0)21.1 (2.2)19.5 (2.5)2.6 (1.9-3.5) 2.9 (2.0-4.1) Adjusting for uncertainty in weights using multiple weights (k=10) 41.8 (2.5)21.6 (2.4)19.3 (2.3)2.6 (2.0-3.4) 3.0 (2.1-4.3)

29. Multivariate Response and Outcome models Response model for EACH cohort logit(p ij ) = j + Z i T –where Z i may include outcome Y i and other characteristics collected. Outcome model adjusting for confounders. –Inverse response probability weights. –Multiple weights K=10.

30. The multivariate response model Gulf cohort only SE( ) OutcomeFatigue0.770.28 Military statusStill in military servicebaseline Discharged-0.850.23 RankOfficer1.030.32 otherbaseline Also adjusted for employment, education, age, smoking, alcohol intake, marital status.

31. Fatigue Frequency (95% CI)Adjusted Odds ratios (95% CI) GulfBosniaEraG vs BG vs E Responders only 46.9%25.8%20.5%2.2 (1.9-2.6) 3.6 (3.2-4.2) Adjusting for non-response Multivariate response model 38.7% (34.1-43.4) 22.2% (17.9-26.5) 19.4% (12.6-26.2) 2.2 (1.6-3.0) 3.2 (2.3-4.5)

32. Post traumatic stress reaction Frequency (95% CI)Adjusted Odds ratios (95% CI) GulfBosniaEraG vs BG vs E Responders only 13.2 %4.7%4.1%2.6 (1.9-3.4) 3.8 (2.8-4.9) Adjusting for non-response Multivariate response model 10.5% (8.0-12.8) 3.8% (2.1-5.4) 3.7% (1.9-5.6) 3.0 (1.9-4.6) 3.9 (2.5-6.1)

33. Conclusions Participants responding earlier had more symptoms than those responding later or not at all, particularly amongst Gulf veterans. Observed excess of symptoms in Gulf veterans reduced but not eliminated. Standard errors were increased when allowing uncertainty in response probabilities.

34. Discussion (1) Relax modeling assumptions –a common effect of covariates/outcomes on the probability of response across all waves. –Not possible: logit(p ij ) = j + Z i T j –Possible: logit(p ij ) = j + Z i T ( 0 + 1 j) Never responders?

35. Discussion (2) Estimation procedures –Considered full likelihood methods EM algorithm Bayesian approach, WinBUGS –Produce similar results –No need to use multiple weights (all-in-one methods). Extension to dealing with item-non-responders and refusals –little change in results.