Presentation is loading. Please wait.

Presentation is loading. Please wait.

Verbal Autopsy Modules in Surveys _______________________________ Henry Kalter, MD, MPH Johns Hopkins Bloomberg School of Public Health Baltimore, MD,

Similar presentations


Presentation on theme: "Verbal Autopsy Modules in Surveys _______________________________ Henry Kalter, MD, MPH Johns Hopkins Bloomberg School of Public Health Baltimore, MD,"— Presentation transcript:

1 Verbal Autopsy Modules in Surveys _______________________________ Henry Kalter, MD, MPH Johns Hopkins Bloomberg School of Public Health Baltimore, MD, USA

2 Sneak Preview Some available VA questionnaires Some available VA questionnaires Use of VA in national surveys (DHS) Use of VA in national surveys (DHS) Misclassification error and possible solutions Misclassification error and possible solutions

3 Available VA tools and ongoing developments WHO standard VA for infants and children (1999) WHO standard VA for infants and children (1999) Developed by WHO/JHSPH/LSHTM Tanzania Adult M&M Project (AMMP) Tanzania Adult M&M Project (AMMP) INDEPTH Network standardized VA (2003) INDEPTH Network standardized VA (2003) Built on WHO infant/child and AMMP adult formats SAVVY VA for neonates, children and adults SAVVY VA for neonates, children and adults Developed by Measure Evaluation in collaboration with HMN for use in a nationally representative sample or selected sentinel area Also can be used in surveys or censuses laims better ascertainment of births and deaths than single surveys Used by India SRS, which claims better ascertainment of births and deaths than single surveys

4 Available VA tools and ongoing developments WHO consensus group VA for neonates, children and adults (2003–2007) WHO consensus group VA for neonates, children and adults (2003–2007) Sponsored by HMN: to replace SAVVY VA tool and be part of HMNs Stepping Stones resource kit for strengthening national vital statistics systems Modules: birth-27 days, 28 days-14 years, 15+ years Ongoing Harvard/JHBSPH/Queensland neonatal, child and adult VA validation study (Gates GC13) Ongoing Harvard/JHBSPH/Queensland neonatal, child and adult VA validation study (Gates GC13) Tanzania, Philippines, India (2 sites) Modules: birth-27 days, 28 days-11 years, 12+ years Will compare results of three analytic methods Individual causes, by algorithms and physician readers All causes of death at once, by symptom profiles

5 National surveys that use VA MACROs DHS (16/187 surveys, 1987–2007) MACROs DHS (16/187 surveys, 1987–2007) Stillbirths, child (NN, 1-11 mo, 12-59 mo), maternal deaths No non-maternal adult deaths UNICEFs MICS (limited COD information) UNICEFs MICS (limited COD information) AIDS: anyone aged 18-59 years who died in past 12 months and was seriously ill for 3/12 months before death MM: sisterhood method + death during pregnancy, childbirth or within 6 weeks after the end of pregnancy Other national health surveys, e.g., Turkey 2003 Other national health surveys, e.g., Turkey 2003

6 CountryDHS with VASubsequent DHS without VA Morocco19871992, 2003-04 Egypt19881992, 1995, 2000, 2005 Cameroon19911998, 2004 Namibia19922000, 2006 Bolivia19941998, 2003 CAR1994-95-- Haiti1994-952000, 2005 Chad1996-972004 Nigeria19992003 Bangladesh2004 2007 (but 1993-94 & 1996-97 w/VA: showed declines in most causes) Cambodia2005-- Honduras2005-- Nepal2006-- Pakistan2006-- Angola2006-- Uganda2007--

7 CountryModules Reference periodAnalysis Morocco <6 years: identified death but not cause-- Egypt <5 years: accident, 8 Sxs, 2 DxsChild: 5 years NN, 1-11 mo, 12-59 mo: Sxs (sum >100%) Cameroon <5 years: accident or illness type, 9 SxsChild: 5 years NN, 1-59 mo: combine mothers opinion &/or algorithm, e.g., diarrhea=algorithm, malaria= mother or algorithm (sum>100%) Namibia NN: COD, 9 Sxs 1-59 mo: COD, 19 Sxs MAT: preg/deliver/6wks Child: 5 years MAT: sisterhood NN, 1-59 mo: non-specific algorithms (e.g., measles: age >4 mo + rash (sum>100%) MAT: direct (age & year of death) BoliviaMAT: preg/deliver/6wksMAT: sisterhoodMAT: direct Central African Republic NN: COD, 9 Sxs 1-35 mo: COD, 19 Sxs MAT: preg/deliver/6wks Child: 3 years MAT: sisterhood NN, 1-35 mo: combine mothers opinion &/or algorithm (sum>100%) MAT: direct Haiti NN: COD, 7 Sxs 1-59 mo: COD, 16 SxsChild: 5 years NN, 1-11 mo, 12-59 mo: combine mothers opinion &/or algorithm (sum>100%)

8 CountryModules Reference periodAnalysis Chad NN: COD, 9 Sxs 1-59 mo: COD, 19 Sxs MAT: preg/deliver/2mo Child: 5 years MAT: sisterhood NN, 12-59 mo: combine mothers opinion &/or algorithm (sum>100%) MAT: direct NigeriaMAT: preg/deliver/2moMAT: sisterhoodMAT: direct (but data quality prblm) Bangladesh NN, 1-59 mo: detailed format for each groupChild: 5 years NN, 1-11 mo, 12-59 mo: detailed algorithms, w/hierarchical assignment of cause(s) (sum=100%) CambodiaMAT: preg/deliver/2mo Child: 3 years MAT: sisterhood NN, 1-59 mo: mothers opinion & algorithm (separately); MAT: direct Honduras NN: COD, 31 Sxs 1-59 mo: COD, 28 SxsChild: 5 yearsNN, 1-11 mo, 12-59 mo: ? Nepal Stillbirth, NN, 1-59 mo: detailed format for each group MAT: preg/deliver/2moMAT: sisterhood Stillbirth, NN, 1-11 mo, 12-59 mo: detailed algorithms, w/hierarchical assignment of cause(s) + MD review of undetermined cases (sum=100%) MAT: direct Pakistan Stillbirth, NN, 1-59 mo: detailed module for each MAT: detailed VA Child: ? MAT: ? Child: ? MAT: MD review to determine if maternal, direct/indirect, cause & CS

9 CountryModules Reference periodAnalysis Angola NN, 1-59 mo: 3-page format for each groupChild: ? Uganda NN, 1-59 mo: 6-8-page format for each groupChild: ?

10 Verbal autopsy in DHS surveys Survey design issues (Deaths of children born in the last 3-5 years) Survey design issues (Deaths of children born in the last 3-5 years) 1-year maximum recall recommended for child deaths Variable recall depending on age (shorter for older children) Age and cause distributions distorted Disproportionately captures deaths of younger children This also distorts the all-ages cause distribution Nepal design does not distort: All child deaths in past 5 years

11 Verbal autopsy in DHS surveys Most use a sparse questionnaire Most use a sparse questionnaire No adult (non-maternal) deaths Based on validation studies, but perhaps too few items with insufficient detail 21-60% (high end for NN deaths) of cases with undetermined cause of death Bangladesh & Nepal VAs longer, based on standard formats Require re-visit to administer 1.1-3.4% (Bangladesh) and 4.5-11.4% (Nepal) of cases with undetermined cause of death by algorithm Consider developing a standard DHS VA questionnaire based on current best practices and whats practical

12 Verbal autopsy in DHS surveys Unusual methods for coding VA diagnoses Unusual methods for coding VA diagnoses Usual methods: physician readers or one algorithm/COD Most DHSs examine multiple algorithms for each cause Most DHSs combine maternal opinion with algorithms Unclear decision tree (when to use which method?) Diagnoses add to >100%, but usually unclear as to which children have >1 diagnosis Consider developing a standard DHS coding method based on current best practices (change may be on the horizon)

13 VA diagnosis misclassification ALRI (sensitivity/specificity) ALRI (sensitivity/specificity) Cough >3 days and difficult breathing >3 days Bangladesh: 64%/84% Uganda: 51%/68% Malaria (sensitivity/specificity) Malaria (sensitivity/specificity) Fever and convulsions or loss of consciousness Namibia: 45%/87% Uganda: 44%/77% Measles (sensitivity/specificity) Measles (sensitivity/specificity) Age >120 days, rash and fever >3 days, rash on face (Nam.) or rash anywhere except extremities (Phil.) Namibia: 83%/86% Uganda: 98%/93%

14 VA diagnosis misclassification Methods of dealing with misclassification Methods of dealing with misclassification Do nothing (usual method) False positives and false negatives may counter-balance each other to produce an accurate cause-specific mortality estimate, but this is uncertain Despite uncertainties, there is some evidence that VA can usefully measure changes in cause-specific mortality Improve VA performance Accommodate for misclassification Adjust for misclassification Go around misclassification

15 VA diagnosis misclassification – Improve VA performance Attempts to improve WHO standard neonatal VA Attempts to improve WHO standard neonatal VA Addition of stillbirth module Additional details on pregnancy and L&D complications New signs of NN illnesses strive for increased specificity Breathed immediately after birth (was able to breathe) Breathed immediately after birth (was able to breathe) Sucked normally during the first day (was after birth) Sucked normally during the first day (was after birth) Attempt to improve coding of VA diagnoses Attempt to improve coding of VA diagnoses Compare physician readers to algorithms Improvements may be elusive Improvements may be elusive Best sensitivity and specificity depend on disease prevalence Disease mix can affect specificity (and perhaps sensitivity)

16 VA diagnosis misclassification – Accommodate for misclassification

17

18 Sensitivity/Specificity that can Estimate Cause-Specific Mortality Fraction Within + 20% of the True Level 5% MF 70-100%/100% 10% MF 80-100%/100% 50-70%/95% 20% MF 80-100%/100% 60-100%/95% 50-80%/90% 50-60%/85% 30% MF 80-100%/100% 70-100%/95% 60-95%/90% 50-85%/85% 50-70%/80% 50-60%/75% 50%/70% 40% MF 80-100%/100% 80-100%/95% 70-100%/90% 60-95%/85% 50-90%/80% 50-80%/75% 50-70%/70% 50-60%/60%

19 VA diagnosis misclassification – Accommodate for misclassification Objectives Objectives Determine how different (cultural and disease) settings affect VA performance Identify algorithms with consistent and appropriate performance in similar settings Method: Conduct validation studies with standardized methods in multiple settings Method: Conduct validation studies with standardized methods in multiple settings Much of the apparent variability in VA performance may be due to inconsistent study methods Determine the effects of site characteristics on performance Different cultural settings Different disease mixes (e.g., with and without malaria) Malaria sites with different transmission intensities

20 VA diagnosis misclassification – Adjust for misclassification Back-calculate to adjust for misclassification Back-calculate to adjust for misclassification Uses sensitivity/specificity estimates of VA algorithms from hospital-based validation studies Very sensitive to inaccurate estimates caused by: Hospital-based study biases: Differences in hospital/community disease mix Medical exposure Cultural, SES, etc. differences Differences between validation study and survey sites Basis of the problem: composite nature of specificity vs. yes/no classification CSMF = (VA + Sp – 1) / (Sn + Sp – 1)

21 VA diagnosis misclassification – Back-calculate to adjust for misclassification Diarrhea-Specific True Fraction and Estimated Levels Determined by Verbal Autopsy and Back-Calculation (using the average sn/sp from the other two sites)

22 VA diagnosis misclassification – Go around misclassification Calculate disease probability from symptom profiles Calculate disease probability from symptom profiles P(S) = P(S|D) P(D) 2 K x 1 2 K x J J x 1 P(S|D) Symptom profileCOD_1COD_2COD_3P(S) 0000.090.080.04 0010.370.270.110.32 0100.140.120.040.11 0110.00 1000.120.290.380.27 1010.000.180.150.09 1100.100.060.000.07 1110.180.000.280.10 All profiles1.00 P(D)???

23 VA diagnosis misclassification – Go around misclassification Calculate disease probability from symptom profiles Calculate disease probability from symptom profiles Does not require VA algorithms, sensitivity/specificity estimates or physician readers Estimates mortality fractions of all CODs at once [P(D)] Eliminates biases caused by dichotomizing COD Uses P(S|D) estimates from hospital-based study Less sensitive to inaccurate estimates than VA algorithms: Symptom profiles can be manipulated at will to find differences in P(S|D) between diseases Does not require big differences in P(S|D)s Allows multiple P(S|D)s for each disease (vs. one yes/no algorithm) Still liable to bias due to inaccurate P(S|D) estimates P(S) = P(S|D) P(D) 2 K x 1 2 K x J J x 1

24 VA diagnosis misclassification – Go around misclassification Validation in Tanzania for adults (left graph), children (middle), and infants (right). In each graph, a direct estimate of cause-specific mortality is plotted horizontally by our verbal autopsy estimate plotted vertically (G. King, Y. Lu, 2006; data: Setel et al. 2006)

25 Summary and Conclusions – DHS often uses sub-optimal VA methods DHS often uses sub-optimal VA methods Sparse modules, no adult (non-maternal) module, problematic analytic method Sparse modules, no adult (non-maternal) module, problematic analytic method Convene a study group to improve modules and analytic methods based on current knowledge and practices Convene a study group to improve modules and analytic methods based on current knowledge and practices Ongoing research holds promise for improvements Ongoing research holds promise for improvements Identify algorithms with increased sensitivity/specificity Identify algorithms with increased sensitivity/specificity Gain better understanding of how cultural and diseases settings affect VA performance Gain better understanding of how cultural and diseases settings affect VA performance New (experimental) analytic method decreases bias in VA estimates due to disease misclassification New (experimental) analytic method decreases bias in VA estimates due to disease misclassification

26 THANK YOU


Download ppt "Verbal Autopsy Modules in Surveys _______________________________ Henry Kalter, MD, MPH Johns Hopkins Bloomberg School of Public Health Baltimore, MD,"

Similar presentations


Ads by Google