1. Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist MCHEPI Program Team Leader, CDC

2. Objectives of the presentation 1. Review validity and the EBPH process 2. Review EPI principles guiding evidence application as a MCH decision driver 3. Review on current examples based on programmatic information in MCH

3. Relevant questions based on the previous presentation Is there a need for EBPH? What do we call evidence? How do we assess evidence? How do we provide recommendations based on the best evidence? How do we implement an action plan? How do we evaluate impact?

4. Changes or new programs are based on recommendations A. Good evidence to support decisions B. Fair evidence to support decisions C. Poor evidence that does not provide direction to do or don’t do D. Fair evidence to support don’t do E. Good evidence to support don’t do

5. Let’s change the paradigm… MCH covers maternal, child, health Other close fields (injury, infections, death, behaviors, etc” MCH belongs to public health We are temporary MCH practitioners Let’s talk about public health

6. Basic concepts about EBPH WORKING DEFINITION (PH): Promoting healthy people in healthy communities

7. Public Health Prevents epidemics and spread of disease Protects against environmental hazards Prevents injuries Promotes and encourages healthy behaviors Responds to disasters and assists communities in recovery Assures the quality and access to services APHA

8. Services Servicesdata-based

9. Public Health data-action loop: Case recollection Population information Risk factor data (PRAMS) ANALYSIS Programs and policies RATES AND PROPORTIONS 1.absolute risk 2.population “mapping” 3.tendencies Complex Analyses Cause risk factors costs morbidity Program evaluation INFORMATION Inside the box outside the box

10. Sources of evidence in PH “soft” information: review processes, personal information, “gut” feelings “adequate” information: routinely collected information, case review programs, passive systems “strong” information: active surveillance, some clinical studies “very strong”: randomized clinical trials

11. 2. EPI components in public health problems Examples Referral distance is associated to mortality Implications of fetal intervention (clinical and public health) to pediatrics and PH

12. p:0.38 r:0.48 Example 1

13. So the main issue here is Validity Validtruth Unbiased (bias needs to be avoided) True and tolerable level of chance (chance needs to be measured) Took into account the presence of characteristics that change the relationship of interest (controlled for confounding)

14. LBW - SGA LAPRAMS data 1998-1999 Population at risk LA 1998-1999: 130,294 pregnancies Smoking OR: 3.5 Wt-Gain OR: 3 Counseling OR: 1.7 Prevalence: LBW: 7% (9,120) VLBW: 2% (2,605) SGA: 15% (19,544) AFp: LBW: 9% (820)(+?) VLBW: 2% (52) (+?) SGA: 2% (390)(+?)

15. Why the concern? Knowledge is rapidly expanding The use of “EB decision-making” is common Large amount of published (scientific) literature Larger amounts of (unused) stored data Lack of guidelines for the EBPH process Large degree of uncertainty about change

16. Critical appraisal of the evidence The most difficult step (for PH officials) Needs technical evaluation (epi) Needs “special” skills Less than 10% of active personnel has the skills required. Less than 1% has a method. Time-consuming step Source of “biased” decisions

17. Critical appraisal of the evidence Needs assessment, surveillance (diagnosis): –Prevalence –Sensitivity –Specificity –PPV / NPV –LRP / LRN –Pretest odds / postT odds –post-test probability –Receiver-operator curve Prognosis: –time variables –survival curves –time series –prognosis estimates –precision –relationship with screening and diagnosis

18. Critical appraisal of the evidence Intervention: –Blindness (evaluation) –Randomization (evaluation) –Control event rate –Relative Risk reduction –Absolute Risk reduction –Number needed to treat Risk: –Odds ratio –Risk ratio –Incidence –Prevalence –Exposure –inference –probability distribution

19. Critical appraisal of the evidence Other more “general” terms: –Bias –confounders –probability distribution and chance –p value –confidence intervals –logistic regression, multivariate analysis –univariate analysis

20. Surveillance Systems Epidemiological Studies Prevention Programs  Risk factors  Protective factors  Public concerns  Prevention strategies  Public policy  Education  Prevalence rates  Registry of cases for study or referral  Monitor prevention Example # 3:

21. Birth Certificates Predictive Value Positive 76% Sensitivity 28% Hospital Discharge Data Predictive Value Positive 85-95% Sensitivity 70-90% Example # 3: Evaluation of Data Studies

22. Conflict in PH To do things right To do the right things (right) DRIVING FORCE: best evidence for the best practice PROBLEMS: How is this done? How to do it always? How to do it always the same?

23. Best Evidence Makes sense (relevant) Unbiased Available Statistically significant (chance) Significant to Public Health Leads to correct decisions

24. Best Evidence Unbiased: well designed (bias cannot be measured) random error average: 10 (very precise)average: 4

25. Best Evidence Unbiased: well designed (bias cannot be measured) Systematic error random erroraverage: 4

26. Best Evidence Available: Published (strength of evidence) Surveillance systems Routinely collected information Peer information Smart opinion Other

27. Evidence I - Evidence from RCT II-1 - Well designed non-randomized trials II-2 - Cohort, Case Control analysis II-3 - Comparisons of places, time, interventions, better more than 1 center III - Opinion of authorities, descriptive studies, expert peer groups or committees

28. Best Evidence Statistically significant: Chance (randomness): p values Confidence intervals 95% p no association no difference Measurable!

29. Evidence Statistical significance Meaningful to Public Health BOTH goodbestfair We have been taught to accept statistical significance. If large samples (as in many cases), we are bound to have it, even if it is not meaningful.

30. Best Evidence Leads to correct decisions: MSAFP vs. Folic Acid for NTDs UK and Northern Ireland: one of the highest prevalences in the world Prenatal screening for more than 35 years Prenatal termination of the affected pregnancies MSAFP

31. Folic Acid Fortification

32. 4. Change PH practices Public Health is about: Research Advocacy Community Services Education Wisely invest as little money as possible to make the biggest and better change possible

33. Changes are based on recommendations A. Good evidence to support decisions B. Fair evidence to support decisions C. Poor evidence that does not provide direction to do or don’t do D. Fair evidence to support don’t do E. Good evidence to support don’t do

34. Evidence I - Evidence from RCT II-1 - Well designed non-randomized trials II-2 - Cohort, Case Control analysis II-3 - Comparisons of places, time, interventions, better more than 1 center III - Opinion of authorities, descriptive studies, expert peer groups or committees

35. Framework for evaluation Do I want (have) to evaluate the study? Outline the study Is the study believable? What is the Public Health relevant finding? –Are the variables of interest included? –Are findings explainable by chance, bias or confounders? –Is the finding believable within our knowledge? Will the study help me with my population? –Is my population similar? –Is my problem similar? –Will the findings benefit my programs and policies?

36. Inside the Box Workshop Summary We have: 1.Reviewed concept and situation of “evidence based” practice of public health 2.Reviewed relevant basic EPI concepts 3.Reviewed EPI aspects of public health 4.Provided element to build a framework to evaluate evidence 5.Evaluated the role of evidence in PH decisions