Download presentation
Presentation is loading. Please wait.
Published byAvice Stevenson Modified over 9 years ago
1
Design and conduct of evaluations of CVD control programs (part I) Gilles Paradis, MD, MSc, FRCPC Jennifer O’Loughlin, PhD McGill University Health Center Department of Epidemiology and Biostatistics McGill University
2
Outline Part I l Why evaluate? l What’s evaluation l Evaluate what? l Scope of evaluation l Methodological issues
3
Why evaluate? 1. Accountability Report on the attainment of objectives and use of limited resources 2. Improvement Treatment, program performance. 3. Advocacy Enhance programs, build consensus, support coalitions
4
Why evaluate? l Social responsibility beyond "Primum non nocere" l Many (well established) interventions have been subsequently shown to be useless or harmful l M.I.: Prolonged bed rest Magnesium Class I antiarrhythmics Ca ++ channel blockers l Prevention: carotene HRT (?)
5
What is evaluation? Process of systematic data collection or information gathering to shed light on some aspects of an action or intervention Respond to specific questions regarding a program "Who is being reached by…?" Support decision making "Which of two alternative strategies is more effective?"
6
What is evaluation? Enhance community participation "What are key community concerns?" Improve the understanding of mechanisms of action "How can I reach low SES populations with this program?" Support community mobilization "What do key stakeholders expect from a coalition?"
7
Evaluate what? l Primary prevention programs Reduce exposure to risk factors Decrease incidence l Secondary prevention Prevent progression among affected asymptomatic individuals (HBP, …) Screening, case-finding
8
l Individual practice Diagnostic, preventive, therapeutic l Organizational or community changes Structural (inputs, resources mobilized) Process (quality of services) Outcomes (attainment of objectives) l Tertiary prevention Decrease morbidity, mortality among symptomatic individuals Improve QOL, functioning
9
Scope of Evaluation Broad approaches 1 - Normative 2 - Evaluative research
10
Scope of Evaluation 1.1 - Quality of preventive care l GOAL: Compare practices to standards of excellence or criterias l EXA:Rules for use of resources l Who gets fasting lipoprotein profiles? l Who gets 24 hour BP monitoring? l Streptokinase or tpa? Criterias of quality preventive care l Management of HBP, type II diabetes l Management of pts with IHD l METHODS:Chart audits Surveys 1 - Normative
11
Scope of Evaluation 1.2 - Quality of programs l GOAL:Structural:Appropriate use of resources? Process:Target population attained? Program implemented as intended? Impact:Were objectives achieved? l EXA:HBP screening in worksites l Methods:Review of reports, existing databases Key informant interviews Surveys 1.3 - Evaluation of (public health) organizations l Structure, functioning, planning, etc. 1 - Normative
12
Scope of Evaluation l Efficacy l Effectiveness l Efficiency (cost-benefit, cost- effectiveness) l Quality of preventive care (decision analysis ) 2 - Evaluative research
13
1 - Specification of theoretical model 2 - Design 3 - Measures (what and how) 4 - Biases 5 - Analysis Methodological issues
14
1 - Theoretical model l Avoid “Black Box” phenomenon l Observe connecting processes between inputs and outputs l Key to understand and improve interventions l Describes how program produces the effect l Blueprint for selection of variables, guiding analysis, interpreting results Methodological issues
16
2 - Design l General model Initial state Subsequent state t 0 Intervention t 1 Effect of intervention, time or other? Initial state Subsequent state t 0 Intervention t 1 Initial state Subsequent state Methodological issues
18
2 - Design l Repeat cross-sectional surveys l Cohort l RCT l (Case-control) Methodological issues
19
2 - Design Cohort l Individual behavior change l Non-anonymous participation l Attrition related to behavior evaluated l Repeat testing, co-intervention l Maturation, aging l More long-term residents ll l 1- Repeat C / S l Community-wide prevalence l Anonymous ll ll ll l Less of a problem l Cross-contamination l 1- Methodological issues
21
2 - Design RCT l Unbiased allocation l Similar distribution of R.F. (known or unknown) to groups l Comparability of groups l Validity of statistical tests l Feasability, costs l Other options to minimize biases (matching, stratification, …) Methodological issues
23
3 - Measures 3.1 - What? l Mortality, morbidity l Q O L l Risk factors l Behaviors l Physical and social environments Proximal impact easier to measure than distal Methodological issues
24
3 - Measures 3.2 - How? Reliability Validity l Self-reported behaviors l Social desirability l Pre-testing instruments l Objective measures / Gold standard l Environmental measures(shelf space, no-smoking signs, …) l Surrogate reports from next of kin l Bogus measurements Methodological issues
25
4 - Biases “Distortion in the estimate of effect of an exposure” due to l Selection of subjects l How information is collected l Confounding Methodological issues
26
4 - Biases Community programs particularly prone to biases l Random allocation is rare l Limited # of clusters l Important differences between groups (absolute and secular trends) l Multiple co-interventions l Blinding is impossible Methodological issues
27
Solutions: l Matching: l # of pairs l # measurements Methodological issues
29
5 - Analysis Effects measured at the individual level but allocation and intervention are at the community level 1- High intra-class correlations Biased standard error at the individual level ( false - positive results) Standard error must be computed at the community level requires N adjustment for sampling procedures # data collection Methodological issues
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.