Evaluation of Health Promotion CS 652 Sarah N. Keller.
Published byModified over 6 years ago
Presentation on theme: "Evaluation of Health Promotion CS 652 Sarah N. Keller."— Presentation transcript:
Evaluation of Health Promotion CS 652 Sarah N. Keller
Overview Terms Methods Approaches Future Research
Terms Formative evaluation Needs assessment; baseline data collection; audience research; results feed into message/campaign design Process evaluation Monitoring; tracking progress & activities; short- term evaluation; in-process impact Impact evaluation Outcomes evaluation; end-user changes in KAP
Methods Focus groups Develop clearer understanding of perceptions, values, tastes, psychosocial factors N = 8-12 Data = qualitative data recorded & transcribed Procedure = group interview, moderated by a single facilitator &/or note-taker Analysis = qualitative analysis for key phrases, research questions, language, etc. Purpose = formative research; message design; pre- testing; monitoring research; audience reactions & feedback
Methods Surveys Written, telephone, in-person, or online Questions = closed-end or open-end Data = quantitative & qualitative data N = 200 (informal market sample) – 1,500 (minimum for national representation of any subgroup) Analysis – Bivariate correlations; cross-sectional; multivariate regression Purposes = compare baseline to outcome indicators; pre-/post-test comparison; descriptive information; correlations of two or more variables with each other
Methods In-depth Interviews Process = face-to-face; one-on-one; structured or unstructured Data = qualitative, recorded & transcribed Analysis = qualitative Purpose = to uncover personal, sensitive or in-depth information; psychosocial factors, anecdotes, personal histories
Online Methods Online Surveys Samples = Convenience sample of who comes to a site; Internet users at large, or of a specific demographic of health risk category Experimental design = can direct one group to look at a Web site, and not others (treatment and control) Process = can create online forms attached to Web sites that automatically spit data into a database Data = quantitative or qualitative Analysis = same as with other surveys
Online Methods Usability studies Purpose = Find out how users search for a topic or interface with a site; to revise or pre-test a Web site. Process = Convene a group of users in a PC room & ask them to conduct “scavenger hunts” Data = audio, video, observation notes, log data, etc.
Online Methods Online Focus Groups Can be conducted in chat rooms, email, or by recruiting volunteers via a Web site form Purpose = Assess users’ opinions of a topic, Web site, or health area; etc. Data = Qualitative Limits = Invasion of privacy, reliability of self-reports, and informed consent process
Approaches: CPP Evaluating Community Planning Define the evaluation goals Document that CPP has taken place Document whether program goals are being met Identify strengths & weaknesses Holtgrave, D.R., et al. (1996). Methodological issues in evaluating HIV Prevention Community Planning. Public Health Reports, 111 (suppl. 4).
Approaches: CPP Draw a Logic Model of CPP Process Logic model = Graphic representation of an intervention; purpose is to show logical connections between conditions, activities & outcomes. Conditions/problems = What CPP designed to change Activities = Components of CPP undertaken Outcomes = Short-term program goals
Approaches: CPP Develop Evaluation Plan Goal #1 (Process objective) – To document that the activities have taken place as planned. Goal #2 (Outcomes) – To document whether short- term program goals are being met. Specific, measurable outcomes derived from general goals. Goal #3 – (Lessons learned) Identify strengths & weaknesses (e.g., an objective might not be reasonable, or more technical assistance may be required).
Approaches: CPP Translating plan into action Core objectives – National assessment of community planning does not require data on all process & outcome variables that might be useful at a local level. Budgetary reporting Case studies
Example: Hospital Marketing Goals: Determine attributes that physicians consider important in deciding to join hospitals Determine which factors most affect physicians’ evaluation of hospital quality Cronin, JJ., Joyce, M.L. (1987). Medical staff perceptions: Implications for the design of hospital marketing programs. JHCM, 7 (3).
Example: Hospital Marketing Lit Review Shows physicians are important influential agents in determining consumer hospital selection Methods Identify research variables Hospital performance factors Attributes that define hospital’s “offering” to physicians
Approaches: Hospitals Methods Focus group (n=8) physicians to discuss “What actually defines what a hospital has to offer to you in order to attract you to use facility?” AND, “What determines your evaluation of a particular hospital?” Analysis Attributes categorized into 11 indicators Evaluative dimensions categorized into 4 groups Used for survey design
Approaches: Hospitals Methods Mail survey (n=169) – of physicians responded Setting – 4 hospitals in Florida 3-part instrument Questions to capture main Research Q’s Demographics
Approaches: Hospitals Instrument Respondents asked to recall decision to join hospital and rate importance of each of 11 attributes on a scale from 1-5. Respondents asked to indicate perception of performance of area’s 4 hospitals based on 11 attributes Respondents asked to evaluate 4 hospitals using 4 evaluative dimensions from focus groups
Example: Hospitals Quantitative data analysis Means – Mean response for each of 11 attributes Correlation – Relationship b/t physicians’ perception of hospitals performance (using 11-attributes as IVs) & their evaluation on dimensions (4 dimensions as DVs). Multiple regression See which factors (clusters of attributes) explained most variance in perceptions (evaluation of hospitals)
Limitations Research settings & samples need to be expanded for generalizability Privacy, informed consent an issue Cost of research Process vs. outcome Use of theory