Session Goal To provide participants with a basic knowledge of formative evaluation and its use when developing an injury prevention program.
Describe the purpose of formative evaluation Describe several formative evaluation methods Provide resources for each formative evaluation method Session Objectives
The 4 Stages of Evaluation Review Formative Will this work? Process Are we doing what we set out to do? Impact Can we see any changes? Outcome Have we reduce injuries?
The 4 Stages of Evaluation Review Process Evaluation Impact Evaluation Outcome Evaluation Long-term: Timely Feedback Baseline Data Collection Project Implementation Pre-TestingMonitoring Health Outcomes Looks at Changes in Target Audience Formative Evaluation Short-term: Knowledge, Attitudes & Practice
Formative Evaluation Defined The process of testing and /or obtaining expert review of program plans, messages, materials, strategies, or modifications, for weaknesses and strengths before they are put into effect.
Formative Evaluation Purpose Get target audience perception of the possible interventions Gain community support Ensure program materials & activities are acceptable or culturally appropriate Identify problems with program concepts or design Used in conjunction with process evaluation
Formative Evaluation When to Conduct New program New materials Existing program is: modified, having problems, or adapted for a new setting, population, problem, or behavior Ongoing at all stages of program
Formative Evaluation Who to Involve - Stakeholders The people with a vested interest in a program and the future use of its evaluation
Formative Evaluation Stakeholders Example: Car seat distribution program Who should we include? Implementers Partners Participants Decision makers
Formative Evaluation Data Collection Methods 1.Focus Groups 2.Key Informant Interviews 3.Town Hall Meetings 4.Expert Reviews 5.Surveys
Formative Evaluation Focus Groups - Purpose To gather information on a particular topic by conducting an informal discussion between 6 to 10 individuals similar to target population
Formative Evaluation Focus Groups 8 Important Steps Secret Ingredients: -Focus group similar in makeup to target population -Requires an experienced facilitator
Formative Evaluation Key Informant Interviews - Purpose To obtain detailed information by asking broad open-ended questions and by probing interviewee
Formative Evaluation Key Informant Interviews 7 Important Steps Secret Ingredients: -Take care to select informants with various points of view -Remember to first establish rapport
KEY INFORMANT EXERCISE Scenario: Local IPS is seeking to determine attitudes and beliefs about ATV related injuries and possible interventions. They will interview two key community members.
Formative Evaluation Town Hall Meetings - Purpose To gather a diverse group of community members together to think and talk about an issue
Formative Evaluation Town Hall Meetings 8 important steps Special Ingredients Plan early Make sure there is plenty of time for interaction between audience and presenters
Formative Evaluation Expert Reviews - Purpose To identify mistakes, concerns, and general areas for improvement.
Formative Evaluation Expert Reviews 7 important steps Special Ingredients The committee should be continuously involved in the process. Include detailed instruction in the job description.
Formative Evaluation Surveys - Purpose To obtain SPECIFIC information from a targeted group of people using a set of defined questions.
Formative Evaluation Surveys 7 important steps Special Ingredients Avoid overloading the survey Design questionnaire such that its clear and simple.
Formative Evaluation Examples in IHS Sleep Safe and Ride Safe Curriculums – Focus Groups, Key Informant Interviews, Expert Review Animal Control Project - Surveys Child Passenger Safety Training - Expert Review IHS Area Injury Prevention Program Evaluations - Expert Review
Formative Evaluation Conclusion Formative evaluation Tests program plans, messages, materials, strategies, or modifications for weaknesses and strengths before they are put into effect. Allows the evaluator to probe the feelings, beliefs, and impressions of participants. Generates community support for the proposed project or program.