The Fundamentals of Evaluating Prevention Programs Colleen Ryan, Canada’s Drug Strategy Secretariat, Health Canada

2 Presentation Overview Why Evaluate? Why Don’t We Evaluate? Role of Evaluation and the MEM Challenges in the Field of Prevention Basic Steps in Evaluation and What Will Be Accomplished in This Meeting

3 Why Evaluate? Decision-making and overall good management of initiatives Program/project improvement Understanding what work’s and what doesn’t Tracking unanticipated outcomes Overall accountability for investment Public relations/fundraising

4 Why Don’t We Evaluate? Time Financial Resources Expertise We already “know” the program is effective Philosophy Long-term change vs. short-term funds Fear

5 Evaluation and the MEM The MEM as an important instrument of Evaluation 22 recommendations issued in the 2 nd Round Evaluation ?? recommendations issued in the 3 rd Round Evaluation Capacity issues (financial and technical) identified as primary reason for lack of progress to date

6 Challenges in the Field of Prevention Need to establish “perceived” risk for substance use/abuse at the outset Difficult to measure longer-term impacts Costly to track cohorts over a prolonged period of time Challenges in establishing valid control or comparison groups

7 Basic Steps in Evaluation  Step 1: Identify goals (anticipated outcomes) – Logic Model  Step 2: Describe the project/program – Logic Model  Step 3: Identify the evaluation questions you want answered – Logic Model and Performance Indicators  Step 4: Identify data sources and data collection tools - Performance Indicators  Step 5: Collect the information (focus of next meeting)  Step 6: Organize the information (focus of next meeting)  Step 7: Analyze the data (focus of next meeting)  Step 8: Report the results – reassess goals and program content (focus of next meeting)