Mission Accomplished! How one coalition demonstrated a

Mission Accomplished! How one coalition demonstrated a
contribution to big outcomes Paul Evensen, PhD Raymond Tamasi President, CSG LLC President/CEO, Gosnold on Cape Cod Cindy Pharis, MS Patricia Mitrokostas Community Evaluator, CSG LLC Director of Prevention, Gosnold on Cape Cod Amanda Sutter, MA, MSW Community Evaluator, CSG LLC © 2014 CSG, LLC Helping you create healthier communities

Falmouth Prevention Partnership
Context Falmouth Prevention Partnership History of the Partnership Relationship with CSG © 2014 CSG, LLC Helping you create healthier communities

Contribution vs. Attribution
Analysis of Contribution Five Step Process Document the Dose Establish a Relationship in Time Demonstrate a Mechanism of Effect Account for Alternative Explanations Show Similar Effects in Similar Contexts Contribution vs. Attribution 3 – 5 minutes Paul – Overview of process and why © 2014 CSG, LLC Helping you create healthier communities

1. Document the Dose What is it that coalitions can do that individual members cannot do (or do as well) alone? Strategic Roles of Community Coalitions: Change Agent Services / Supports Optimizer 5 minutes Paul Community Conversation Focus Resources © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

1. Document the Dose The Partnership implemented a monitoring system in the Fall of 2008 to document all accomplishments via an on-line system (ODSS). All accomplishments were reviewed and “scored” by local partnership staff and the evaluators. The protocol followed is specifically recommended by the C.D.C., the W.H.O. and other scientific authorities for community partnerships1. Cindy begins and takes through 12 1 The CDC published recommendations for adopting the methodology in See Fawcett, S.B., Sterling, T.D., Paine-Andrews, A., Harris, K.J., Francisco, V.T., Richter, K.P., Lewis, R.K., & Schmid, T.L. (1995). Evaluating community efforts to prevent cardiovascular diseases. Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Atlanta: GA. The World Health Organization followed suit in See Fawcett, S.B., Paine-Andrews, A., Fancisco, V.T., Schultz, J.A., Richter, K.P, Berkley-Patton, J., Fisher, J., Lewis, R.K., Lopez, C.M., Russos, S., Williams, E.L., Harris, K.J., & Evensen, P. (2001). Evaluating community initiatives for health and development. In I. Rootman, D. McQueen, et al. (Eds.), Evaluating health promotion approaches. (pp ). Copenhagen, Denmark: World Health Organization - Europe. © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

1. Document the Dose Services Provided are events that impart information, develop skills, and give social support such as through classes, screenings, alternative activities, and workshops. Records on services provided include the number of classes conducted, participants, length, and location. Examples: The Falmouth Prevention Partnership provided an in-service training for K-12 Adjustment and Guidance Counselors in Falmouth Public Schools on use of evidenced-based prevention strategies. FPP participated in a "Credit for Life" Community Service Booth. The event is for high school seniors to prepare them for financial literacy and community volunteerism. The Alcohol and Marijuana Action Team brought the "It's All About YOU" presentation by Richard Campbell to the Junior and Senior Falmouth High School students in a one-hour assembly format. © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

1. Document the Dose 3212 youth & 3120 adults received
services facilitated by the Partnership. *2013 data are for the first 9 months of the year. 7 © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

1. Document the Dose Media are instances of coverage of the initiative, its projects, or issues in the newspaper, on-line (“new media”), radio, television, and other channels. Records on media include the number of impressions and the media channel. Examples: "Program Aims To Educate Parents Who Serve Alcohol“ article in Falmouth Enterprise describes how parents of high school students will receive a letter in the mail warning them of the legal consequences of allowing teenagers to drink alcohol in their homes. "Parents Who Host Lose The Most" public service announcement ran 55 times from Dec.1 -Dec.24, 2011 on local public access television, Channel 13. Dr. Mike Bihari, FPP Chair, wrote the Friday, September 27, 2013 “Risky Business” column in the Falmouth Enterprise. The article contained statistics on underage drinking and the correlation between teen perception of parental disapproval and a decrease in underage drinking. © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

1. Document the Dose “Parents Who Host Lose the Most” Campaign Average Impressions per Falmouth Resident 2011 2012 2013 6.11 22.32 9.88 Patty react to slides 5 – 10 to include the use of the ODSS (challenges and benefits of keeping track); how well data describes or summarizes their work; key points or ideas to emphasize about the content and planning of media or services © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

1. Document the Dose Community Changes are new or modified programs, policies and practices facilitated by the partnership to reduce substance abuse in Falmouth. This includes modifications to the physical environment. Examples: For the first time a Social Host Lesson at Falmouth High School was implemented as part of the health curriculum for students in the 9th through the 12th grade. 375 students attended the social host laws class. (New Program: May 1, 2012) The Falmouth Police Department dispatching added a new category to track liquor law violations for underage persons. Prior to this there was not a specific way to collect data on underage liquor law violations. (New Practice: September 13, 2011) The FPP, Falmouth Public Schools and the Falmouth Police Department co-sponsored and required attendance at a Social Host Pre Prom Assembly for the parents/caretakers of children attending the Junior Prom. (New Program: May 9, 2012) © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

1. Document the Dose The Falmouth Prevention Partnership facilitated:
64 Community changes intended to reduce underage alcohol use. 42 Community changes intended to build opportunities for pro-social involvement or that targeted all drug use. 30 Community changes intended to reduce Rx abuse. 28 Community changes to increase community capacity (such as new partnerships). 164 Documented new or modified programs, policies, and practices facilitated by the Partnership from 2009 to in order to reduce substance abuse in Falmouth, MA. © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Dimensions of Intensity1
1. Document the Dose Dimensions of Intensity1 Dimension Scoring Rationale Duration High Medium Low Ongoing More than Once One Time Emerging evidence and expert consensus suggest that community changes that are sustained are more likely to affect behavior change and population-level outcomes. Dimension Scoring Rationale Duration High Medium Low Ongoing More than Once One Time Emerging evidence and expert consensus suggest that community changes that are sustained are more likely to affect behavior change and population-level outcomes. Reach ≥ 21% of population = 6% to 20% of population ≤ 5% of population Emerging evidence and expert consensus suggests that exposure to community changes is important to affect outcomes. Dimension Scoring Rationale Duration High Medium Low Ongoing More than Once One Time Emerging evidence and expert consensus suggest that community changes that are sustained are more likely to affect behavior change and population-level outcomes. Reach ≥ 21% of population = 6% to 20% of population ≤ 5% of population Emerging evidence and expert consensus suggests that exposure to community changes is important to affect outcomes. Strategy Policy, Consequences, Access Social Support Information, Skills Emerging evidence and expert consensus suggests that community changes addressing policy and access are more likely to achieve outcomes. Paul begins here and goes through slide 19. 1 Adapted from Collie-Akers, V.L., Fawcett, S.B., and Schultz, J.A. (2013). Measuring progress of collaborative action in a community health effort. Rev Panam Salud Publica, 34(6): © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Dimensions of Intensity2
1. Document the Dose Dimensions of Intensity2 Community Change Duration Reach Strategy Score Launch of the sticker shock campaign in 18 liquor retail stores with 10, 000 stickers. Medium Low 1.20 Community Change Duration Reach Strategy Score Launch of the sticker shock campaign in 18 liquor retail stores with 10, 000 stickers. Medium Low 1.20 Pre Prom Assembly which is mandatory for all parents of students going to the Junior prom. High 1.65 Community Change Duration Reach Strategy Score Launch of the sticker shock campaign in 18 liquor retail stores with 10, 000 stickers. Medium Low 1.20 Pre Prom Assembly which is mandatory for all parents of students going to the Junior prom. High 1.65 For the first time participated in "Credit for Life" Community Service Booth. 0.30 2 Adapted from Collie-Akers, V.L., Fawcett, S.B., and Schultz, J.A. (2013). Measuring progress of collaborative action in a community health effort. Rev Panam Salud Publica, 34(6): © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

sum annual intensity score
Intensity of Changes to Reduce Underage Alcohol Use facilitated by the Falmouth Prevention Partnership Provide Information Provide Support Build Skills Change the Physical Design Access & Barriers Policy Change Rewards & Consequences © 2014 CSG, LLC sum annual intensity score 15 minutes Patty and Ray reaction post-explanation – How is seeing the changes presented this way helpful or not (by intensity and strategy)? How does this help you demonstrate progress, value or impact? Year (First 3 Quarters) n = 15 i = 15.75 n = 16 i = 23.70 n = 36 i = 38.70 n = 32 i = 45.15 n = 26 i = 37.95 n = number of changes i = total intensity score © 2014 CSS, LLC

Analysis of Contribution Five Step Process Document the Dose Establish a Relationship in Time Demonstrate a Mechanism of Effect Account for Alternative Explanations Show Similar Effects in Similar Contexts Contribution vs. Attribution © 2014 CSG, LLC Helping you create healthier communities

2. Establish a Relationship in Time
Falmouth implements a survey of students modeled on the Communities That Care instrument (PRIDE Survey). Measures the incidence and prevalence of substance use, delinquency, and related problem behaviors and the risk and protective factors that predict those problems in communities. Surveys were conducted in 2009, 2011, and 2013. Students in 8th through 12th grades were surveyed (In 2009 students in 7th grade were also included). © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Relationship in Time Between Changes to Reduce Underage Alcohol Use and Self-Reported Rates of Alcohol Use in the Past 30 Days by 11th Graders 100 90 80 70 60 50 40 30 20 10 Sum Intensity Score % of 11th Grade Students Reporting Alcohol Use in the Past 30 Days. sum annual intensity score i = 15.75 i = 23.70 i = 38.70 i = 42.60 i = 37.50 % = 58.30 % = 56.50 % = 33.50

Past 30 Day Use of Alcohol by Class Cohort
2. Establish the Relationship in Time Past 30 Day Use of Alcohol by Class Cohort Baseline Data Class of 2017 is showing change starting in the 8th grade (p = .05) Intervention Data Change occurs around high school with Classes 2014, 2015, and (p = .05) Helping you create healthier communities Helping ou create a healthier communities

3. Demonstrate a Mechanism of Effect
By what means could the changes, services, and media implemented by the Partnership have affected rates of underage alcohol use? 1. Facilitated changes, services, and media implemented interventions with an existing evidence-base. Social Host Laws Parents Who Host Lose the Most Sticker Shock Guiding Good Choices Pro-Social Opportunities © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

By what means could the changes, services, and media implemented by the Partnership have affected rates of underage alcohol use? 2. Facilitated changes, services, and media implemented a comprehensive set of behavior change strategies. Provide Information Provide Social Support Build Skills Change Access / Barriers Alter Rewards / Consequences Change the Physical Design of the Environment Change Rules & Policies © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

By what means could the changes, services, and media implemented by the Partnership have affected rates of underage alcohol use? 3. Pathway (logic model) Past 30 Day Alcohol Use Availability Retail Home Pro-Social Supports Parental Attitudes © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Alcohol Use and Related Factors by Class Cohort
2. Establish the Relationship in Time Alcohol Use and Related Factors by Class Cohort Baseline or “Pre” After Interventions or “Post” Helping you create healthier communities Helping ou create a healthier communities

By what means could the changes, services, and media implemented by the Partnership have affected rates of underage alcohol use? 3. Pathway (logic model) Past 30 Day Alcohol Use Availability Retail Home Pro-Social Supports Parental Attitudes Past 30 Day Alcohol Use Availability Retail Home Pro-Social Supports Parental Attitudes © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

4. Account for Alternative Explanations
A. Secular Trends - National data from Monitoring the Future survey shows a downward trend in 30 day use of alcohol over the same time period. - State data from the Youth Health Survey shows a downward trend in 30 day use of alcohol over the same time period. + Trends in Falmouth exceed those observed nationally or at the state level. + Secular trends not followed on other substances such as marijuana. © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Concurrent Efforts To better understand concurrent efforts in Falmouth key informant interviews were conducted with: State prevention staff Local leaders in youth work and prevention © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Concurrent Efforts State informants report no concurrent implementation of prevention efforts reaching Falmouth youth and families (one effort in Vineyard located elsewhere on the Cape). Local informants reported three major efforts: VIPS (focused on skills and mentoring) Jack’s PACT Together We Can (after-prom; recreation center) FPP Partnered with all of these (funded, coordinated, co-implemented, helped to establish). © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Conclusions Influences on the national and state trends in rates of alcohol use may also be influencing rates in Falmouth. VIPS, Jack’s Pact, and Together We Can are also making contributions to improved youth outcomes. Much of these partner’s work is accounted for in the Partnership’s documented accomplishments / dose. Additional dosage data from these community partners is required to quantify their potential contribution. © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Conclusions Only the Partnership was found to have implemented interventions consistent with the pathway effect observed. National trends are inconsistent with the amount, timing, and specific nature of the observed effects (sudden, marked, and alcohol/parental attitudes specific). © 2014 CSG, LLC Helping you create healthier communities Helping ou create a healthier communities

Evaluating Community Partnerships
Paul Evensen, PhD Amanda Sutter, MA, MSW President, CSG LLC Community Evaluator, CSG LLC Cindy Pharis, MS Community Evaluator, CSG LLC 5 minutes (1 min per person/1 min to overview) Self introductions Overview of Session © 2014 CSG, LLC Helping you create healthier communities

Mission Accomplished! How one coalition demonstrated a

Similar presentations

Presentation on theme: "Mission Accomplished! How one coalition demonstrated a"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Mission Accomplished! How one coalition demonstrated a

Similar presentations

Presentation on theme: "Mission Accomplished! How one coalition demonstrated a"— Presentation transcript:

Similar presentations

About project

Feedback