Aligning Outcomes with Population Needs Slides based on Carol Hays’ Identifying and Selecting Evidence Based Strategies presentations approved by CSAP For FL March 2007, where were based on Carol McHale’s CADCA 2007 NREPP presentation. Kathy Atwood and Deirdre Danahar SAMHSA/CSAP’s Southeast CAPT July 10, 2007

Key Principles of SPF Based on a public health approach Focused on outcomes based prevention Widens the scope to population-based prevention Provides a systematic strategic planning process using epidemiologic data to drive decision making Provides a framework for building systems capacity Refresher- make link between SPF Overview and this session Note this small group session focuses on making the link between using the information gained and decisions made in Assessment, Capacities, Strategic Planning steps informs the successful implementation of strategies and evaluation of these. 2 2

Integration of Substance Abuse and HIV Prevention Efforts Complex social problem Best addressed by targeting the local circumstances, conditions and individual-level behaviors. Requires comprehensive solutions at both the individual and community level Variety of intervention approaches directed to multiple aspects of the problem Determining optimal mix of strategies is complicated by limited information on evidence of effectiveness of hybrid (Substance and HIV prevention) interventions Note NREPP FY 08 Priority area of HIV/Substance abuse problems (June 4th Fed. Reg.) Small group discussion: What have been the greatest challenges they have encountered in developing their strategic plan to address integration of substance abuse and HIV prevention strategies?

SAMHSA/CSAP Expert Workgroup on Evidence-Based Programming Convened nationally recognized prevention experts to develop recommendations and guidelines for State and community prevention planners Prevention research indicates that few problems are amenable to change through direct influence Most problems must be influenced indirectly by addressing underlying factors that contribute to the problem Need broad array of evidence-based interventions Flexibility to choose options that fit community circumstances

Underlying Factors Conditions that lead to the development of problems and consequences May include specific local policies and practices, community realities and population shifts Risk and protective factors Present across the course of human development Make individuals and groups either more or less prone to substance abuse and HIV risk behaviors in certain social contexts Note that the phrases “ Causal Factors”, “Underlying factors”, “Risk & Protective Factors” and “Underlying conditions” are often used interchangeably.

Your Program Logic model is a… Picture of your program or intervention Graphic representation of the “theory of action” – what is invested, what is done, and resulting changes in participants and/or community conditions Core of planning and evaluation Provides a common framework for your work Offers a simple description of how the activities carried out within the program are related to expected outcomes

Program Logic Model Design INPUTS OUTPUTS OUTCOMES Program investments Activities Participation Short Medium Long-term Example of Program Logic Model Design Inputs - what is invested by the organization(s) Outputs - program activities and numbers served Outcomes – immediate results of program activities Impacts – cumulative changes to community conditions This example looks at single program logic model. If you are implementing one or more programs simultaneously, you need to examine each- looking at its internal dynamics as well as across the (multiple) program efforts addressed by your grant.  What we invest What we do Who we reach Resulting changes

My Program’s Logic Model What outcome(s) is my program aiming to achieve? What activities are we implementing to accomplish this outcome(s)? What changes in causal factors (e.g. risk and protective factors) does the outcome of your program contribute to? What program/policy/practice is being implemented? Small Group Exercise: Program Teams participants to create simple logic model for the MAI work. Can use what ever graphic format works for them, but must minimally address these three question on slide Report out to larger group & large group discussion of common themes, challenges and successes related to program logic models

Benefits of Program Logic Models Enhances team work and provides focus Guides prioritization and allocation of resources Motivates staff Helps to identify important variables to measure; use evaluation resources wisely Increases resources, opportunities, recognition Supports replication Often is required! Ask participants: to name some benefits they have found in using logic models

Benefits of Program Logic Models Provides a common language Helps differentiate between program activities and results --- outcomes Increases understanding of program Guides and helps focus work Increases intentionality and purpose Provides coherence across complex tasks, diverse environments Ask participants: what other benefits have they found in their own use of logic models?

What’s In It For You… Helps you explain your program to others—especially funders Provides a picture of your program on one page Helps you set boundaries: if it doesn’t fit within your logic model, you can say no. Increases your confidence in your program Helps you think as a team

Logic Models are… …Used at different conceptual and practical levels Program level Community level Transitioning here to the larger picture in which the program level logic model must operate. Logic models are used at different conceptual and practical levels:  Program level: bounded by a particular program, or practice, or strategy.  Community level: bounded by the particular issue (ATOD consequence) where we have a desired outcome, the behaviors contributing to the issue, the causal factors (e.g.: risk/protective factors…) contributing to the ATOD issue and the particular strategies which are likely to influence those factors. Now we are going to look at the community level- make reference to CSAP Evidence-Based Guidance Document here

Outcome-Based Prevention Community Logic Model Programs/ Policies/ Practices Substance-Related Consequences and Use Causal (e.g. Risk/Protective) Factors Note that CSAP uses the phrases “ Causal Factors”, “Risk & Protective Factors” and “Underlying conditions” interchangeably. Transitioning here to the larger picture in which the program level logic model must operate. Implementing the Strategic Prevention Framework Planning, Monitoring, Evaluation and Replanning

The Community Logic Model: Key Tool for Community Planning Community logic model serves as a map of the problem (e.g., substance use and HIV risk behaviors) and the factors leading to that problem Community logic model represents systematic plan for attacking local problem within a specific context Community logic model starts by defining problem not choosing the solutions (program, practices, or policies) Small group discussion: How have the participants developed and used logic models in their own MAI work?

Community Logic Model Essential tool for comprehensive, effective community prevention effort Links the risk behavior to underlying factors based on research and local data Provides explicit rationale for selecting programs, policies and practices to address the risk behavior

Community Logic Model Example Reducing alcohol related youth traffic fatalities - Each causal factor provides an opportunity or potential point of entry for types of interventions or strategies (policies, practices, programs) that may change the targeted problem Substance - Substance Related Causal Strategies Use Factors (Examples) Consequences Underage Easy RETAIL ACCESS to Alcohol for youth BINGE Enforce underage DRINKING retail sales laws Low ENFORCEMENT of alcohol laws Social Event High rate of Underage Easy SOCIAL ACCESS to Monitoring and alcohol - DRINKING Alcohol Enforcement related crash AND DRIVING mortality Low PERCEIVED RISK of alcohol use Media Advocacy to Among 15 to Young Adult Increase Community 24 year olds BINGE Concern about SOCIAL NORMS accepting DRINKING Underage Drinking and/or encouraging youth drinking Restrictions on Prevention Strategy: Any effort to reduce the incidence or prevalence of substance use and HIV risk behavior Path to Outcomes: Influence intervening variables that in turn influence substance use and subsequent HIV risk behaviors Young Adult PROMOTION of alcohol use (advertising, movies, alcohol advertising in DRINKING music, etc) youth markets AND DRIVING Bans on alcohol price Low or discount PRICING promotions and of alcohol happy hours 16

Community Logic Models Fit Unique Community Needs Communities must tailor the logic model to fit their particular context, needs, capacities, and readiness Small Group Discussion: In what ways does their own experience reflect this Statement? What lessons have they learned?

Analytic Process to Select Best Fit Prevention Interventions

Conceptual Fit of Strategies Identify strategies that are relevant to the variables in the community’s logic model Strategies that don’t fit the community logic model are unlikely to effectively address the identified problem Select strategies to effect the community’s most significant risk/protective factors and conditions Target multiple points of entry to impact the target population across social environments and sectors

Comprehensive Community Plans Person Primary Environments Near Environments Broader Environment Time School Peers Family Work Neighborhood Religious Groups Laws and Policies Social Services Media Judicial System Political Influences Cultural Influences Economic Influences

Practical Fit and Utility of Strategies Is the strategy a practical fit with the resources, capacity, and readiness of the community organizations responsible for implementing interventions Developed/adaptable for the target population Age, gender, culture, language Implemented successfully with a similar population Delivered in a similar setting Implementation materials, Training/TA, evaluation tools available to ensure implementation quality

Assess Feasibility of Strategies Cultural fit with community values Political fit with local power structure Organizational fit with mission, vision, culture Administrative fit with policies and procedures Technical fit with staff capability and time Financial fit with existing resources Make links here with Day 1 Cultural Competency sessions, and Day 2 morning sessions

Benefits of logic modeling to community partnerships Provides framework to discuss and articulate joint work Helps facilitate conversation with focus on agreed upon goals Shows contribution of each partner and how result depends upon all Keeps end outcome upfront and center Provides way to communicate about the partnership to others

Prevention System Pre-Alignment Stakeholders i.e. Providers, Police, Preventions professionals, community leaders Stakeholders efforts & resources target different outcomes Community ATOD Coalition Efforts are diluted and not coordinated to effect change Programs/ Policies/ Practices Why develop a common Community Logic Model… helps to foster and sustain alignment of system and efforts. HIV/AIDS and substance use and related consequences too complex issues to be addressed by only one organization Example here uses a coalition, could be some other type of “community task force” or collective working together in an intentional way to plan to address common issues and is working towards a common outcome. community wide reductions in identified ATOD use and related consequences

Prevention System Alignment Stakeholders i.e. Providers, Police, Preventions professionals, community leaders Stakeholders efforts & resources target same outcomes Community ATOD Coalition Efforts are targeted and coordinated to effect change Programs/ Policies/ Practices community wide reductions in identified ATOD use and related consequences

My Community Logic Model What is the outcome my community is aiming to achieve? What behaviors (e.g.: specific substance use) are present in my community? What causal factors (e.g.: risk and protective factors) are supporting the behaviors? What local contributing factors are present? What is the overlap between my community logic model and my program level logic model? Audience breaks in to small groups to develop community model Groups will report out on their findings What is the overlap between community logic model and the program level logic models? Groups to use easel and poster paper provided

Benefits of Community Logic Models Provides common language and understanding among partners Guides and focuses work Provides coherence across work done by partners Guides prioritization and allocation of resources Enhances collaboration and provides focus Ask participants: Name some benefits they have found in using community logic models

Logic Model and Evaluation Questions Needs assessment: What are the characteristics, needs, priorities of target population? What are potential barriers/facilitators? Process evaluation: How is program implemented? Are activities delivered as intended? Are participants being reached as intended? What are participant reactions? Logic model link to evaluation planning, design and questions Outcome evaluation: To what extent are desired changes occurring? For whom? Is the program making a difference? What seems to work? Not work? What are unintended outcomes?