Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Logic Model in Mental Health Program Development

Similar presentations


Presentation on theme: "The Logic Model in Mental Health Program Development"— Presentation transcript:

1 The Logic Model in Mental Health Program Development
Dr. Martin Cortez Wesley Lindsey Wilson College

2 A logic model is your program ROAD MAP
Where are you going? How will you get there? What will tell you that you’ve arrived? _________________________________________________________ A logic model is your program ROAD MAP

3 Provides a common framework
Logic model is a… Picture of your program or intervention Graphic representation of the “theory of action” – what is invested, what is done, and what results Core of planning and evaluation Provides a common framework for your work

4 MODEL Definition LOGIC the principles of reasoning reasonable
the relationship of elements to each other and a whole MODEL small object, representing another, often larger object (represents reality, isn’t reality) preliminary pattern serving as a plan tentative description of a system or theory that accounts for all of its known properties The American Heritage Dictionary, 2nd Ed

5 The accountability era
What gets measured gets done If you don’t measure results, you can’t tell success from failure If you can’t see success, you can’t reward it If you can’t reward success, you’re probably rewarding failure If you can’t see success, you can’t learn from it If you can’t recognize failure, you can’t correct it. If you can demonstrate results, you can win public support. Re-inventing government, Osborne and Gaebler, 1992

6 Language: What do you mean by…
Goal = Impact Impact = Long-term outcome Objectives (participant focused) = Outcomes Activities = Outputs Outputs may signify “tangible” accomplishments as a result of activities

7 Logic Method Research Provides a common language
Helps us differentiate between “what we do” and “results” --- outcomes Increases understanding about program Guides and helps focus work Leads to improved planning and management Increases intentionality and purpose Provides coherence across complex tasks, diverse environments

8 Enhances team work Guides prioritization and allocation of resources Motivates staff Helps to identify important variables to measure; use evaluation resources wisely Increases resources, opportunities, recognition Supports replication Often is required!

9 Logic model is in widespread use
Let’s not think that logic modeling is brand new. Actually, the concepts have been around since the late 1960’s in the writings of Suchman, 1967 and Wholey’s evaluability assessment model. It has come to the forefront again, and is being developed and applied in a variety of settings as a result of a variety of factors: Private sector: part of total quality management and performance measurement movement Public sector, the GPRA has moved all federal agencies to focus on results and link investments to results, not just activities. Non-profit sector is concerned with improving programs to produce valued impacts with the United Way being a frontrunner in outcome measurement using the logic model. International programs. The players in the international arena for a long time have used variations of a logic model. The Log Frame of the US Agency for International Development of the 1980’s is a historical precedent to the current logic modeling discourse. And, professional evaluators have played a prominent role in using and developing the logic model. This is why it is often called an ‘evaluation framework.’ This is a result of evaluators being asked to evaluate impact and finding, too often, that programs didn’t exist, or weren’t being implemented in a way that would achieve the expected impact. Consequently, evaluators began working with programmers to lay out the logic of programs. We see the outgrowth particularly in Chen’s theory-driven evaluation (1990) and Weiss (1997) theory-based evaluation. Private Sector Public Sector Non-Profit Sector International Arena Evaluators

10 Cook, play, talk, laugh, hike
Example: Every day logic model – Family Vacation Family Members Drive to state park Family members learn about each other; family bonds; family has a good time Budget Set up camp Car Cook, play, talk, laugh, hike Camping Equipment Actually, we use logic models everyday. Let’s look at this… We want to take a family vacation and what we really hope is that we’ll have a good time and enjoy being together. We have had experience and know (our own personal research tells us) that camping is something we all enjoy doing together. So, in order to take a camping trip, we need.. If this…, then that…. Logic models involve a mental process. A logic model shows the series of connections and logical linkages that is expected to result in achievement of our goal.

11 Example: Professional Counselor Shortage
Situation: Eastern Kentucky has a huge shortage of Mental Health Professionals to address the tremendous need in the area. INPUTS OUTPUTS OUTCOMES Lindsey Wilson College invests time and resources LWC starts several regional sites to begin to train people to be Mental Health Professionals Local residents become educated professionals, change service practices, and Mental Health Services improve Let’s apply this to a typical Extension example WHAT WE INVEST WHAT WE DO WHAT RESULTS

12 OUTCOMES INPUTS OUTPUTS Situation:
Example: One component of a comprehensive parent education and support initiative Situation: During a county needs assessment, majority of parents reported that they were having difficulty parenting and felt stressed as a result OUTCOMES INPUTS OUTPUTS Parents increase knowledge of child dev Parents identify appropriate actions to take Develop parent ed curriculum Staff Improved child-parent relations Targeted parents attend Parents better understanding their own parenting style Money Deliver series of interactivesessions Parents use effective parenting practices Strong families Partners Parents gain skills in effective parenting practices Research Facilitate support groups Assumptions: External factors:

13 Example: Smoke free worksites
Situation: Secondhand smoke is responsible for lung cancer, respiratory symptoms, cardiovascular disease, and worsens asthma. Public policy change that creates smoke free environments is the best known way to reduce and prevent smoking. Inputs Outputs Outcomes Assess worksite tobacco policies and practices Demonstrations of public support for SF worksites Increased awareness of importance of SF worksites Coalition Time Dollars Partners Including youth Worksite owners, managers Develop community support for SF worksites SF worksites Increased knowledge of SF worksite benefits & options SF worksites policies drafted Unions Workers; union members Organize and implement strategy for targeted worksites SF worksite policies passed Increased commitment, support and demand for SF worksites Public Adherence to smoke-free policies

14 Logic model of a training workshop
Situation: Funder requires grantees to include a logic model in their funding request; grantees have limited understanding of logic models and are unable to fulfill the funding requirement INPUTS OUTPUTS Increase knowledge of logic models Increase ability to create a useful logic model of program Increase confidence in using logic models OUTCOMES Create meaningful logic models Use logic models in own work Fulfill requirement of funder 3 hour training Interactive activities Group work Practice Q and A Trainer Funds Equipment Research base Training curriculum Grantees Improved planning Improved evaluation Accountable here

15

16 Feedback loops and multi-dimensions
Programs aren’t linear Feedback loops and multi-dimensions INPUTS OUTPUTS OUTCOMES Program investments Activities Participation Short Medium Long-term What we invest What we do Who we reach What results

17 Chain of outcomes SHORT MEDIUM LONG-TERM Seniors increase
knowledge of food contamination risks Practice safe cooling of food; food preparation guidelines Lowered incidence of food borne illness Participants increase knowledge and skills in financial management Establish financial goals, use spending plan Reduced debt and increased savings Community increases understanding of childcare needs Residents and employers discuss options and implement a plan Child care needs are met Empty inner city parking lot converted to community garden Youth and adults learn gardening skills, nutrition, food preparation and mgt. Money saved, nutrition improved, residents enjoy greater sense of community

18 Focus of outcomes Individual Group Agency, organization System
Child is ready to enter school; farmer implements nutrient management practice Families control spending to maintain family financial stability Agency institutes policy that encourages physical activity of staff Family serving agencies share resources to better meet clientele needs Communities develop and preserve decent safe and affordable housing Individual Child, parent, client, resident Group family, team, community group Agency, organization System Community

19 Writing good outcomes SMART objectives: Specific, measurable, attainable, results-oriented, timed Who/what Change/desired effect In what By when Families participating in the Family Resource Center increase their use of community resources and services within one year of joining 4 school boards adopt policies to improve student nutrition and physical activity by Dec 2007

20 What does a logic model look like?
Graphic display of boxes and arrows; vertical or horizontal Relationships, linkages Any shape possible Circular, dynamic Cultural adaptations; storyboards Level of detail simple complex Multiple models _________________________________________________________

21 Multiple logic models Multiple models may be needed to describe and explain complex initiatives or systems. Multi-level programs: A series of linked models that depict varying levels such as national-state- county levels OR, institution-division-unit levels Multi-component programs: A series of models to depict various components (goals, sites, target populations) within a comprehensive initiative

22 State level logic model: Reducing and preventing youth tobacco use
Outcomes - Impact Inputs Activities Reach Short Medium Long Increased awareness of need to eliminate youth access to tobacco products, including tobacco industry tactics, laws, noncompliance Promote community involvement in restricting tobacco access to youth Establish baseline of current practices Inform/educate Eliminate self-service Facilitate active enforcement of laws Increased compliance and enforcement of laws and policies Decreased access to tobacco for minors Community Parents, Caretakers Law enforcement Retailer Health Department Coalition Members Decreased supply to minors Increased commitment to eliminate access/sources Funding Social norms less supportive of youth tobacco use Increased knowledge and skills in participating in policy change Increased # of youth actively engaged in policy change Partners Local Regional State Facilitate youth involvement in policy change Recruit youth Involve youth/adults Educate Community org, Businesses Policy makers Adults Youth serving org Youth Increased commitment by youth and adults for youth to participate in policy change Increased adoption of policy changes that involve youth in the change process Reference: Wisconsin Tobacco Control Program Work Group including Christine Dobbe and Jennifer Leahy of UW-Extension, Based on: Workgroup CDC guidelines and best practices CIA Sub-Group technical assistance packet CIA Sub-Group reviewers CIA Sub-Group Suggested Goals “Restaurant Ordinances vs. All Worksites Including Restaurants” (CIA Sub-Group, 2003) DPH 2004 Boundary Statement and template objectives Promote school and community based prevention programs and policies Establish baseline of existing resources Educate Assist with planning and implementing programs/services Research and best practices Delayed average age at first use; reduced initiation Schools Community Families Youth serving org Youth Increased knowledge about tobacco dependence; benefits and options for youth prevention (e.g, CDC guidelines, school-family initiatives) Increased # of effective prevention programs or policies adopted Reduced morbidity and mortality Increased # of youth participating in prevention programs See Treating Tobacco Addiction Youth Logic Model Promote youth cessation services and policies Increased commitment to adopt effective programs/policies for youth prevention

23 Component Logic Model Youth: Youth Advocating for Policy Change
Outcomes - Impact Inputs Activities Reach Short Medium Long Coalition members Establish baseline for policy change in community with help from youth Increased # youth, community members who: Understand tobacco use issues in their communities Know how to advocate for policy change Increased # of youth actively engaged in advocating for policy change in community Community organizations, businesses policy makers Increased number of tobacco policies in community Time Educate youth and adults on policy change options and how to achieve them Increased # of activities or increased intensity of activities that involve youth to accomplish policy change Adults Funding Identify partners, including youth serving organizations and schools, for engaging youth in policy change Partners Local Regional State Social norms less supportive of youth tobacco use Youth serving organizations Increased # youth wanting to be involved in advocating for policy changes Increased adoption of policies that involve youth in the policy change Counter industry influence Promote clean indoor air Decrease availability of tobacco products in the community Develop strategy for and promote engagement of youth in policy change Schools Effective practice strategies Increased # youth skilled in being able to advocate for policy change Delayed average age at first use; reduced initiation Assist with development of youth advocacy skills YOUTH Local media outlets Promote community support for youth involvement in community affairs/ policy change Increased support for youth involvement in policy change

24 Multi-Org partnership
Programs as “systems” Single organization SHORT-TERM OUTCOMES INPUTS OUTPUTS Org 1 MED-TERM OUTCOMES Multi-Org partnership Org 2 LONG-TERM OUTCOMES Org 3 Community Org 4 Source: Adapted from CDC:

25 Getting started Group process Determine purpose of logic model
Who will use it? For what? Involve others Set boundaries for logic model Understand situation Explore research, knowledge base, what others are doing/have done Group process

26 Check your logic model Is it meaningful? Does it make sense?
Is it doable? Can it be verified?

27 Limitations Logic Model… Represents reality, is not reality
Focuses on expected outcomes Challenge of causal attribution Many factors influence process and outcomes Doesn’t address: Are we doing the right thing? Challenge of causal attribution: Does not suggest/’prove’ that program caused the outcome

28 Where does evaluation fit?
From beginning to end

29 What do you want to know? How will you know it?
PLANNING: start with the end in mind What do you want to know? How will you know it? EVALUATION: check and verify

30 What does evaluation mean to you?
Evaluation means asking good, critical questions about programs to improve programs and help them be accountable for the wise use of resources.

31 Logic model and evaluation
Needs/asset assessment: What are the characteristics, needs, priorities of target population? What are potential barriers/facilitators? What is most appropriate? Process evaluation: How is program implemented? Fidelity of implementation? Are activities delivered as intended? Are participants being reached as intended? What are participant reactions? Outcome evaluation: To what extent are desired changes occurring? For whom? Is the program making a difference? What seems to work? Not work? What are unintended outcomes?

32 EVALUATION: What do you (and others) want to know about this program?
Parents increase knowledge of child dev Develop parent ed curriculum Parents identify appropriate actions to take Staff Targeted parents attend Improved child-parent relations Money Deliver series of interactivesessions Parents better understand their own parenting style Partners Parents use effective parenting practices Research Parents gain skills in effective parenting practices Strong families Facilitate support groups What amount of $ and time were invested? How many sessions were actually delivered? How effectively? # and quality of support groups? Who/how many attended/did not attend? Did they attend all sessions? Supports groups? Were they satisfied – will they come again? To what extent did knowledge and skills increase? For whom? Why? What else happened? To what extent did behaviors change? For whom? Why? What else happened? To what extent are relations improved? Does this result in stronger families?

33 Prioritize Lots of questions and so little time
Prioritize evaluation questions Evaluation purpose Need Context Process Outcomes Stakeholder needs

34 You – staff Participants Funder
Who wants to know what about your program? WHO might use the evaluation? WHAT do they want to know? HOW will they use the info? You – staff Participants Funder

35 Developing an evaluation plan based on your logic model
1. Focus: 2. Questions 3. Indicators 4. Timing 5. Data collection Sources Methods Sample Instruments Inputs Outputs Outcomes

36 Logic model and reporting
University of Wisconsin-Extension, Program Development and Evaluation

37 “We build the road and the road builds us.”
-Sri Lankan saying


Download ppt "The Logic Model in Mental Health Program Development"

Similar presentations


Ads by Google