The Center for Youth and Communities The Heller School for Social Policy and Management, Brandeis University Creating and Using Logic Models for Project Compass Ginger Fitzhugh and Chris Kingsley February 15 & 16, 2008

Center for Youth and Communities, Brandeis University2 “Wouldn’t it be nice if just once they’d give us money, skip all the falderal, and say ‘go crazy.’”

Center for Youth and Communities, Brandeis University3 What we’ll cover What is a logic model? Why construct one? How do you create one? A few hints

Center for Youth and Communities, Brandeis University4 What is a logic model? A picture of how your initiative works – what you will do, with whom, why, and what will result (your theory of change) If we do X, then Y should happen, if the assumptions on which the initiative is based are true  If we work together and change our policies and practices, our underserved students will remain enrolled and graduate.

Center for Youth and Communities, Brandeis University5 The Project Compass logic model asks: What is the problem to be addressed? Who will be served through your initiative? What are your assumptions about how and why your initiative will work? What strategies/activities will you use? What outcomes do you expect? How will you measure success? What are the long-term impacts you hope to achieve?

Center for Youth and Communities, Brandeis University6 Parts of the Project Compass logic model For Whom Assumptions Strategies/ Activities Outcomes Measures of Success Long-Term Impacts Target group(s) What you know, think and believe about what’s needed and will work Activities needed to achieve desired outcomes Reasonably measurable year-by-year changes in policies, practices or target group(s) Information needed to show whether initiative is successful Ultimate outcomes for initiative Name of Initiative Problem Statement: What is the problem to be addressed?

Center for Youth and Communities, Brandeis University7 Examples For Whom AssumptionsStrategies/ ActivitiesOutcomesMeasures of Success Long- Term Impacts Students of color Faculty and staff in academic departments Stakeholders from across campus (administrators, faculty, students and community members) must collaborate for institutional change to be possible and sustained. Faculty lack information about how to advise underserved students who arrive academically underprepared. Conduct cross-campus committee meetings to identify ways to streamline delivery of academic student support services (President appoints, year 1 meet 3x each semester). Develop and deliver professional development workshops to help faculty better advise underprepared students re: what courses to take and what support services are available on campus (2 opportunities each semester, starting with faculty who have academic advising responsibilities). (A) Academic student support services are streamlined and, (B) underserved students are able to more easily access the services they need. Faculty advisors provide more accurate and complete information to underserved students re: coursework and support services. (A) Coordinators of services make accurate referrals. (B) The number of underserved students accessing students support services increases. The number of underserved students reporting that their faculty advisors informed them of appropriate coursework to take and support services increases. (Add questions to our end- of-year survey.) Underserved students experience higher levels of persistence and success “ Clamshell University” Project Compass Initiative Problem Statement: Students of color at “Clamshell University” currently experience disproportionally lower rates of academic success, are more likely to drop out, and are less likely to graduate.

Center for Youth and Communities, Brandeis University8 We hate logic models that… Are useless academic exercises Are created and never looked at again Are too generic Aren’t shared with stakeholders Are never used

Center for Youth and Communities, Brandeis University9 Benefits of the process of creating a logic model Clarifies thinking Uncovers different understandings Makes it easier to identify gaps May lead to consideration of new ideas Helps focus complex work Helps decide what outcomes are important to track Helps create shared ownership

Center for Youth and Communities, Brandeis University10 Benefits of having a logic model product Aids in planning Shows cause-and-effect relationships between strategies and outcomes Makes it easy to share a project description with others Supports accountability and evaluation Assists in replication Often is required!

Center for Youth and Communities, Brandeis University11 How do you create a logic model? Bring key stakeholders together. Then: 1.Define the problem you will address 2.Determine individuals or group(s) you will target 3.State long-term impacts and work backwards 4.Decide on outcomes 5.Ask, “If we want to achieve these outcomes, what strategies should we use?” 6.Ask, “If we’re successful, how will we know it?” 7.Consider, “What assumptions underlay our approach?” 8.Work backwards and forwards across categories

Center for Youth and Communities, Brandeis University12 Steps for Creating a Project Compass logic model For Whom Assumptions Strategies/ Activities Outcomes Measures of Success Long-Term Impacts Target group(s) What you know, think and believe about what’s needed and will work Activities needed to achieve desired outcomes Reasonably measurable year-by-year changes in policies, practices or target group(s) Information needed to show whether initiative is successful Ultimate outcomes for initiative Name of Initiative Problem Statement: What is the problem to be addressed?

Center for Youth and Communities, Brandeis University13 Quick Aside re: Assumptions Behind each project is a set of beliefs and/or knowledge held by those who plan a project about how change will come about The validity of the connection between strategies, outcomes and impacts depends on the existence of conditions necessary for success We call these assumptions

Center for Youth and Communities, Brandeis University14 Core Assumptions for Project Compass (abbreviated) Underserved students are an asset to the institution. Sustained, institution-level change supporting increased success and retention of underserved students requires ongoing collaboration from across the college. “Islands of excellence” which retain underserved students but exist at the margins of the institution will not result in broad-based cultural change unless they are scaled up. Change in institutional culture supporting the success and retention of underserved students must be supported by research. An ongoing culture of evidence and inquiry is essential to the formulation, implementation, and improvement of practices and policies supporting underserved students. Colleges and universities committed to institutional change to better retain underserved students will benefit from collaborative relationships with peer institutions with like commitments.

Center for Youth and Communities, Brandeis University15 A Few Hints

Center for Youth and Communities, Brandeis University16 “I think you should be more explicit here in Step Two.”

Center for Youth and Communities, Brandeis University17 Strategies vs. Outcomes vs. Measures of Success Examples STRATEGIESOUTCOMES MEASURES OF SUCCESS What you doWhat changesHow you know Offer faculty development in culturally responsive pedagogies. Faculty increase use of culturally responsive pedagogies. Faculty report implementing new teaching strategies (survey/interviews) and students’ assessments of faculty teaching improve (course evaluations). Develop social opportunities and groups for underserved students. Student participants have a greater sense of belonging and engagement. Students who participate in new opportunities report groups helped them feel more engaged (survey). Develop and use statistical methods to predict which students are at greatest risk for dropping out. Advisors use better information to identify underserved students who might need additional support early in students’ academic careers. The number of underserved students identified as needing additional support increases.

Center for Youth and Communities, Brandeis University18 Questions to ask yourself as you work on your logic model Is it reasonable to expect the initiative’s planned strategies will lead to the expected outcomes? Are all the outcomes supported by strategies? Are all target groups, strategies, and outcomes included? Are there sufficient resources to undertake the activities? Do all the components lead to one or more outcomes? Are the outcomes really outcomes, not activities? Are the outcomes reasonably measurable? Are all stakeholders in agreement about the logic model? Adapted from Porteous et al., 2002

Center for Youth and Communities, Brandeis University19 Final Hints Focus on being outcome-driven not activity- driven Check logical connections across categories Revisit and revise periodically Can use to develop more detailed action plans OK to develop a logic model that is multiple pages It will get easier!

Center for Youth and Communities, Brandeis University20 Summing up Logic models: Create shared understanding Help with planning, implementation, evaluation and reporting Link activities to results: fill gaps and avoid mismatches across categories

Center for Youth and Communities, Brandeis University21 “My question is: Are we making an impact?”