Why Shared Measurement Matters Srik Gopalakrishnan, FSG April 2013

About FSG and Shared Measurement Nonprofit consulting and research firm founded in 2000 Staff of 120 in Boston, San Francisco, Seattle, Washington DC, Geneva Success in strategy and evaluation with over 400 foundations, corporations, and nonprofits Thought leader: articles published in HBR, SSIR, American Journal of Evaluation including Collective Impact (SSIR 2010) Breakthroughs in Shared Measurement and Social Impact (funded by Hewlett Foundation) published in 2009: examined 20 approaches to performance, outcome, impact measurement FSG is driven by the same passion that drives our clients: a passion for greater social impact using data as a key lever

The Premise: There Are Several Types of Problems Simple Complicated Complex Right “recipe” essential Gives same results every time Example: Baking a cake “Formulas” needed Experience built over time and can be repeated with success Example: Sending a rocket to the moon No “right” recipes or protocols Outside factors influence Experience helps, but doesn’t guarantees success Example: Raising a child Traditional approach in social sector has been to treat problems as simple or complicated Source: Adapted from “Getting to Maybe”

Current approaches not conducive to complex problems: Traditional Ways of Approaching Social Change Are Not Working to Address Our Toughest Challenges Current approaches not conducive to complex problems: Funders select individual grantees that offer the most promising solutions Grantees work separately, compete to produce the greatest independent impact Evaluation attempts to isolate a particular grantee’s impact Large scale change assumed to depend on scaling a single organization Corporate, government, philanthropy and nonprofit sectors often disconnected from efforts of foundations and nonprofits Isolated Impact

Imagine a Different Approach Isolated Impact Shift mindset from a “technical” to an “adaptive” approach: Understand that social problems – and their solutions – arise from interaction of many organizations within larger system Large scale impact depends on increasing cross-sector alignment and learning among many organizations Nonprofits, government, philanthropy and corporates actively coordinate their action and share lessons learned Progress depends on working toward the same goal and measuring the same things – SHARED MEASUREMENT Collective Impact

Benefits of Using Shared Measurement Shared Measurement Is a Critical Piece of Pursuing a Collective Impact Approach Definition Identifying meaningful common metrics for tracking progress toward a common agenda across organizations, and providing scalable platforms to share data, discuss learnings, and improve strategy and action Benefits of Using Shared Measurement Tracking Progress Toward a Shared Goal Enabling Coordination and Collaboration Learning and Course Correction Catalyzing Action in the Field Audience engagement activity: Which of these benefits is most important for your organization? (hold up a finger) Are there other benefits?

Developmental Phases in Creating a Shared Measurement System 1 2 3 Design Develop Deploy Shared vision for the system and its relation to broader goals, theory of change or roadmap View of current state of knowledge and data Governance and organization for structured participation Identification of metrics, data collection approach, including confidentiality/ transparency Development of data collection tools and technology platform Refinement and testing of platform and tools Staffing for data management and synthesis Learning forums and continuous improvement Ongoing infrastructure support Improve system based on a pilot, review, refinement, and ongoing evaluation of usability and impact

Examples of Shared Measurement in Use

Shared Measures for 100Kin10 Partner Organizations Advance Develop & Improve Induct Place/Hire Prepare Recruit STEM Teacher Experience DRAFT Excellence Outcomes (quality of teachers) Candidates demonstrate: Academic proficiency Leadership potential Appropriate mindset to succeed in STEM classrooms and school environments* Candidates demonstrate basic: Instructional practice STEM content knowledge Pedagogical content knowledge Preparedness for the context in which they will teach Qualified STEM teachers are: Placed/hired Prepared for the context in which they are placed/hired New STEM teachers demonstrate improved STEM-related: Instructional practice Pedagogical content knowledge STEM Content knowledge Student engagement STEM teachers demonstrate improved STEM-related: Instructional practice Pedagogical content knowledge STEM Content knowledge Student engagement Student achievement STEM teachers: Take on instructional leadership roles Move into administrative roles Engage more deeply with community stakeholders Influence peers and colleagues * Mindset persists throughout STEM teaching career

Shared Measures for Portland Metro STEM Partnership

Ways Network Partners Use Shared Measurement System To Learn with and from Peer Programs and Organizations To Inform Internal Decision-making About Practices To Identify Practices of High-performing Organizations To Identify Other Relevant Partners To Uncover Areas Where Further Data or Research Is Needed

Challenges in Developing and Implementing Shared Measurement Systems Difficulty in coming to agreement on common outcomes and indicators Concerns about relative performance / comparative measurement across providers working in the same space Limited capacity (time and skill) for measurement and data analysis within participating organizations Aligning funders to ask for the common measures as part of their reporting requirements Time and cost of developing and maintaining a system, both for human capital and technology Audience engagement: Talk with a partner – which of these challenges resonate with you the most?

Critical Factors in the Development of Shared Measurement Systems Help Overcome These Challenges Effective Funding Relationships Strong leadership and substantial funding throughout multi-year development period Independence from funders in devising indicators and managing the system Broad and Open Engagement Broad engagement in design process by many organizations in field, with clear expectations about confidentiality/ transparency Voluntary participation open to all relevant organizations Infrastructure for Deployment Effective use of technology Ongoing staffing to provide training, facilitation, and to review accuracy of data Pathways for Learning and Improvement Testing and continually improving system through user feedback Facilitated process for participants to gather periodically to share results, learn from each other, and coordinate efforts Source: Breakthroughs in Shared Measurement and Social Impact, FSG, 2009

FSG Lessons Learned in Implementing Shared Measurement Systems Don’t Underestimate the Value of Partner Engagement Proceed Iteratively Focus on Learning and Use Address Both Technical and Political Challenges Ensure Long-term Funding and Sustainability A well designed and structured Shared Measurement effort will pay dividends for years to come