1. Measuring the Performance and Impact of Community Indicator Systems: Key Performance Indicators and Evaluation Charles Thrift & Bobbie Macdonald February 17, 2011

2. Outline 1. Introduction 2. Why Evaluation and KPIs? 3. Peg evaluation framework 4. Challenges & lessons learned 5. Discussion & Next steps

3. Introduction Purpose: 1. To demonstrate the value of KPIs 2. To present the Peg evaluation framework 3. To elicit feedback and discussion on KPIs Key Performance Indicators (KPIs) are quantitative or qualitative measures that assess the performance, progress, and impact of a project, program, or organization. http://mentat.pineda.ca/blog/?p=1262

4. Introduction Program/Project Evaluation is a formal approach to assessing progress, performance, and impact. KPIs provide data and information on which to base findings and conclusions. “ Action without reflection is wrongheaded; reflection without action is self-indulgent. ” - Nancy Scheper- Hughes, Death Without Weeping, p. 171

5. Why Evaluate a CIS? to assess CIS outcomes or impacts to assess whether CIS objectives are being met to find out how CIS is operating to understand who is using CIS and how to find out how well users’ needs are being met to improve the CIS (to better meet community needs, better manage the initiative or make it more sustainable) to understand why CIS or elements of CIS do or do not work

6. Why use Key Performance Indicators? Reduce the number of decisions that are based solely on instinct and make decisions based on objectivity and facts. Quantify the achievement of goals by setting, monitoring and measuring against a standard or target. As your CIS grows it becomes more difficult to remain as close to the operational details as you once were. Allows you to focus on facts and see where you need to improve.

8. Peg evaluation framework Process: 1. Develop long list of KPIs 2. Develop evaluation framework for Peg 3. Select KPIs directly relevant to Peg

9. 1. Develop list of KPIs: Legwork: indicator system contributions + brainstorming = 133 KPIs 115 Quantitative, 18 Qualitative indicators http://www.mynetresearch.com += http://pickettsmill.typepad.comhttp://pickettsmill.typepad.com/http://www.dreamstime.comhttp://www.dreamstime.com/

10. 2. Develop evaluation framework for Peg Inputs - resources required to undertake project activities Receiving - persons receiving our information Seeking - persons seeking information related to our work Engaging - joint efforts to foster change Using - persons using our framework or information for other purposes Observed Change - actual change in policies and indicators

11. 3. Select KPIs directly relevant to Peg 54 Selected: 48 Quantitative, 6 Qualitative Exit

14. Challenges 1. Finding KPIs relevant to community indicator systems current list not exhaustive 2. Deciding which KPIs to use choosing too few vs choosing too many internal vs external indicators http://www.pmthink.comhttp://www.pmthink.com/

15. Lessons Learned 1. Existence of an evaluation and KPI gap in community indicator systems. 2. High value for labour ratio

16. Next steps 1. Extend KPI list to make it exhaustive. e.g. environmental sustainability KPIs, more qualitative KPIs,.... 2. Create dialogue to generate participation, buy-in, and uptake. 3. For Peg, take selected KPIs to internal groups.

17. Discussion 1. Is there value in using KPIs for community indicator systems? 2. How do we move forward with the momentum we’ve built? 3. How are you using KPIs now?

18. Charles Thrift Project Officer, Measurement & Assessment, IISD cthrift@iisd.ca Bobbie Macdonald Research Student, Measurement & Assessment, IISD macdonald-b3972@webmail.uwinnipeg.ca Comments/Questions? Peg: www.mypeg.ca