Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Performance Measurement 101 PART 1 Oregon Public Performance Measure Association (OPPMA) Annual Meeting 8:45 – 11:45 Session with Rita Conrad, Ken Smith.

Similar presentations

Presentation on theme: "1 Performance Measurement 101 PART 1 Oregon Public Performance Measure Association (OPPMA) Annual Meeting 8:45 – 11:45 Session with Rita Conrad, Ken Smith."— Presentation transcript:

1 1 Performance Measurement 101 PART 1 Oregon Public Performance Measure Association (OPPMA) Annual Meeting 8:45 – 11:45 Session with Rita Conrad, Ken Smith and Laura Wipper Willamette University, Salem, OR

2 2 2 Todays Session Part 1: The basicsPart 1: The basics (~60 min) –Intro & Brief History –Criteria –Logic models Part 2: Small groupsPart 2: Small groups (~75 min) –Practice –Report-out Part 3: Making it happenPart 3: Making it happen (~60 min) –Three approaches –Dialogue

3 3 Late 1800s-1930s New York Municipal Research Board Audit of Portland, Oregon International City Management Association (ICMA) –Herbert Simon, Nobel Prize winner

4 4 1970s Productivity Optimistic Views –Harry Hatry (1972; 1978) Negative or Cautionary Views –Quinn (1978); –Burkhead and Hennigan (1978)

5 5 1980s – Why Accountants? Governmental Accounting Standards Board (GASB) Government Finance Officers Association (GFOA) Association of Government Accountants (AGA) –Oregon is the best state at performance reporting – by a long shot (Smith, Cheng, Smith, Schiffel 2006 – available online)

6 6 Original Oregon Model @ ODOT Roots in work of Glenn Felix –Productivity matrix Baseline – or average – levels of performance Historical best or optimum goals Relative weights, performance levels and performance index Efficiency and Effectiveness –Cost vs. Quality –Gainshare pilot at ODOT proves value is in the measures and the conversation

7 7 Striving for Balance Cost per product, service or result Labor per product, service or result Direct hours Indirect hours Administrative hours Outcome/Goal Whats Important Along the Way Customer Satisfaction Work Life Quality

8 8 At ODOT Regularly gauged the capacity of the organization –Sick leave –Safety –Turnover –Employee Survey –Equal Opportunity Employment –Training

9 9 Development of Measures Series of workshops –Involved almost all staff –Aligned activities to purpose, outcomes, goals –Measures were customized by staff Alignment through Key Result Areas Some were roll-ups

10 10 Statewide Rollout Statewide measurement initiative began in 1991 with 16 agencies – Phase 1 More phases followed – 100+ agencies Agency-level measurement facilitated by mentors from previous phases Expanded upon ODOT model –Efficiency and effectiveness –Tied goal-orientation to Oregon Benchmarks

11 11 Some 90s Success Stories DMV –Positive experience with a negative index Ways and Means liked the candid conversation Adult and Family Services –Moved focus from application processing to family self- sufficiency Lead the nation in welfare reform Vocational Rehabilitation –Decided federal goal of two months back on the job not high enough to justify cost Set goal at 18 months Department of Corrections –Sharper focus on direct recipient of program efforts 180 degree swing on recidivism

12 12 Original Oregon Model Adopted across the US Implemented nationwide by US Army Corps of Engineers –One of first two agencies to act following federal Government Performance Results Act of 1993 At one point, 70 Federal agencies were implementing the Oregon model following success of Corps

13 13 Elements of Oregons Key Performance Measure System Tiered, linked system for external reporting Common language Logic models Criteria Budget

14 14 SB555 gives Progress Board first time evaluation function HB 3358 gives performance measurement role to Progress Board Linking state performance to benchmarks 939495969798990001020304050607 Measuring Up by Jonathan Walters 98 Oregon laws required agency performance measures 93 Progress Board prepares Performance Measure Guidelines for State Agency Budget Instructions LFO and BAM assume more performance measurement and management functions 09 08 Progress Board prepares KPM- Benchmark Crosswalks for W&M subcommittees

15 15 Oregon Benchmarks and Performance Measures

16 Activity #1 Getting to Know You

17 17 Why measure performance? To get results. Its at the core of results-based management Fosters internal learning and improvement Is the ship running well? Provides greater external accountability Is the ship on course? Informs policy-making Should the ships change course?

18 18 CRITERIA What makes a good performance measure? 1.Common language 2.Alignment (line-of-sight) 3.Accurate and reliable data 4.A few key measures 5.Time-framed targets 6.Someone in charge 7.Comparisons 8.Customer satisfaction

19 19 OUTCOME, high-level = Societal result OUTCOME, intermediate = Entity result OUTPUT = Product or service INPUT = Time or money EFFICIENCY =Input per output or Input per outcome Criterion #1. Common language 19

20 20 3. High-Level Outcomes (HLO) – Is the world you are attempting to affect changing? Oregon Benchmark #65, Juvenile Recidivism – % of juveniles referred w/in 12 months … Practical application 2. Intermediate Outcomes – Is the strategy producing the desired result? % of treated youth with reduced risk factors 1. Outputs – Is the strategy getting done? # of youth completing treatment 20

21 21 Criteri on #2. Alignment - line-of-sight to goals and high-level outcomes Goal Key Performance Measures Intermediate Outcomes Outputs HLO - Mission or Benchmark Impact Strategies to achieve the goal 21

22 22 Criterion #3. Accurate and reliable data Trustworthy data is essential. Example: verifiable records trump estimates Per measure - at least one data point, preferably several Data should match the measure.

23 23 Criterion #4. Few key measures – different measures for different purposes More agency influence More public interest Goal: Prevent Juvenile Crime % of grantees trained INCREASES. Output So That % of high risk youth completing program INCREASES. Intermediate Outcome So That % of served youth with mitigated risk factors INCREASES. Intermediate Outcome So That Juvenile recidivism DECREASES. High-Level Outcome How high up do you want to go? 23

24 24 Criterion #5. Time-framed targets TARGET = Desired level at a given point in time –Ambitious but realistic –Set targets using: trend data comparisons expert opinion 2007 Target: 15 2003 Actual: 20 Example: hours of travel delay per capita per year 24

25 25 Criterion #6. Someone in charge Management, staff and stakeholders need to know where the buck stops Critical for improving performance 25

26 26 Criterion #7. Measures of customer satisfaction This is a required Key Performance Measure (KPM) for Oregon state agencies. Rationale: Those who pay for their government should be satisfied with their government. 26

27 27 Criterion #8. Comparisons How does actual performance compare to industry standards? Rationale: to better understand whether action is needed. Difficult but not impossible! Agency suggestion: peer agencies develop common measures and comparator We resolved complaints within three weeks on average. Is this good or bad? 27

28 28 LOGIC MODEL - a tool for thinking it through Getting started 1.IMPORTANT: Engage those whose performance will be measured. 2.Have a clear statement of goals. 3.Look up to mission & high-level outcomes (benchmarks) 4.Look down to what you are doing and how you are measuring performance Agency Goals Performance Measures High-level outcomes

29 29 Logic model example #1 – Citizen involvement in land use planning Impact STRATEGY Jointly sponsor, with cities, regional educational events for private citizens every quarter. INTERMEDIATE OUTCOMES % participating citizens with improved understanding Customer satisfaction ratings GOAL: Citizen involvement (C.I.) in land use planning HLO: % of cities with neighborhood organizations. Agency Performance Measures OUTPUTS # citizens trained. # C.I. guidelines distributed.

30 30 Logic model example #2 – Juvenile crime Impact STRATEGY Award grants to local contractors to conduct best practice juvenile crime prevention programs (JCP). INTERMEDIATE OUTCOMES % of juveniles in JCP programs with significantly mitigated risk factors. GOAL Reduce juvenile crime. HLO Juvenile Arrests (OBM#61) Agency Performance Measures OUTPUTS # grants awarded by county # days of TA delivered by county

31 31 Logic model example #3 – Healthy, thriving children Impact STRATEGY Award grants to local contractors to design/deliver best practice parent education classes INTERMEDIATE OUTCOMES % of children from participating (trained) families entering school ready to learn GOAL Healthy, thriving children. HLO: % of kindergarteners ready to learn (Benchmark #18) Agency Performance Measures OUTPUTS # grants awarded by county

32 32 PART 2: Small group exercise – 20 min. 1.Pick an organization from within your group 2.Pick a goal from that organization 3.Work through the logic model as a team 4.Select two key measures – the most important – for box #3 5.Select a spokesperson to report out 32

33 33 Performance Measurement 101: Part 3 Oregon Public Performance Measure Association (OPPMA) Annual Meeting 8:45 – 11:45 Session with Rita Conrad, Ken Smith and Laura Wipper Willamette University, Salem, OR

34 34 Part 3 – Making it Happen Performance Management –Integrated across many systems/disciplines –Multiple Steps that usually include: Strategy Reports with data Discuss/Argue Learn/Change Revisit Strategy –No single dominant view – but common tools: Balanced Scorecard, Strategy Maps, LEAN, Six Sigma, TQM, Variance Analysis, Organization Alignment/Incentives, etc. –Numerous barriers, challenges and gaps

35 35 Some Models for Measurement

36 36 Background Laura has worked in ODOT and other agencies for almost 20 years Ken has studied performance reporting by cities and states for over 17 years

37 37 Why create a model Our hope is to help participants create their own mental models of: –Where they and their organization are at –What are the next steps/models they can use –What choices and priorities they should consider as they move forward The models are not intended to be judgmental – where one level is always better or more appropriate….the costs/benefits for each approach need to be carefully considered….not every model works for every person

38 38 Measuring to Managing

39 39 Three Models Overview Single MeasuresOperating Unit systemIntegrated MACRO System DescriptionLists one or many measures – does NOT show logic model how the measures connect in a logical fashion Illustrates or describes how various measures or explanatory information impacts the desired outcome for a single entity or system Illustrates or describes how various measures or explanatory information impacts the desired outcome for a several interacting entities/systems ExamplesSome of the Oregon APPRsBalanced Scorecard Oregon Benchmarks actual Carl DeMaios Perf Logic Model GASB ideal with explanatory info Oregon Benchmarks ideal City of Coral Springs – case study StrengthsEasy – can be great place to start Collecting and sharing raw data often starts good conversations Logic model connections internal to the system/entity Exhaustive model WeaknessesLack of logic model limits usage – and allows different interpretations No connection to explanatory factors outside the entity such as other orgs & socio- economic environment Very complicated – especially if using statistical tools like regression EquationIf we measure – performance will improve Outcome = org measure 1 org measure 2 org measure 3 Outcome = Org measures + Outside Orgs + Environment

40 40 A Model for Integrated Measures

41 41 An Example using Education

42 42 An Example using ODOT

43 43

44 44

45 45

46 46

47 47

48 48 5 Principles by Bob Paladino Similar to many other models – I like his since he uses both government and business examples – and book available on Amazon $25 I also like his model since it focuses explicitly on the need for: –a Chief Performance Officer (CPO) –Using multiples tools (BSC, LEAN, Maps, etc) p.s. I also like him because he is a CPA

49 49 Five Key Principles 1.Establish CPM (Corporate Performance Management) Office and Officer 2.Refresh and Communicate Strategy 3.Cascade and Manage Strategy 4.Improve Performance 5.Manage and Leverage Knowledge

50 50 Illustration of 5 Key Principles

51 51 Challenges/Barriers 9 out of 10 companies fail to implement their business strategies Paladino p. 11 –Hey there government folks – the so called business sector struggles with putting these good ideas into action just like we do

52 52 Paladinos Four Barriers to Implementation Vision –Only 5% of the workforce understands the strategy Management –85% of executive teams spend less than one hour per month discussing strategy Resource –60% of organizations dont link budgets to strategy People –Only 25% of managers have incentives linked to strategy

53 53 Bob Behn - Harvard lecturer (Has a nice monthly newsletter) Why Performance Management is NOT sweeping the world? (2002) –Practical reasons: It just does not work –Political reasons: Does not win elections –Managerial reasons: It is da?n hard –Psychological reasons: Creates legit fear and Requires organization changes

54 54 Building Your Plan Three different tools – same basic idea to self-evaluate using a Gap Analysis –ActiveStrategy online Strategy ActivatorStrategy Activator 19 questions –Paladino Chapter 9 – 3 Step Self-Diagnostic 35 best practices (Score yourself – Gap – Action) –Weidner Accountability and Transparency 17 questions

55 55 Additional Case Studies: (time permitting) City of Coral Springs, Florida – North Carolina Benchmarking Project – ExPeRT website @ Willamette –

56 56 A few links to useful sources BOOKS Balanced Measures for Strategic Planning, by Kathleen Monahan (2000)Balanced Measures for Strategic Planning Balanced Scorecard, by Robert Kaplan and David Norton (1 st book in 1992 – several additional books and articles) Five Key Principles of Corporate Performance Management, by Bob Paladino (2007) BLOGS is very good – 10 or so writers like Miller, Metzenbaum, COMPUTER CONSULTANTS Active Strategy – 9 phase model9 phase model ORGANIZATIONS National Performance Management Advisory Commission

57 57 Contact Information Rita Conrad Ken Smith Laura Wipper

Download ppt "1 Performance Measurement 101 PART 1 Oregon Public Performance Measure Association (OPPMA) Annual Meeting 8:45 – 11:45 Session with Rita Conrad, Ken Smith."

Similar presentations

Ads by Google