1 Linking Performance Measures to Benchmarks in the Budget Process March-April 2002 Department of Administrative Services Oregon Progress Board

2 Overview Why measure performance? Why Oregon Benchmarks? What makes a good performance measure? What is required in the budget process? Getting started

3 Handouts Logic models –Logic model worksheets (yellow) –Logic model examples (ochre) Submission forms –Links to Oregon Benchmarks (blue) –Performance Measure Data Summary (green) Evaluation forms –PM criteria worksheet (off-white) –Today’s training evaluations (purple)

4 Why measure performance? It’s at the core of results-based management Provides greater accountability Is the ship on course? Fosters internal learning and improvement Is the ship running well? AND…it has been required since See Appendix B.

5 Why link to Oregon Benchmarks? They articulate Oregon’s hopes and expectations. “High-level outcomes” or measures of societal well-being. Beacons for the “ship” and the “fleet”. For budget, link only to those that relate to your core mission and goals (“primary linkages”).

6 Oregon has ninety benchmarks in three broad categories. Economy Education Civic Engagement Social Support Public Safety Community Development Environment

7 What happens if your agency does not link to an Oregon Benchmark?. That’s OK. You have two options: –You may submit other high-level outcomes to gauge how Oregon is doing relative to your mission. –Small agencies: if this is not feasible, you can “look up” to your mission and/or mandate. All high-level outcomes should pass the “so what” test. Do Oregonians care? So what??

8 Logic models define the links. Goal (generally unmeasurable) Performance Measures Impact Intermediate Outcome Measures Agency Inputs and Activities Output Measures High-level outcome(s) (measurable)

(Increase) % of offenders with intake assessments Output “So That” % of offenders engaged in work, training, education and/or treatment (is increased) Intermediate Outcome “So That” % of offenders showing a measurable improvement in behavior and/or skill level (is increased) Intermediate Outcome “So That” % of paroled offenders convicted of a new felony within three years (is decreased) High-Level Outcome (Benchmark #61) A logic model embeds a continuum of measures in a“so that” chain.

10 What makes a good performance measure? BASIC criteria required for Performance measures should: 1.Use GASB* terms and definitions 2.Gauge progress towards agency goals and benchmarks or other high-level outcomes 3.Focus on a few key indicators 4.Have targets 5.Be based on accurate and reliable data *Governmental Accounting Standards Board

11 OUTCOME = Result (the best kind of measure) –High-level (societal) = OBM#11, Per capita income –Intermediate = Average wage of agency job placements OUTPUT = Product or service (“widget”) –# of job placements per quarter INPUT = Time, money, material or demand –FTEs in the “Job Placement Unit” –Dollars allocated to the “Job Placement Unit” –Case load or number of complaints –INPUTS ARE NOT STAND-ALONE PERFORMANCE MEASURES EFFICIENCY = Input per output –# of days required to process a job application Basic criteria #1. Use GASB definitions

12 Two kinds of intermediate outcomes: chunks and stones EXAMPLE: Benchmark #18, Ready to Learn 1.A “chunk” of the population is measured for the high-level outcome (HLO) % of children of served families who are ready to learn (versus % of all children in the county who are ready to learn) 2.“Stepping stone” toward the HLO is measured. % of trained parents who read regularly to their children (reading to kids is a stepping stone to being ready to learn)

# of intake assessments completed Output % of offenders engaged in work, training, education and/or treatment Intermediate Outcome % of offenders showing a measurable improvement in behavior and/or skill level Intermediate Outcome % of paroled offenders convicted of a new felony within three years High-Level Outcome (Benchmark #64) Basic criteria #2. Measure progress towards agency goals and benchmarks Goal to “reduce repeat offenders” is UNMEASURABLE MEASURES gauge progress

14 Basic criteria #3. Focus on a few key measures. Represent the scope of agency responsibility Number 30 max (except for mega-agencies) Include the best measures for: –“Is the ship on course?” –“Is the ship running well?” Additional measures internal to your agency can provide more detailed management information.

Agencies should decide how “high up” to go for their key measures. More agency influence More policy intent Consider level of agency INFLUENCE # of intake assessments completed Output % of offenders engaged in work, training, education and/or treatment Intermediate Outcome % of offenders showing a measurable improvement in behavior and/or skill level Intermediate Outcome % of paroled offenders convicted of a new felony within three years High-Level Outcome (Benchmark #64)

16 Basic criteria #4. Performance measures should have targets. TARGET = Desired level at any given point in time Should be ambitious but realistic Target setting is an art and a science based on –trend data –comparisons –expert opinion Targets not required until Jan Recidivism now Recidivism TARGET

17 Basic criteria #5. Accurate and reliable data. Without trustworthy data, the system is meaningless. Example: verifiable employment records are better than estimated job creation Each measure should have at least one data point, preferably several. Data should describe what is being measured.

18 Performance measure criteria ADVANCED = required for biennium Performance measures should: 6.Link to an organizational unit 7.Cover organizational outcomes like efficiency and customer satisfaction 8.Allow comparisons More training on Advanced Criteria later

Annual Performance Reports submitted to DAS/LFO. (Annually in September) Submit Links to Oregon Benchmarks (March - August 2002) TA & Training on Performance Measures Budget Instructions Comments & Measures Accompany Governor’s Recommended Budget (November 2002) See Guidelines pp.10 & 11 Budget Timeline for Performance Measures (April – August 2002) Adjustments (Optional) Performance Measure Data Summary to Ways & Means (January - June 2003 ) Agencies adjust measures and targets per legislature (June 2003) Criteria-based review (April – Aug. 2002)

20 Hypothetical example #1 Impact AGENCY INPUT/ACTIVITY Award grants to local contractors to conduct “best practice” juvenile crime prevention programs (JCP). INTERMEDIATE OUTCOMES % of juveniles in JCP programs with significantly mitigated risk factors. GOAL Reduce juvenile crime. HLO Juvenile Arrests (OBM#61) Agency Performance Measures OUTPUTS # grants awarded by county # days of TA delivered by county

21 Hypothetical example #2 Impact AGENCY INPUT/ACTIVITY Award grants to local contractors to design/deliver “best practice” parent education classes. INTERMEDIATE OUTCOMES % of children from participating (trained) families entering school ready to learn. GOAL Healthy, thriving children. HLO: % of kindergarteners ready to learn (OBM#18) Agency Performance Measures OUTPUTS # grants awarded by county. “Best practice” guidelines done by

22 Hypothetical example #3 Impact AGENCY INPUT/ACTIVITY Jointly sponsor, with cities, regional educational events for private citizens every quarter. INTERMEDIATE OUTCOMES % participating citizens with improved understanding Customer satisfaction ratings GOAL: Citizen involvement (C.I.) in land use planning HLO: % of cities with neighborhood organizations. Agency Performance Measures OUTPUTS # citizens trained. # C.I. guidelines distributed.

Related Oregon Benchmarks (OBMs) or High-Level Outcomes (HLOs): % of cities with active neighborhood organizations Agency Goal OBM# HLO# Key Performance Measure PM # PM Since New or Mod.? 2000 Valu e 2005 Target Lead Division or Unit (Optional) Citizen involvement in land use planning 1 Percent of participants with improved understanding Ag# New 55 % 70 % Communications Pertinent Benchmark or High-level outcome(s): Links to Oregon Benchmarks Form HLO 1 - Percent of cities with active neighborhood organizations.

Performance Measure Definition (numbered as shown below) DataTargets Agency # - 1 Agency # - 2 Agency # - 3 Agency # - 4 Agency # - 5 Agency # - 6 Agency # - 7 Agency # - 8 Performance Measure Data Summary (for Ways and Means) 55%62%70%60%65% Percent of participants with improved understanding

25 Helpful websites  Governmental Accounting Standards Board GASB home page  National Center for Public Productivity, Rutgers A Brief Guide to Performance Measurement in Local Government (1997)  John F. Kennedy School of Government, Harvard An Open Memorandum to Government Executives - Get Results Through Performance Management (2001)

26 Additional resources Book and reports –Measuring Up, Jonathan Walters (1998) –The Reinventor’s Fieldbook, David Osborne and Peter Plastrik, Chapter 7 (2000) –Making Results-Based State Government Work, The Urban Institute (2001) Oregon Progress Board –Technical Assistance –Training –Strategic Planning

27 George Dunford Performance Measure Manager, DAS (503) Jeffrey L. Tryens Executive Director, Progress Board (503) Rita Conrad Senior Policy Analyst, Progress Board (503) DAS/Oregon Progress Board