1. 1 Training Session Sponsored by the Association of Government Accountants A Systems Approach to Implementing Performance-Based Management and Budgeting Audio Conference March 21, 2012

2. Today’s Presenters Stephen L. Morgan, President, EGAPP, Inc., and former Austin City Auditor can be reached at egappmorgan@yahoo.com Sam McCall, City Auditor, Tallahassee, Florida and former Deputy State Auditor of Florida, can be reached at Sam.McCall@talgov.com 2

3. 3 A Systems Approach to Performance Based Management and Budgeting I.Introduction – Performance Accountability System II.Historical Overview – Where We Have Been in Austin and Beyond III.Performance Planning IV.Performance Budgeting V.Performance Measurement and Reporting VI.Performance-Based Decision Making VII.Conclusion – What We Have Learned and Where We Are Going

4. 4 PLAN Strategic & Annual Planning PLAN Strategic & Annual Planning DO Performance Budgeting DO Performance Budgeting CHECK Performance Measurement & Reporting CHECK Performance Measurement & Reporting ACT Performance-Based Decision Making ACT Performance-Based Decision Making I. 1. Introduction: Government Performance Accountability System

5. 5 BUSINESS PLANNING Program/Activity Objectives Organizational and Individual Performance Measures Structural Alignment PERFORMANCE BUDGETING Performance Targets Accounting System PERFORMANCE MEASUREMENT & REPORTING Individual SSPR Evaluations Organizational Performance Assessment Performance and Measurement Audits PERFORMANCE-BASED DECISION-MAKING Citizens Council Managers Employees I. 2. Managing for Results Framework City of Austin

6. 6 Use existing data whenever possible Find a balance between too few and too many measures Audit the data regularly Modify measures when necessary Centrally located staff to analyze data and coordinate the system elements Technological infrastructure to support the system I. 3. Characteristics of a Successful System

7. 7 Data forms should have space for explanatory information and detail Tie measures to budgetary allocation and reward system Support of top management Over the long run should affect bottom line performance of the organization Citizens will be better informed and more participative I. 3. Characteristics of a Successful System (continued)

8. 8 1992 – Council Resolution on Performance Measurement and Reporting 1994 – First Performance Measurement & Reporting System Audit 1996 – Second Performance Measurement and Reporting System Audit; Program Budgeting implemented 1998 – Third Performance Measurement and Reporting System Audit II. 1. Where We’ve Been (in the City of Austin) …

9. 9 1998 Corporate Managing for Results Initiative Defined 1999 Corporate Partnership Implements CMO Initiative Simplify our System Clarify the Information We Provide Develop Measures that are Meaningful to our Employees Focus on Cost Developed a Standard manual--The Resource Guide Trained over 200 managers Developed a Single Accounting System Identified Key Performance Measures for Executive SSPRs Corporate Review Team II. 2. Where We’ve Been…

10. 10 2002 Fourth Audit of the Performance Management System 2003-2008 Continuous Improvement Ongoing Integrated System Information Used for Operational Management Measures Are Relevant and Reliable Budgets Are More Data and Results Driven Managers and Supervisors Fully Trained Performance Measures Supported by More Robust Technology Improvements Made to City’s Website and Stakeholder Access to Performance Information Citizen and Employee Surveys Provide Data for Selected Performance Measures II. 3. Where We’ve Been…

11. 11 2008-Current Website Robust with Capacity to “Drill Down” and Search” through Performance Measures Database “Managing for Results” Used as Business Planning and Performance Monitoring Model for More than a Decade--Now Part of City Culture Performance Report on Website tracks 115 Key Departmental Measures, of these 21 are Designated Citywide Key or “Dashboard” Measures Performance Comparisons Presented in Graphics with Goal/Targets and Measures Tracked Over Five Years Performance Report for 2009-2010 Received “Certificate of Excellence” from ICMA in October 2010 Annual Citizen Surveys Strengthened to Include Focus Groups and Presentations to City Council “Best Practice Citizen Centric” External Performance Accountability Report Is Needed II. 3. Where We Are Now…

12. II. 4. Beyond Austin Federal Government Performance Management Continues to Evolve  Government Performance and Results Act of 1993  Executive Order 13450-Improving Government Program Performance, Nov 13, 2007  OMB 10-24: Performance Improvement Guidance under GPRA for 2011-2012  Government Performance and Results Act Modernization of 2010 (signed Jan. 4, 2011)

13. II. 4. Federal Agencies with Well Developed Performance Management Systems  Social Security Administration  Department of Interior  Government Accountability Office  Nuclear Regulatory Commission  Office of Personnel Management 13

14. Some Local, State, & Provincial Governments Have Established Performance Management Mandates

15. State and Local Governments With Well Developed Performance Management  States of Florida, Washington, Texas, Missouri, and Oregon (may have been recognized for individual State departments who are mature and excel in developing and applying performance management systems)  Local governments include Austin, King County, Phoenix, Bellevue, Charlotte, Portland, Palo Alto, and Tallahassee  Auditors have played key roles in many performance measurement and management initiatives

16. 16 III. Performance Planning III. 1.Establishing programs, activities, and potential performance expectations III. 2.Developing annual business/performance plans with performance expectations and measures III. 3.Reviewing business/performance plans to support improvement and accountability

17. III.1. Service Delivery System (Program Model) Input Output Process Community Impact Intermediate Outcome Long-term Outcome Other Contributing Factors

18. 18 Inputs Processes Outputs Outcomes Service Efforts Service Accomplishments Financial Inputs/Outputs = Unit Cost Outputs/Physical Inputs = Productivity Inputs/Outcomes = Cost Benefit and Cost Effectiveness III. 1. Service Delivery System Cause/Effect Relationships

19. 19 Government Performance Expectations MISSION PERFORMANCE OBJECTIVES/GOALS INPUT Economy & Sufficiency PROCESS Efficiency OUTPUT Effectiveness OUTCOM E Effectiveness Financial: Amount, timing Physical: Quantity, quality, timing, price Capacity vs. demand Productivity Unit Costs Operating ratios Quantity Quality: Products, delivery Timeliness Price or cost Mission & Outcome Goal Achievement Financial Viability Cost-Benefit Cost- Effectiveness

20. 20 III. 1. Service Delivery System: Auditing Program Audit Program or Activity Inputs Staff Funding Equipment Facilities/Rent Processes Audit Process- (Survey, fieldwork, & reporting) Outputs Reports Briefings Presentations Outcomes Qualitative – Policy/system/ management improvements Quantitative – Cost savings/ revenue enhancement Preventive – Deterrence/ detection

21. 21 III. 1. Program/Activity Mapping Template InputsProcess Outputs (Services Delivered) Outcomes (Results)

22. 22 III. 1. Austin’s Definition of Programs Activity = Input Process Output Outcome Program = group of activities with a common purpose Example: Audit Program consists of four activities: Performance Audits Investigations Consulting and Assistance Quick Response

23. 23 III. 2. Overview of the Development of Business Plans

24. 24 III. 2. Business Plan Alignment Worksheet with Definitions Business Plan ElementResults City of Austin Vision:We want Austin to be the most livable community in the country. Vision: (optional)Describes the desired future state or set of circumstances. Mission:Comprehensive statement of the Department’s purpose. Identifies Department’s primary customers and identifies the products or services that are provided. Goals:A broad statement describing the desired outcome for an organization or its programs. Defines the significant results to be achieved over the next 2 – 5 years. Program:Two or more activities grouped together to form a common purpose to define a program. Program Objective:Clear statement of the objective of the program. Program Results Measure(s)Key results this program is expected to achieve. Activity:A set of services with a common purpose that produce outputs and results for customers. Activity Objective:Clear statement of the purpose of the activity. Services that comprise the Activity: A service is a set of actions that produce a product, output, or result directly with or for customers. Activity Performance Measures: Results: The impact that an activity has on customers/citizens Efficiency: Unit cost of an output Demand: The amount of services requested or expected by customers of the activity Output: Units of services provided, products provided, or people served through the activity Responsible Employee:Department Executive/Manager responsible for Activity ALIGNMENT WORKSHEET BY ACTIVITY

25. 25 III. 2. Sample Business Plan Alignment Worksheet Business Plan ElementResults City of Austin Vision:We want Austin to be the most livable community in the country. Vision: (optional)Our community will be the healthiest in the nation. Mission:The purpose of the Austin/Travis County HHSD is to work in partnership with the community to promote health, safety, and well being. Goals:The over-all goal of the Austin/Travis County HHSD is to promote a healthy community which reflects social equity. This over-all goal will be achieved through: 1) Minimizing the public’s exposure to health and environmental hazards. Program:Environmental Health Services Program Objective:The purpose of the Environmental Health Services is to provide protection and enforcement service to the public in order to minimize environmental health hazards. Program Results Measure(s) Average response time to complaints/requests Confirmed cases of food-borne illness Percent of customers satisfied with complaint/request processing Activity:Health and Safety Code Compliance Activity Objective:The purpose of Health and Safety Code Compliance is to provide inspections, investigations, consultations, and training for the public in order to minimize public exposure to food-borne illness and other environmental health hazards. Services that comprise the Activity: Inspection services Investigation services Provide training to food operation employees Activity Performance Measures:Results: Confirmed cases of food-borne illness Efficiency: Cost per food establishment permit Average inspections/investigations per inspector Cost per food manager trained Output: Number of complaints/requests completed Number of food establishment, mobile food vendor inspections Number of temporary food inspections Responsible Employee:Donald Smith

26. 26 Review Team Budget Office, Organizational Development, City Management Structure Does it provide for alignment of results? Does it permit illumination of results and cost information in a manner useful to decision makers? Results Do objectives and measures match? Was template used for best impact? Measurability Are goals measurable? Are program and activity measures useful? III. 3. Reviews: Corporate Improvement and Accountability

27. 27 Program and Activity Objectives: MFR Template Performance Measures: A Family of Measures The purpose of ________________ is to provide___________________ to ___________________________ so they can __________________ Result Measure…then Outputs: How many? Efficiency: At what cost? Anticipated Demand III. 3. Plans: Consistent Process & Product

28. 28 The purpose of theCombat Operations (program) is to provide/produceemergency incident response (service or product) toanyone in the service area (customer) in order tosave lives and minimize property damage (planned benefit) III. 3. Activity Objective Statement (example)

29. 29 Result: Number of fire deaths per capita Percent of fires confined to the room or area of origin after arrival of AFD (per census track) Efficiency:Average cost per call Output:Number of calls (call volume) Demand: Number of fire alarms (calls) expected III. 3. Performance Measures (example)

30. 30 IV. 1. Link annual plans and budgets IV. 2. Establish targets IV. 3. Collect cost accounting information IV. Performance Budgeting

31. 31 Ensure clear linkage between the plan’s programs and the budget’s programs Ensure congruence between the plan’s goals, objectives, and targets and the budget’s goals, objectives, and targets IV. 1. Link Annual Performance Plans and Budgets

32. 32 In the Budget Document Business/Performance Plan Activity and Program Pages Performance Measures: definitions, etc. Using the Performance Budget to “Tell Your Story” Changing the Conversation This Result…At This Cost IV. 1. The Budget – Linking Results, $$$, and People

33. 33 IV. 2. Establish Targets Targets for each program and activity measure Sources of criteria for setting targets Historical trends and baselines Program requirements or intent Customer expectations or demands Industry or sector standards Benchmarking within the organization Benchmarking outside the organization

34. www.AuditorRoles.org34 Sources of Performance Expectations --The process for identifying expectations and setting targets should be rigorous. --All sources have pros and cons so all should be considered when setting targets.

35. 35 I.V. 2. Examples of Performance Targets and Measures Model ComponentTarget (Expectation)Measure (Actual) Input EconomyIn FY 2011, decrease the purchasing office’s personnel allocation by five positions. Number of purchasing office positions deleted in FY 2011. Process EfficiencyIn FY 2011, provide vehicle preventive maintenance services at the unit cost $500 or less per vehicle serviced. Average vehicle preventive maintenance unit costs in FY 2011. Output Quality (accuracy) In FY 2011, reduce the restaurant critical inspection error rate by 10 percent. Percentage reduction in the restaurant critical inspection error rate in FY 2011. Output QuantityIn FY 2011, expand “green energy” electrical services to 1000 additional homes and businesses. Number of additional homes and businesses in FY 2011 receiving “green energy.” Output TimelinessIn FY 2011, all Level 1 emergency calls will be responded to with a unit on site within six minutes. Response times (range) to Level 1 emergency calls in FY 2011. Outcome Effectiveness/ customer satisfaction In FY 2011, increase convention center customer satisfaction rate from 4.5 to 4.7 on a 5.0 scale. Change in convention center customer satisfaction rate during FY 2011..

36. 36 © Activity-Based Costing (ABC) © Identify Direct and Indirect Costs IV. 3. Base program budgets on unit costs that support desired program outputs and outcomes as reflected in targets

37. www.AuditorRoles.org37 IV. 3. Performance Budgeting Long-sought “ideal” of budgeting experts: Performance-driven budgeting. Best-case reality: Performance-informed budgeting.

38. 38 V. 1. Individual Performance Appraisal V. 2. Organizational Performance Assessment and Reporting V. 3. Performance and Measurement Certification Audits V. Performance Measurement and Reporting

39. 39  Every employee in the organization contributes to the City Vision  Every employee in the department contributes to the Mission of the department.  Every employee in the department contributes to at least one Business Plan Goal.  The Alignment Worksheets show employees how the Services they provide support specific Activities, Programs, and Goals in the Business Plan.  Performance Measures show citizens, City Council and employees how well we are doing.  Every Business Plan Measure must be written into at least one employee’s SSPR.  Every employee, including department executives, will have at least one Business Plan Measure in their SSPR. V. 1. Establishing Accountability Key Points of Business Plan Alignment/ SSPR Integration

40. 40 Alignment Worksheet MissionProgram Objective Goals Program – Activity – Services that comprise Activity Activity Objective  Activity Performance Measures Activity Results Measure Results: Efficiency: Description of Services Demand: Output: Individual Performance Measure Same as the Activity Performance Measure Part of the Activity Performance Measure or, Contributes to the Activity Performance Measure Employee SSPR V. 1. Individual Performance Appraisal

41. 41 Management Data Component Analysis Component Action Component Performance Goals Program and Levels Performance Indicators Intended Uses Data CollectionData Processing Measurement of Current Performance Levels Comparison of Current Performance with Criteria (performance goals) Decisions Concerning Goals Decisions Concerning Programs and Levels Decisions Concerning Monitoring & Evaluation V. 2. What is a Performance Monitoring System?

42. 42 Design monitoring system to track and analyze the selected measures (efficiency, outputs, and outcomes are essential). V. 2. Ensure Performance Measure Definitions/Formulas are Established

43. 43 V. 2. Ensure the Results of Performance Measures are Available for Analysis and Decision Making Design a reporting system that is easy to use, accessible to all interested parties, and enables management decisions.

44. 44 Design reporting formats and decide frequency of reporting. Austin reports include: Quarterly Performance Reports Annual Performance Reports Community Scorecard V. 2. Establish Performance Reporting “Best Practices”

45. 45 Use performance reports to identify and direct analysis of program performance Use analysis to identify the causes of inadequate program performance and focus improvements on causes Use performance reports to identify high performance programs V. 2. Use Performance Reports to Improve Performance

46. 46 V. 2.1. City of Austin Performance Report Departmental Performance Measures Total of 115 Measures Grouped into Public Safety, Community Services, Infrastructure, and Utilities/Enterprise Departments Each performance graphic includes: Measure Description, Calculation Method, Results, Assessment of Results, Next Steps, and Contact for More Information

47. 47 V. 2.1. City of Austin Performance Report Decisions influenced by: Stakeholder/citizen priority or demand Stakeholder/citizen satisfaction Results shown City Council and Management priorities

48. 48 V. 2.2. Why the City of Tallahassee Supports Citizen Centric Reporting Government officials have a responsibility to be good stewards, to spend monies provided wisely, and to report financial and performance information back to citizens on accomplishments and challenges. Citizen reporting has its history in Efficient Citizenship Theory Citizens are not the customers of government, they are the owners of the government Democratic government is best shaped by the choices of well informed citizens

49. 49 V. 2.2. Why is the City of Tallahassee Issuing a Citizen Centric Report? We have a responsibility to inform our citizens about: What we are responsible for doing Where the money comes from that runs the City and where it goes What we have accomplished with monies received and expended, and What challenges face the City moving forward We believe informed citizens make for better government

54. 54 V. 2.2. Issuance of Citizen Centric Reports and Media Coverage Four Citizen Centric Reports have been issued Received front page newspaper coverage Received television coverage Report page 3 data verified by City Auditor Citizen groups have received reports and have been requested to provide audit topic suggestions Reports are available in hard copy and online

55. 55 V. 2.2. Distribution of Citizen Centric Reports City Neighborhood Associations City Advisory Committees Tallahassee Regional Airport Community and Senior Centers Other governments to include the County, FSU, FAMU, TCC League of Women Voters Leon County District Schools Middle School Civics Program

56. 56 V. 2.2. Purpose of the Citizen Centric Report To Demonstrate: – Transparency – Accountability To Promote – Dialog – Two way communication To Build Trust – Over time – One citizen at a time

57. 57 V. 2.2. Building on to the Citizen Centric Report Performance Measurement Report – Issued August 2011 – 83 pages – Includes all major departments – Report contents for each department Mission statement, organization and services provided Goals and objectives Key performance measures Accomplishments and challenges Expenditures by program and line item – Report includes tables, graphs, charts, and photos

59. 59 V. 2.2. Website and Contact Information Citizen Centric Report http://www.talgov.com/auditing/pdf/citizenreport2010.pdf Performance Measurement Report http://www.talgov.com/auditing/ sam.mccall@talgov.com (850) 891-8397

60. 60 V. 3. Conduct Performance and Measurement Audits Audit departmental and program performance Audit relevance and reliability of performance measures

61. www.AuditorRoles.org 61 V.3.1. Auditing Government Performance Measure or assess performance during an audit or other study based on authoritative auditing standards. (See Austin, Florida OPPAGA, Kansas City, Phoenix on www.AuditorRoles.org) — Identify the program’s inputs, processes, outputs, and outcomes — Develop and implement “ad hoc” performance measurement system — Using performance expectations as “criteria” and measures as “condition,” analyze program performance — Identify causes of variances and develop audit recommendations

62. Available at: www.theiia.org/bookstore

63. V.3.2. Self Assess or Audit Performance Measures Using Asserted Criteria Relevance—Measures should be aligned, complete, and useful Reliability—Each measure and its data should be accurate, valid, and consistent www.AuditorRoles.org63

64. www.AuditorRoles.org64 Aligned Linked to mission, goals, objectives Complete Includes essential aspects of performance Useful Timely Understandable Comparable Responsive to change Meets broad needs of users Measures should be: V.3.2. Test relevance or reliability. Assessing the Relevance of Performance Measures: Sample Criteria

65. www.AuditorRoles.org65 Accurate Computed correctly Neither overstated nor understated Appropriately precise ValidCorresponds to the phenomena reported Correctly defined Data & calculation comply with definition Unbiased Consistent Consistent with previous periods Controlled by adequate systems Each measure and its data should be: V.3.2. Test relevance or reliability. Assessing the Reliability of Performance Measures & Data: Sample Criteria

66. 66 VI. Performance-Based Decision Making – Includes Stakeholders, Elected Officials, Managers, and Employees VI. 1.Using performance information to support decision making VI. 2. Examples of decision making

67. 67 VI. 1. Performance Information Used for Different Decisions Budgetary Decision making Manage & Improve Operations Accountability Reporting

68. 68 ©Assess and adjust program performance service levels, and resources, ©improve existing programs and services, ©improve internal management systems, ©revise performance plans and reports, ©initiate new programs and services, and ©bottom-line—use performance information to support continuous improvement and public accountability. VI. 1. Performance Information Supports Decisions to:

69. www.AuditorRoles.org69 V. 1. Strategic Performance Budget Decision Model Performance Results LOW GOOD HIGH Strategic Success in Achieving Community or Program Outcomes Useful Contributor to Government Success Target for Increased Funding Target for Funding Cuts POOR Use of Strategic Goals and Performance Results in Prince William County Budget Decisions Strategic Importance

70. www.AuditorRoles.org70 VI. 1. Strategic Performance Budget Decision Model: In Reality: Varies with Fiscal Environment Performance Results LOW GOOD HIGH POOR Strategic Success in Achieving Community or Program Outcomes Useful Contributor to Government Success Target for Increased Funding Target for Funding Cuts Use of Strategic Goals and Performance Results in Prince William County Budget Decisions Less funding available More funding available Strategic Importance

71. 71 VI. 2. Output Effectiveness (Quantity) Recreation Centers

72. 72 External Costs: City/County corporate overhead (payroll, human resources, treasury, controller’s office, etc.) along with capital improvement projects, rent, depreciation, etc. Internal Costs: Physicians, nurses, medical supplies, pharmacy, etc. Health Clinics - Adjusted total cost comparison among clinics for billable and non billable encounters combined. VI. 2. Health Clinics (efficiency) FY10 Target FY 09 Average

73. 73 VI. 2. Examples of Performance-Based Decision Making (Inputs & Mission)

74. 74 Supporting our Vision... Creating A Results Orientation: Services, Activities and Programs Creating Accountability: Measures and Indicators Creating Integration: Making it Happen at the Operational Level VII. Conclusion: Creating a Managing for Results Culture

75. 75 ©“Bottom-Up” approach neglected broad performance areas and alignment Department key indicators ©Accounting structure is a major hurdle ©Definition of “services” not clear ©Results orientation difficult when template not used effectively ©Poor use of template = poor measures ©The “not something I control” syndrome VII. Lessons Learned

76. 76 Assessing and Improving the Reliability of Reported Measures Data Collection Infrastructure Certification Program Providing Further Training Using Information in Management Using Information in Operations Re-enforce Cultural Shift at the Operational Level Passing and Implementing a “Best Practice” Performance Accountability Ordinance VII. Where we are going…

77. 77 VII. Performance Measurement and Accountability: Best Practices Checklist Obtain active participation by top-level managers and decision makers Create a clear vision of why and how performance measures will be used internally and externally Understand the limits of performance measures what they can and cannot do Sustain organizational commitment over a long period despite barriers and the potential for bad news. Integrate the performance measurement and reporting system with organizational planning, service delivery, and decision making systems Through planning align mission, goals, objectives/targets, and measures Design goals and objectives/targets that specify a single aspect of performance Design aggressive yet realistic goals and objectives/targets that encourage progress beyond past performance levels Involve employees, customers, and stakeholders in developing goals, objectives/targets, and measures

78. 78 VII. Performance Measurement and Accountability: Best Practices Checklist (Continued) Identify all programs which will be measured and define them through an input process/activity output outcome model Design a “family of measures” for each program which provides key information to support decisions Periodically evaluate current performance measure; change when needed but try for comparability over time Define each measure and identify data sources and data collection procedures Produce performance information (including explanatory information) which is clear and useful to management, employees, customers, and stakeholders Educate, encourage, and reward managers for using performance information to make decisions which improve program management and service delivery

79. Training/Assistance to Get There EGAPP, Inc. provides training in all aspects of performance management and auditing. (Brochure Available) Auditor Roles Project provides training in assessing/auditing performance management systems and measures. Assistance can also be arranged. Email to: egappmorgan@yahoo.com 79

80. Thank You. More questions. More comments. Thank you, again. 80