Presentation on theme: "January 2014 – Public Release 1.0 Performance Benchmarking for Effectively Managed Water Utilities (Project 4313) User Guide for the Self-Assessment Tool."— Presentation transcript:
January 2014 – Public Release 1.0 Performance Benchmarking for Effectively Managed Water Utilities (Project 4313) User Guide for the Self-Assessment Tool
Updated Page 2 Overview The Self-Assessment Tool includes Practice Areas for each of the 10 EUM Attributes as well as key metrics to measure performance for each of those practices. The Tool enables utilities to select and weigh their relevant attributes and practices, and then assess those practices for both current and target performance. This guide explains the steps necessary to use the tool and leverage its capabilities. For additional information on Effective Utility Management, please visit
Updated Page 3 Select Attributes Select Practice Areas Select Performance Measures Create Custom Assessment Assess each Performance Measure Verify Assessment Review Attribute Summaries Return to Setup Delete Performance Measures View Summary Results Model Roadmap Each of the following steps will be addressed in the guide:
Updated Page 4 Opening the Tool 1.When the Tool is opened, a simple disclosure “splash screen” is presented. Exit the application or start the EUM Tool by clicking the associated button.
Updated Page 5 Attributes 1.Weigh each Attribute in terms of relative importance to the utility compared with the other selected Attributes. If a weight of 0 is selected, the Attribute will not be included in the assessment; otherwise scales of 1-10 or are examples of weighting options that will work. 2.After weighting, click this button to navigate to the next screen. Select Attributes Select Practice Areas Select Performance Measures Create Custom Assessment Assess each Performance Measure
Updated Page 6 Practice Areas 1.Only Attributes that have been weighed appear on this screen. Next, weigh the relative importance of each Practice Area that should be included in the assessment. As with the Attributes, if a weight of 0 is selected, the Practice Area will not be included in the assessment; otherwise a scales of 1-10 or are examples of weighting options that will work. 2.After weighting, click this button to navigate to the next screen. Select Attributes Select Practice Areas Select Performance Measures Create Custom Assessment Assess each Performance Measure
Updated Page 7 Performance Measures 1.Enter “Y” or “N” to select Performance Measures that should be included in the assessment. Select Attributes Select Practice Areas Select Performance Measures Create Custom Assessment Assess each Performance Measure 2.After choosing the Performance Measures, click this button to create the customized self- assessment.
Updated Page 8 Select Attributes Select Practice Areas Select Performance Measures Create Custom Assessment Assess each Performance Measure 2.The Tool also creates summary sheets for each of the selected Attributes containing the current and target scores along with the weighted totals. 3.The next step is to select these links or navigate through the Performance Measure tabs to assess each Performance Measure. 1.The Tool creates a scoring matrix for each of the Performance Measures under each Attribute. The worksheet names are coded using the Attribute #, Practice Area #, and Performance Measure #. Customized Menu
Updated Page 9 Select Attributes Select Practice Areas Select Performance Measures Create Custom Assessment Assess each Performance Measure 1.The column and row headings can be edited by double- clicking the appropriate cell if you’d like to modify the pre- defined criteria descriptions. 4.Clicking the squares within the matrix will turn on/off “Current,” “Target,” or “Current & Target.” 5.Select current performance before selecting a target. Clicking activated squares will reset them for re-scoring. 6.After completing all of the performance measure assessments, click on 2.Find the intersection between the correct level of performance with degree of implementation for your utility. Click to enter your current performance and repeat the process to enter the target. Perform Assessment 3.In defining the performance measures, some rows and columns were intentionally left blank. Please use caution to avoid selecting the blank rows/columns.
Updated Page 10 Verify Assessment Review Attribute Summaries Return to Setup Delete Performance Measures View Summary Results 1.Clicking this button will test to see if there are missing data that must be entered before continuing. 2.Use these links to navigate to specific Attribute(s) for the next step. Verify Assessment
Updated Page 11 Verify Assessment Review Attribute Summaries Return to Setup Delete Performance Measures View Summary Results 1.The first potential error is that a Performance Measure has not yet been assessed. This is resolved by navigating to the appropriate Performance Measure(s) and recording the current and target performance. 2.The second potential error is that a Practice Area has been weighted, but no Performance Measures have been selected. This is resolved by either setting the Practice Area weight to zero or entering “Y” in one or more Performance Measures related to that Practice Area. 3.The third potential scoring error is that an Attribute has been weighted, but no Practice Areas have been selected. This is resolved by either setting the Attribute weight to zero, or entering a numerical weight for one or more Practices Areas related to that Attribute, and then entering “Y” in one or more Performance measures related to each selected Practice Area. Potential Assessment Errors
Updated Page 12 Review Attributes The individual Performance Measures are listed, with links to their sheets. If necessary, click the links to revise assessments. Verify Assessment Review Attribute Summaries Return to Setup Delete Performance Measures View Summary Results Current and target performance scores are listed based on the matrix selections. Practice Area scores (arithmetic mean of Performance Measure scores) are listed in the table and illustrated in the graph.
Updated Page 13 The following two slides illustrate the process that can be used to add/remove assessment elements and to delete Performance Measures, respectively. Removing assessment elements takes them out of the assessment summaries but does not delete the underlying Performance Measure sheets and scoring from the tool; Deleting Performance Measures deletes the underlying Performance Measure sheets and scoring from the tool. Removing vs. Deleting
Updated Page 14 Return to Setup Verify Assessment Review Attribute Summaries Return to Setup Delete Performance Measures View Summary Results If you determine that it is appropriate to add or remove elements from what is shown in the assessment, you can add or remove Attributes, Practice Areas, and Performance Measures, even after scoring, by selecting the element and making appropriate adjustments.
Updated Page 15 Delete Performance Measures Verify Assessment Review Attribute Summaries Return to Setup Delete Performance Measures View Summary Results 1.If it is determined that certain Performance Measures are no longer needed, click this button to remove them. 2.This pop-up window will appear with each of the Performance Measures listed. Click the check boxes for the Performance Measures you’d like to remove and then click OK. 3.This process does not affect the settings on the setup menu; therefore, in order to prevent the deleted Performance Measure(s) from reappearing, they should be switched from “Y” to “N” in the setup menu.
Updated Page 16 Review Results After the assessment has been completed, the summary results for each Attribute (based on the Practice Area weights) are listed and illustrated on the graph. The total weighted utility score is also provided, based on the specific Attribute weights. Verify Assessment Review Attribute Summaries Return to Setup Delete Performance Measures View Summary Results
List of Performance Measures
Updated Page 18 1.Product Quality 1.Comply with Regulatory and Reliability Requirements 1.Degree of Success in Achieving Target Drinking Water Compliance Rate (percent) 2.Degree of Success in Achieving Target Wastewater Treatment Effectiveness Rate (percent) 3.Degree to Which Near Compliance Misses are Minimized (water) 4.Degree to Which Near Compliance Misses are Minimized (wastewater) 5.Meet National and Other Applicable Objectives for Residuals Quantity and Quality 6.Extent to Which Supplemental Local Goals are Addressed 2.Address Customer Needs 1.Achievement of Target Drinking Water Flow and Pressure (percent) 2.Achievement of Target Fire Suppression Flow and Pressure (percent) 3.Extent to Which Service Interruptions are Reduced to Target Levels (percent) 4.Extent to Which Sewer Backups are Reduced to Target Levels (percent) 5.Extent to Which Sanitary Sewer Overflows are Reduced to Target Levels 6.Extent to Which Combined Sewer Overflows are Reduced to Target Levels 7.Extent to Which Safe Drinking Water Act Secondary Standards are met 3.Address Public Health and Ecological Needs 1.Achievement of Water Reuse Targets (percent) 2.Achievement of Beneficial Biosolids Usage Targets (percent)
Updated Page 19 2.Customer Satisfaction 1.Minimize Customer Complaints 1.Extent to Which Customer Service Complaint Rates Fall within Target Levels 2.Extent to Which Technical Quality Complaint Rates Fall within Target Levels 2.Achieve Target Level of Customer Service Delivery 1.Degree of Field Call Responsiveness (water) 2.Degree of Field Call Responsiveness (wastewater) 3.Degree to Which Error-driven Billing Adjustments Are Minimized 4.Degree to Which Service Start/Stop Responsiveness Falls Within Target Levels 5.Extent to Which Customer Issues are Resolved by Call Center/Customer Service at the time of the First Call 6.Degree to Which Customer Service Responsiveness Falls Within Target Levels 7.Degree of Abandoned Calls 3.Receive Positive Customer Perceptions 1.Degree of Positive Customer Feedback Received 2.Perception of Effectiveness at Dispute Management and Resolution 4.Efficiently Deliver Customer Service 1.Efficiency of Customer Service (Ratio of Cost/O&M Spending)
Updated Page 20 3.Employee and Leadership Development 1.Recruit Appropriate Talent 1.Degree of Success in Implementing a Recruitment Strategy 2.Extent to Which Job Descriptions are Implemented 3.Adequacy of Interview and Selection Processes 2.Retain Existing Talent 1.Degree of Implementation of Retention Management Plans 2.Degree of Implementation of Onboarding Programs 3.Degree of Implementation of Job Satisfaction Programs 4.Degree of Success in Employee Engagement 3.Address Succession Planning Needs 1.Adequacy of Workforce Forecasting 2.Adequacy of Succession Plan 4.Strengthen Core Competencies 1.Degree of Success in Implementing Performance Management Systems 2.Degree of Implementation of Learning Programs 3.Level of Management Training Achieved
Updated Page 21 4.Operational Optimization 1.Provide for Ongoing Operational Improvements 1.Track Record in Providing for Ongoing Operational Improvements 2.Extent to which Operational and Technology Improvements Are Deployed 3.Extent to Which Automated Regulatory Monitoring & Reporting Systems are Implemented (Water) 4.Extent to Which Automated Regulatory Monitoring & Reporting Systems are Implemented (Wastewater) 2.Minimize Resource Use and Losses from Day to Day Operations 1.Degree of Implementation of Resource Optimization Plans 2.Degree of Energy Optimization 3.Degree of Labor Optimization 4.Degree of Chemicals Optimization 5.Degree of Residuals Optimization 6.Degree of Maintenance Optimization
Updated Page 22 5.Financial Viability 1.Develop Sound Financial Plan 1.Degree to Which Financial Planning Efforts Support Strong Bond Ratings 2.Degree to Which Rate Increase Level is Supported by Planning Studies 3.Appropriateness of Rate Planning Horizon 4.Appropriateness of Balance of Capital Spending Between Debt and Equity Expenditures 2.Provide Financial Integrity 1.Financial Policy Integrity 2.Financial Procedural Integrity 3.Achieve Budget Management Effectiveness 1.Appropriateness of Annual Revenue to Expenditure Ratio (operating) 2.Appropriateness of Annual Revenue to Expenditure Ratio (capital) 3.Appropriateness of Annual Revenue to Expenditure Ratio (Overall) 4.Degree to Which Long-term Life-cycle Accounting Employs Risk-Based Considerations 5.Adequacy of Operating Reserves
Updated Page 23 6.Infrastructure Stability 1.Develop and Implement an Asset Management Program 1.Degree of Implementation of an Asset Management (AM) Framework 2.Degree of Implementation of Levels of Service 3.Degree of Development of an Asset Management Plan 2.Maintain Knowledge of Assets and Costs 1.Level of Asset Inventory 2.Level of Asset Condition Information 3.Level of Asset Performance Information 4.Availability of Cost of Asset Ownership Information 3.Incorporate Risk-Based Analysis into Decisions 1.Degree to Which Risk of Failure Analysis is Incorporated in Decision-Making 2.Extent to Which Critical Assets are Identified 3.Extent to Which Risk Mitigation Options are Identified and Evaluated as Part of Decision-Making
Updated Page 24 7.Operational Resiliency 1.Incorporate Risk Assessments into Decision-making 1.Extent to Which Vulnerability Assessments (VAs) are Conducted and Recommendations Implemented 2.Implement Risk Mitigation 1.Emergency Response and Recovery Planning 2.Business Continuity Planning / Continuity of Operations Planning 3.Crisis Communications 4.Business Risk 3.Sustain Employee Resiliency 1.Achievement in Employee Injury Time Reduction 2.Health and Safety Training 3.Health and Safety Plan
Updated Page 25 8.Community Sustainability 1.Utility Organization 1.Leadership 2.Sustainability Plan 3.Management System 4.Sustainability Reporting 2.Infrastructure Project Sustainability 1.Project Planning, Design, and Engineering 2.Infrastructure Procurement 3.General Procurement 3.Natural Environment 1.Water Management 2.Water Conservation 3.Stormwater 4.Energy 5.Waste Reduction 6.Ecosystem Services 7.Pollution Prevention 8.Climate Change Adaptation 4.Economic Strength 1.Economic Competitiveness 2.Local Sourcing 3.Local Payroll 5.Social Equity 1.Use of Strategic Financial Planning to Minimize Overall Rate Impacts 2.Appropriate Use of Affordability Programs to Protect Specially Impacted Groups 3.Degree of Success in Keeping Customer Bills Within Affordable Levels 4.Community Support
Updated Page 26 9.Water Resource Adequacy 1.Achieve Water Supply Adequacy 1.Adequacy of Long-Term Water Supply 2.Adequacy of the Ratio of Short-term Supply to Short-term Demand (12-mo rolling average) 3.Degree of Implementation of a Basin Supply Plan 4.Degree of Implementation of Drought Management Plan 5.Degree of Implementation of a Source Water Protection Plan 2.Optimize Reduction of Non-Revenue Water 1.Degree of Success in Achieving Target Levels of Annual Non-revenue Water (NRW) 2.Degree of Success in Achieving Target % of Real Water Loss 3.Implement Water Conservation 1.Degree of Success in Achieving Target Per Capita Use 4.Achieve Water Supply Reliability 1.Degree of Implementation of Demand Management/Demand Reduction Plans 2.Frequency of Full Satisfaction of Demand 3.Consideration of Supply/Demand Uncertainty 4.Degree of Success in Managing Raw Water Turbidity to Target Levels
Updated Page Stakeholder Understanding and Support 1.Stakeholder Identification 1.Degree of Success in Stakeholder Identification and Analysis 2.Stakeholder Engagement Plan 1.Degree of Success in Developing and Executing a Stakeholder Engagement Plan 3.Oversight Body Engagement Strategy 1.Success in Gaining Oversight Body Understanding 4.Media Interaction Program 1.Degree of Success in Interacting with Media (Traditional and Emerging) to Achieve Accurate Coverage 2.Degree of Success in Establishing Positive Coverage by the Media (Traditional and Emerging) 5.Stakeholder Support Performance Measurement System 1.Level of Stakeholder Support Received for Critical Strategic and Operational Directions