1. Logic Model Training City of Bellevue February 26, 2013 Brian Willett, Performance Analyst

2. Today’s outline 2  Introduction – Fun with Logic Models  Defining your process and system  Practice defining a system  Ken Miller video clip #1 – Widgets  Relevance – Matching messages to interests  Outcomes  Purpose and structure of a logic model  Fill-in the blanks logic model - Handout  Building your own logic model - Handout  Testing your logic model  Ken Miller Video Clip #2 – Customers (Extra Time)

3. What we’ve learned about performance measures  You can’t afford to measure everything.  Too many measures tend to be, “True, but useless.”  Performance measures must be RELEVANT.  Measuring stuff by itself doesn’t improve anything. 3  Just because you can measure it, doesn’t mean you should.  Performance measurement, reporting, and performance management sound similar, but are very different things.

4. Define your process Input - Funding Input - Information Input – Others’ work Output – What you make Permits Contracts Plans Reports Citations Input – Raw materials Rework Scrap Waste 4 Process – What you do Methods Tools Rules/Regulations Staffing Environment Materials Process – What you do Methods Tools Rules/Regulations Staffing Environment Materials

5. Terms & definitions  Outcomes – The purpose of the system. Why does anyone perform this work in the first place? What should be better if this work is done well?  Outputs (Widgets) – The product that is actually delivered or produced.  Customers – The people, organizations, or work units that actually use the product or service produced or delivered.  Process – The way work gets done or the way service is delivered.  Rework – The time and expense needed to fix the inputs so they are usable for the process, or to fix the outputs so they are acceptable to the customer.  Scrap/Waste – Byproducts or leftovers created by the process.  Inputs – The raw materials/information needed to do the work or provide a service.  Suppliers – The people, organizations, or work units that provide the materials/information needed to perform the work. 5

6. Expand your process to define the system Process OutputsInputs Users & Brokers (Customers) Suppliers Outcomes: Organizational Benefits User Benefits Financial Stakeholder Benefits 6

7. Washington State Ferry System Example InputsOutputsProcessCustomers Desired Outcomes Efficiency Suppliers & Stakeholders Planned and actual ridership levels Planned vs. actual farebox revenue levels Total cost per sailing (by run) Number & causes of missed trips (Error rate) Number & type of complaints per 100k customers State Legislature Employees Unions Ship builders Vendors Fuel vendors Island/Sound residents & business owners Department of Ecology Transportation Commission Transportation funds Fare box revenue Employee labor Diesel fuel Rules, regulations & permits Strategic plans Loading & unloading Sailing Docking/Departure Snack bar operations & vending Fare collection Boat maintenance & repair Facility construction, maintenance & repair Sailings/Trips Snacks, food, & drinks sold Repairs to boats & terminals Ferry riders (commuters & tourists) Adjacent property & business owners Increase timeliness Increase revenue Increase customer satisfaction Increase efficiency Increase ridership Improve safety (employees & passengers) Reduce wait times Decrease CO2 emissions On-time performance rates Number of sailings/trips Number of passengers Number of vehiclesPercent of state ferry terminals in fair or better condition (by system) Metric tons of CO2 emissions Number of marine and terminal reported injuries Average vehicle wait time on the dock during peak hours (by route) Ship maintenance costs (by route) Vending machine & cafeteria profits Number & type of reported employee injuries Number, type & cost of reported vehicle damage claims Number and cost of boat and terminal maintenance backlog issues Toxic spills – Number, type, cause, quantity spilled, clean up costs, & fines paid Total amount of diesel fuel purchased Legend Currently published performance measures Unreported, but current measures Possible additional performance measures Number of ferry life- cycle preservation activities completed 7

8. Ken Miller Video Clip #1 Widgets 8

9. Define your own system exercise - Handout 9

10. Can you answer these questions with data, not stories:  What do you make/produce/deliver?  How many do you make/produce/deliver?  How long does it take from start to finish to make it?  How much does it cost you to make it?  What do you measure to tell you that you are effective?  How do you know your customers are happy with what you make and the service you provide? In each case, you should provide actual data over time, a baseline, and a target, if applicable. 10 Questions from a Performance Auditor

11. A thing or two about outcomes 11  You rarely work directly on them.  You can’t control them.  Too many other things impact performance.  You can only influence them.  They can be difficult to measure.  Citizens and financial stakeholders want to hold you accountable for achieving them.  You must be prepared to report progress on achieving them.

12. Purpose of a Logic Model 12  Graphically illustrate the theoretical connection between what you actually do (and what you can control) to the desired outcomes you are supposed to influence.  Help identify relevant performance measures to track progress.

13. Structure of a line-of-sight Logic Model 13

14. 14 Example Logic Model – What are we doing here today?

15. Make a Logic Model for the system defined earlier Handout – Logic Model Template 15

16. Logic Model testing 16  Does the “So that…” chain of objective statements seem reasonable?  Do the outcomes match the purpose of the program (Legislative intent, mission statement)?  Are the performance measures actually measures (#, %, $)?  Does that data currently exist?  Are those the right measures?  What factors beyond the control of the program impact performance?

17. 17 How many measures should we have? Balance is more important than number, but most experts agree that no work unit can effectively manage more than 10 to 15 performance measures. Try this on for size:  One or two measures to tell if the work unit is accomplishing its mission.  One or two measures to tell if the work unit’s customers are happy. (timeliness, quality, compliance, complaints)  One or two measures related to finances. (unit cost, efficiency, revenue generation)  A few measures to tell if the work is being done well. (output counts, process cycle times, waste, defect rates)  One or two measures to track progress on support/change initiatives. (training completion, employee evaluations, IT/web usage)

18. 18 Bonus Time Ken Miller Video Clip Customers

19. Performance Measures, T.S. Marshall, 1990. The New Economics, Dr. W. Edwards Deming, 1994. The Leader’s Handbook, Peter Scholtes, 1998. We Don’t Make Widgets, Ken Miller, 2006. Extreme Government Makeover, Ken Miller, 2011. A Performance Management Framework for State and Local Government, National Performance Management Advisory Commission, 2010. http://www.pmcommission.org http://www.pmcommission.org 19 Additional reading and references

20. 20 More Information? Help? Brian Willett Performance Analyst (360) 725-9731 Brian.Willett@sao.wa.gov Larisa Benson Director of Performance Audit (360) 725-9720 Larisa.Benson@sao.wa.gov Website: http://www.sao.wa.gov/PerformanceCenterhttp://www.sao.wa.gov/PerformanceCenter Twitter: www.twitter.com/LocalGovPerformwww.twitter.com/LocalGovPerform Local Government Performance Center