1. © 2005 Klubeck, Langthorne, & Padgett DIY Metrics Copyright Martin Klubeck, Michael Langthorne, Don Padgett 2006. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors

2. © 2005 Klubeck, Langthorne, & Padgett D.I.Y. Metrics Your Presenters Martin Klubeck 574-631-5447 Don Padgett 574-631-3556 Michael Langthorne 574-631-8772

3. © 2005 Klubeck, Langthorne, & Padgett Metrics What are metrics? What is data? What is a measure? What is information? What is a metric?

4. © 2005 Klubeck, Langthorne, & Padgett Help Desk Example Number of trouble calls Number of opened cases Number of closed cases Number of employees Number of survey responses Number of calls per hour Number or cases closed by worker Number of calls for each hour compared to number of workers on shift. Average length of time to close a case, grouped by type Average customer satisfaction rating Data Information Measures

5. © 2005 Klubeck, Langthorne, & Padgett Help Desk Example (cont) Aug Sep Oct Nov Dec Jan FebMar Apr May Jun Jul manning Average number of open cases from 1999-2002 0 25 50 100 125 150 175 200 225 0 5 6 7 8 9 10 11 12 Explanation: The manning over the academic year was not in line with the number of trouble calls received – based on data collected over the last three years. Result: We’ve re-aligned our manning for the coming year to match the level of need each month. manning open cases Max manning Min manning METRIC

6. © 2005 Klubeck, Langthorne, & Padgett Metrics Metric Name Purpose (a go/no go proposition) Metric Area/Category Customer Graphical Representation Explanation Metrics Analysis Measures used to develop metric Collection Schema Schedule Assumptions and Constraints Related Metrics and Data Dependencies Lessons Learned

7. © 2005 Klubeck, Langthorne, & Padgett Implementation Guide

8. © 2005 Klubeck, Langthorne, & Padgett 1. Metric Name: Quantitative or Qualitative Definition: Summary: History: 2. Purpose: The information need that drove the development of the metric. What was the root question? Why do we want to tell the story? What do we hope to achieve?

9. © 2005 Klubeck, Langthorne, & Padgett 3. Metric Area/Category: This is the Metric Area/Category - any way you want to group the metric (Dept/Office, function, customer, etc.) One method is to list the information of the Quality Box 4. Customer: Who will you present the metric to? The customer could be you

10. © 2005 Klubeck, Langthorne, & Padgett 5. Graphical Representation: What Charts/Graphs will be used to tell story? – trends, Pareto, benchmarking, bar, line, etc.): Can be hand drawn in the early stages of development. Insert actual graphical representations when reach that phase of the implementation 6. Explanation: A prose version of the story the metric tells. Using the analysis above, explain how the metric should be interpreted. How to read the picture? Include how the metric will be used - and how it will NOT be used. Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul manning Average number of open cases from 1999- 2002 0 25 50 100 125 150 175 200 225 0 5 6 7 8 9 10 11 12 Max manning Min manning manning open cases

11. © 2005 Klubeck, Langthorne, & Padgett 7. Metrics Analysis: This is where we log any formulas or mathematical equations used. Goals and thresholds also qualify. Format: Threshold: Target: Derived Information: 8. Measures used to develop the metric: The specific data points to be collected and used to develop the metric. The lowest-level view of the data. Data & Measures:

12. © 2005 Klubeck, Langthorne, & Padgett 9. Collection Schema: 10. Schedule: Be as detailed as possible in this section - it should be a guide for the collector to follow Collector: Who will have the joy of actually collecting the data? The less human interaction, the better Source: Where is the information (data points or measures) available from? Frequency: How often should the data be collected/reported? When? Methodology: How does it get collected? Is it automated? Where is it stored? Milestones in the collection, reporting, and use of the metric should be tracked here Estimated completion dates. This helps to show the history of the metric.

13. © 2005 Klubeck, Langthorne, & Padgett 11. Assumptions, constraints, and any known flaws: 12. Related Metrics and Data Dependencies: Log any assumptions made, like data will be available from the source when needed. Also track any constraints you expect to have to deal with such as lack of resources. Any related metrics that should be looked at with this metric to tell a bigger story. Metrics rarely should be looked at in isolation

14. © 2005 Klubeck, Langthorne, & Padgett 13. Lessons Learned: You will eventually log lessons learned from the activity of collecting, reporting, and using the metric here. You can also log lessons learned from the metric itself. It would be useful for you to also log any successes or accomplishments in the improvement arena as a result of the metric. Did the metric meet its purpose?

15. © 2005 Klubeck, Langthorne, & Padgett Discussion / Questions

16. © 2005 Klubeck, Langthorne, & Padgett http://oit.nd.edu/about/Tools.shtml