1. © Richard Mironov, 2001-02. All rights reserved. www.mironov.com rich@mironov.com Tel: 650.315.7394 Service Metrics Measure, improve, measure, improve…

2. www.mironov.com rich@mironov.com 650.315.7394 2 Service Metrics  Decide what’s important  Quantify it  Measure compulsively  Root cause analysis  Love your customers

3. www.mironov.com rich@mironov.com 650.315.7394 3 What’s Important to Users?  Each service has a few fundamental attributes  Postpone less important metrics Examples:  Email: server up, addresses work, mail saved  Next tier: delivery time, disk limits, virus scans  Newswire filtering: automatic, easy, broad search base, inexpensive  Next tier: uptime, speed, delivery method

4. www.mironov.com rich@mironov.com 650.315.7394 4 Specific and Narrow  Easy to paint universal goals  “All functions must be up 99.995%”  Instead, focus on critical service elements first  Example: Uptime for corporate directory  Sub-second password look-up needed 24x7  OK to wait 5 minutes for password updates  New account creation needed only during normal work hours  Backup must be done within 8 hours, by midnight  Remote disaster recovery can use last Friday’s data

5. www.mironov.com rich@mironov.com 650.315.7394 5 Take the Customer’s View  Metrics must be external, as customers experience them  Example: ISP dial-in success rate  What: fraction of attempted calls that get connected  Metric: % success by hour, day, phone number  Tool: “war room” dialers that ring each number 20x daily  Will also spot outside problems like telco outages  Best tools capture user results directly

6. www.mironov.com rich@mironov.com 650.315.7394 6 Quantify and Measure  Ongoing analytical effort  Operations and finance have best talent pools  Daily review with Operations VP  Charts, trends, incident reports  What happened? How can we automate prevention? Where can we simplify?  Weekly summary to Executive Staff Let’s follow an extended example...

7. www.mironov.com rich@mironov.com 650.315.7394 7 1. Customer-Reported Outages  When customers call in, you have already failed

8. www.mironov.com rich@mironov.com 650.315.7394 8 2. Design Metric and Test Tool  Goal: working 99.5%, no transactions lost  Hours of operation: 8AM ET to 6PM PT  Tool: external “robot” submits request every 90 seconds  Notices simple failures…  T-1 down, power out, CPU down, disk full, account closed  Does not test everything…  Value of service, browser versions, data speeds, pricing, network security, tech support helpfulness

9. www.mironov.com rich@mironov.com 650.315.7394 9 3. Read the Charts

10. www.mironov.com rich@mironov.com 650.315.7394 10 4. Isolate and Fix One Problem  Something weird happens every fourth day… …a critical system is being rebooted

11. www.mironov.com rich@mironov.com 650.315.7394 11 5. Repeat  Prominent dips are power outages

12. www.mironov.com rich@mironov.com 650.315.7394 12 6. Turn the Numbers Sideways  Overloaded when most users connect: 9AM-10AM and 5PM

13. www.mironov.com rich@mironov.com 650.315.7394 13 Measure Compulsively  Many causes of problems  Some outside your control  Improves your vendor selection  Opportunity for creative problem-solving  Root cause analysis replaces the “blame game”  Celebrate each time a milestone is reached  Entire organization begins to think about customers  Expand metrics over time

14. www.mironov.com rich@mironov.com 650.315.7394 14 Service Level Agreements  SLAs are owned by Marketing  Designed to win and keep customers  Competitive weapon, not neutral or objective  Loosely related to internal metrics  Keep customer SLA data separate from operational data  What is the penalty for missing SLA targets?  Financial risk balanced against marketplace risk

15. www.mironov.com rich@mironov.com 650.315.7394 15 Love Your Customers Word-of-mouth is the only route to service success, so...  Complaints are an opportunity  Customer Service knows more than you do  Pay novices to make mistakes  Humility is a virtue  First step is the hardest

16. www.mironov.com rich@mironov.com 650.315.7394 16 Summary  Decide what’s important  Define and measure it  Find and fix the biggest problem  Repeat  Love your customers  Be a star