© Richard Mironov, All rights reserved. Tel: Service Metrics Measure, improve, measure, improve…

Service Metrics  Decide what’s important  Quantify it  Measure compulsively  Root cause analysis  Love your customers

What’s Important to Users?  Each service has a few fundamental attributes  Postpone less important metrics Examples:  server up, addresses work, mail saved  Next tier: delivery time, disk limits, virus scans  Newswire filtering: automatic, easy, broad search base, inexpensive  Next tier: uptime, speed, delivery method

Specific and Narrow  Easy to paint universal goals  “All functions must be up %”  Instead, focus on critical service elements first  Example: Uptime for corporate directory  Sub-second password look-up needed 24x7  OK to wait 5 minutes for password updates  New account creation needed only during normal work hours  Backup must be done within 8 hours, by midnight  Remote disaster recovery can use last Friday’s data

Take the Customer’s View  Metrics must be external, as customers experience them  Example: ISP dial-in success rate  What: fraction of attempted calls that get connected  Metric: % success by hour, day, phone number  Tool: “war room” dialers that ring each number 20x daily  Will also spot outside problems like telco outages  Best tools capture user results directly

Quantify and Measure  Ongoing analytical effort  Operations and finance have best talent pools  Daily review with Operations VP  Charts, trends, incident reports  What happened? How can we automate prevention? Where can we simplify?  Weekly summary to Executive Staff Let’s follow an extended example...

Customer-Reported Outages  When customers call in, you have already failed

Design Metric and Test Tool  Goal: working 99.5%, no transactions lost  Hours of operation: 8AM ET to 6PM PT  Tool: external “robot” submits request every 90 seconds  Notices simple failures…  T-1 down, power out, CPU down, disk full, account closed  Does not test everything…  Value of service, browser versions, data speeds, pricing, network security, tech support helpfulness

Read the Charts

Isolate and Fix One Problem  Something weird happens every fourth day… …a critical system is being rebooted

Repeat  Prominent dips are power outages

Turn the Numbers Sideways  Overloaded when most users connect: 9AM-10AM and 5PM

Measure Compulsively  Many causes of problems  Some outside your control  Improves your vendor selection  Opportunity for creative problem-solving  Root cause analysis replaces the “blame game”  Celebrate each time a milestone is reached  Entire organization begins to think about customers  Expand metrics over time

Service Level Agreements  SLAs are owned by Marketing  Designed to win and keep customers  Competitive weapon, not neutral or objective  Loosely related to internal metrics  Keep customer SLA data separate from operational data  What is the penalty for missing SLA targets?  Financial risk balanced against marketplace risk

Love Your Customers Word-of-mouth is the only route to service success, so...  Complaints are an opportunity  Customer Service knows more than you do  Pay novices to make mistakes  Humility is a virtue  First step is the hardest

Summary  Decide what’s important  Define and measure it  Find and fix the biggest problem  Repeat  Love your customers  Be a star