Myths, Lies and Just Plain Misperceptions The Value of Metrics in Continuous Improvement January 16, 2007

Myths, Lies and Just Plain Misperceptions l After this presentation, you should understand: w How we use metrics to separate perception from reality w What types of metrics we are looking at w Why developing metrics is important w How we’re using metrics for continuous improvement

Building a Billy Beane Baseball Team l “Moneyball” l Finding the Right Metrics to Measure l Perception vs. Reality l Changing a Culture l Proving it with Numbers w Not just for Baseball anymore

Perception vs Reality

Who Was Billy Beane, Anyway? (Perception) What the Old School Scouts said: l “the Good Face” l “We were looking at talent” l “He’s a five-tool guy” (???) l “I never looked at a single statistic of Billy’s” Billy was signed for $125,000 in 1980 out of High school by the Mets... and would’ve been their first round draft pick (who was that?)

What Happened to Billy? (Reality) They all missed the clues... l He was still in high school l His BA went from.500 to.300 his senior year l He didn’t know how to fail l He wanted to go to college l He lasted six years in the majors l On Base Percentage =.246 “The draft has never been anything but a …crapshoot. We take 50 guys and celebrate if two of them make it. In what other business is two for 50 a success? If you did that in the stock market, you’d go broke.” Billy Beane

How Billy Turned Baseball Around As General Manager of the Oakland A’s, he: l Hired a Harvard grad to run the stats l Looked at college prospects instead of H.S. l Chose OBP* as the key performance indicator l Determined a guy has to look good on paper, not just look good *On Base Percentage “We’re not selling jeans here.” Billy Beane

Myths, Lies and Just Plain Misperceptions What are our customers saying? What are 2 nd level teams saying? How do we separate fact from fiction? What metrics will help us get better?

Myths, Lies and Just Plain Misperceptions (Perception) l We consistently assign tickets late to 2 nd level teams l We have a lot of reopened tickets l Roll Over No Answer is high; we are unavailable to our customers l Tickets assigned >= 5 times are due to misassignment l Is there a problem with unassigned tickets? At the desk? 2 nd Level? l Analysts change priorities to extend due date

Myths, Lies and Just Plain Misperceptions (Reality) l Tickets assigned off the Desk after 75% of due date w Approx. 4.72% (632 out of 13,397) - 1 st Level w Approx % (85 out of 756) - 2 nd Level w 47% reopened by customer, 29% pending customer response, 26% Tech. Service Desk (TSD) Opportunity l Tickets reopened by the customer w.75% - 1 st Level (51,041 Resolved and 382 reopened) w 1.63% - 2 nd Level (3316 Resolved and 54 reopened) l RONA (Roll Over No Answer) w 5% How are we able to tell this story?

Myths, Lies and Just Plain Misperceptions (Reality) – Cont’d l Tickets assigned 5 or more times w 109,205 closed YTD w 254 reassigned 5 or more times (.23%) w 20% are responsibility of the TSD l Open, overdue, unassigned tickets w 3 – 10 tickets per week (7 week of 1/12) l Percent of tickets w/ priority changes w 100,237 created YTD w 8,955 Priority change (8.93%) w 2,682 Priority change after due date (2.32%) l Much more to come How are we able to tell this story?

Step 1 – Select the Statistics Single Most Important Stat that the Oakland A’s “discovered” was: On-Base Percentage (OBP) Derek Jeter's OBP =? Answer:

Step 2 – Develop the Process Rules of the game to increase OBP: 1. Every batter becomes a leadoff man 2. Every batter should also hit home runs 3. Hitting was less a physical than a mental skill Scoring runs, in the new view, was a process that was routine. Source: Moneyball

TSD-2006 METRICS Mean time to Resolve High/Medium Goal 10/8 (5) 8/8 On-time Resolution of Requests Level 1 Goal 92% (3) % of Incident Tickets Resolved at the Service Desk Goal 67% (6) 97% 65% Roll Over no Answer Calls (RONA) Goal <5% (ACD) 5% Customer Satisfaction 1 st Level Goal 96% (1) Customer Satisfaction 2 nd Level Goal 96% (2) 96% Initial Ticket Closed Success 1 st Level Goal 92% (7) 99% Initial Ticket Closed Success 2 nd Level Goal 92% (7) 98% % Tickets Reassigned after 75% of Elapsed Due Date 1 st Level Goal 4% (4) % Tickets Reassigned after 75% of Elapsed Due Date 2 nd Level Goal 10% (4) 12% 44 sec. 5% On Track and projected to be on track at year end On Track-recovery feasible or currently on track, year end is uncertain Off Track-recovery not possible Average Speed of Answer Goal 35 sec. (ACD) 2006 Year-End Attend Safety Council Meeting Goal 4/4 Safety performance of OSHA injury rate of zero 0 4/4 99% On-time Resolution of Requests Level 2 Goal 92% (3a)

Step 1 – Select the Statistics – Assigned w/in 75% of due date

Step 1 – Select the Statistics – Reopened

Step 1 – Select the Statistics – Reassigned 5 or more times

Step 1 – Select the Statistics - Open, Overdue, Unassigned

Step 2 – Develop the Process l Identify Metrics w Develop Initial Reports w Validate Assumptions w Test Reports w Change Reports This is an investment of our resources’ time

Step 2 – Develop the Process – cont’d l Weekly Continuous Improvement Meeting w Identify Gaps in Existing Processes w Assign Ownership for Resolution w Proactively Look for Trending w Open, candid forum Do It All Over Again!!! Make it Routine!

Step 2 – Develop the Process – Cont’d Where are we in the process? l Reports w Manually run w Modularly built w Automated l Sharing Data/Knowledge w Solid foundation first w Show other teams w Leaders, not followers

Step 3 – Efficiently win games Improvements made for us: l Better visibility into overdue, coming due tickets l Better visibility into our queues l Better visibility into the “Dr. House” hard cases l Greater accountability for vendors l Look at each ticket that was mishandled and determine action plan The Whole Process is designed to help us get better

Step 3 – Efficiently win games – cont’d Improvements made for our customers: l Better availability to our customers w/ focus on RONA l Understanding reopened ticket reasons l Being an advocate for our customers’ concerns l Understanding the relationship between assigned late, reassigned, and reopened The Whole Process is designed to help us get better

Are we hitting our targets? l After this presentation, you should understand: w How we use metrics to separate perception from reality w What types of metrics we are looking at w Why developing metrics is important w How we’re using metrics for continuous improvement

The End