Switching Mind Sets from Traditional Reporting Barry Brooks ServiceNow Sr. Advisory Group

Agenda Reporting and Analytics Lagging and Leading Usage Practical Examples Behavioral Change

Reporting vs. Analytics Incident Problem Change Reporting Information (Reporting) Analysis (Analytics) What is the difference between the 2? What kinds of questions do you/are you being asked about the data? Is anyone asking why? Analytics gives you the capability to affect the process you are measuring. PA should be focused on the action you will be taking in the future. Reporting – Raw Data  Information “How many Incidents did we have last month?” “What groups were closing the most Incidents? What is our FCR?” Reporting validates performance of groups or processes based on a target. Analytics – Information  Action & Insight “What is our optimal staff level?” “How many FTE’s would be required to reduce my backlog by 50%?” Analytics is used to develop new targets drive future performance.

Reporting vs. Analytics Incident Information (Reporting) Analysis (Analytics) Problem Reporting – Raw Data  Information “How many Incidents did we have last month?” “What groups were closing the most Incidents? What is our FCR?” Analytics – Information  Action & Insight “What is our optimal staff level?” “How many FTE’s would be required to reduce my backlog by 50%?” Reporting Change

Lagging vs. Leading Incident Problem Change Reporting Information Analysis (Analytics) Leading Indicators are historical and measure past performance. Lagging indicators are action-able and can be used to affect future performance. While both types can have leading and lagging indicators, they tend to align to either reporting or analytics. Lagging indicators provide info after the fact. How well did we meet sla commitments last month. Typically, I can't affect any change with a Lagging Indicator. A Leading indicator can give me the capability to take actions to affect a change. Use the SLA breach example: “Take a real example from one of my customers. This shop has very robust reporting capability and was trying to reduce the number of breached lower severity incidents. While looking at the data, they determined that in the vast majority of these incidents, the tickets were misrouted or dropped into less used queues and were not touched for the first 24 hours. By using the information, they created a leading indicator that shows them any incident, with a low priority, that hasn’t been touched in the first 24 hours. In this way, they can affect the behavior of the staff and provide additional insight into the queue.”

Lagging vs. Leading Output Based Easy to Measure Hard to Control Incident Information (Reporting) Analysis (Analytics) Problem Lagging – Easy to measure, hard to affect. Reporting Change

Lagging vs. Leading Output Based Input Based Easy to Measure Hard to Control Input Based Hard to Measure Easy to Control Incident Information (Reporting) Analysis (Analytics) Problem Leading – Hard to measure, easy to affect. Reporting Change

Reporting vs. Analytics

Operational Reporting CIO Board Exec Team IT Strategy IT Management Project Managers Process Managers Service Managers Organization Exceptions Role-Based KPIs Targets & Thresholds Operational Reporting Ad-hoc Reports Information Need Performance Analytics Reporting

The Service Model Defined Five pillars of Enterprise Service Management REQUESTER PROVIDER Service Experience Analytics Delivery Assurance Taxonomy What we’re talking about is the notion of a service model. It is just hard to manage service without it. We have requesters on the left and providers on the right. Note that this does not assume that requesters and providers are people, they are just as likely to be systems, whole or in part. There is a taxonomy of services, what is that? We will discuss. We have the actual service experience, how services are engaged and consumed and then the actual service delivery itself. We have the notion of service assurance, basically making sure that the service is delivered to service levels, and finally the analytics that gives us critical insights into our service processes.  

Positioning PA Internally Cost savings Time savings Improved performance Decreased risk Improved communication Introducing analytics-based metrics into the organization will require a cultural and behavioral change. Here are some of the benefits to focus on throughout the project.

Positioning PA Internally Cost savings are found in the implementation time. PA solutions can be deployed in weeks, while home-brew solutions can take months to deploy and need multiple resources to develop and maintain year by year. Cost savings Time savings Improved performance Decreased risk Improved communication Cost savings are found in the implementation time. PA can be deployed in weeks, while home-brew solutions can take months to deploy and need multiple resources to develop and maintain year by year. PA holds over 15 years of development and is offered to customers at a fraction of the development costs.

Positioning PA Internally Consider How much time is spent in your organization on: Creation of weekly or monthly management team reports (Excel, PDF, Powerpoint.) Creation of monthly SLA reports for customers or business lines. Cost savings Time savings Improved performance Decreased risk Improved communication Performance Analytics KPI Dashboard and Scorecards fully automates the collection and processing of KPI related information. Everyday all your KPIs are processed and updated and automatically made available to the end-user community. Many organization who are not using an automated solution have 1 or more teams who spend at least 5 days a month on KPI reporting in Excel or with other tools.
 Consider How much time is spent in your organization on: Creation of weekly or monthly management team reports (Excel, PDF, Powerpoint.) Creation of monthly SLA reports for customers or business lines.

Positioning PA Internally Phase 1: Baseline measurement At the starting point you need to establish at least a trend with 20 data points for each KPI. Phase 2: Reduce lost production hours Start focusing on improving the customer perspective of IT and to optimize the internal workload. Phase 3: Become Innovative Reactive to Proactive activities Phase 4: Raise the bar New targets and CSI Cost savings Time savings Improved performance Decreased risk Improved communication Phase 1: Baseline measurement - At the starting point you need to establish at least a trend with 20 data points for each KPI. Based on the trend of these 20 data points you can analyze volatility and set realistic targets based on actual performance. During this phase you often identify quick wins to reduce the ticket backlog. The focus in this phase is to measure the "activities" of IT staff members with leading indicators and change their behavior, become metrics driven so you can expect future results. Phase 2: Reduce lost production hours - Once indicators of phase 1 are stable, you should see the first results. Now it is time to start focusing on improving the customer perspective of IT and to optimize the internal workload. Indicators as Mean Time to Repair, response times, pickup times and ticket allocation become very important. The focus of this phase is to further optimize the total cycle time of solving IT incidents. As a result, the "lost production hours" caused by IT Incidents is reduced and a higher % of tickets are solved in SLA. Phase 3: Become Innovative - Once indicators of phase 2 are stable, start focusing on shifting time and resources from "reactive activities" to "proactive activities". How much time is spend on solving incidents versus working on changes. The more time IT organization can work on changes, the more stable the IT environment gets and the higher the innovative capabilities of an IT organization. More changes should lead to fewer service disruptions but poorly implemented changes could also lead to more service disruptions. The focus should be to have a stable efficient change process. Important indicators to improve are % of emergency changes and % of urgent incidents while keeping the indicators of phase 1 and 2 stable. Phase 4: Raise the bar - Once the KPIs of phase 3, phase 2 and phase 1 are all stable, it is time to raise the bar. Set new realistic targets (5-10% improvements) based on historic performance, sharpen your SLA's, and do more with less IT staff. In this phase you can start communicating your KPIs with all IT stakeholders including customers since you have now become a predictive IT organization that is able to continuously improve its performance. This new level of transparency will further boost customer satisfaction and allow you to become a trusted partner.

Positioning PA Internally A well functioning set of KPIs with targets and thresholds can help prevent an IT organization from getting SLA penalties, get an ever growing backlog or risk poor data quality that prevents you from delivering quality reports to your customers. The performance history for these KPIs ensure that you are able to set “realistic” targets and implement the right thresholds so you can implement management by exception techniques. Cost savings Time savings Improved performance Decreased risk Improved communication A well functioning set of KPIs with targets and thresholds can help prevent an IT organization from getting SLA penalties, get an ever growing backlog or risk poor data quality that prevents you from delivering quality reports to your customers. The performance history for these KPIs ensure that you are able to set “realistic” targets and implement the right thresholds so you can implement management by exception techniques.

Positioning PA Internally “Perception is everything” - IT today is typically unable to communicate performance improvement effectively. Are things getting better? Having a clear set of KPIs and performance history enables IT organizations to change the conversation with IT stakeholders. It helps developing stronger relations, boost moral and create better partnerships. Cost savings Time savings Improved performance Decreased risk Improved communication “Perception is everything” - IT today is typically unable to communicate performance improvement effectively. Are things getting better? Imagine being able to communicate that you have improved 7 of your top 10 KPIs in the last 30 days and that overall improvement has been 4% in the last 60 days. Imagine instead of having to report to a customer that you again failed to meet their SLA last month, you can communicate that although you missed your SLA last month, you improved 5% compared to previous months and that if you continue to improve you expect to be SLA compliant next month. Having a clear set of KPIs and performance history enables IT organizations to change the conversation with IT stakeholders. It helps developing stronger relations, boost moral and create better partnerships.

Steps for Performance Management are: Identify the processes/activities/domains of what needs to be managed Articulate the goals to be met Define the metrics/indicators that measures if goals are being met Collect the data for the indicators Take improvement actions Introducing analytics-based metrics into the organization will require a cultural and behavioral change. Here are some of the benefits to focus on throughout the project.

KPI Example #1: % of closed incidents without a Configuration Item (CI) Identify the processes/activities/domains of what needs to be managed Incident Management Articulate the goals to be met An increasing trend of incidents being closed with being related to a CI, whether component or Business Service, is critical for building a history of ‘what’ the incidents are associated with As a result of improving on this KPI, an IT organization will be capable of understanding and reporting on trends as they relate to CI’s – which is a key input into the Problem Management process Define the metrics/indicators that measures if goals are being met Number of closed incidents without a CI / Number of closed incidents * 100 Suggested improvement actions Although CI’s may not always be identifiable at ticket creation, set configuration item field to be a mandatory field upon setting the incident state to resolved or closed Consider using coaching loops or weekly exception reporting to identify undesired behavior in assigning CI’s to incidents What do the ITIL books say? Relating incidents to CI’s is essential for quality reporting Measuring MTBF, MTTR, and MTBSI are all dependent on being able to correlate incidents being logged against devices/services (CIs) When customers ask if they should assign a CI to an Incident record, what do you say? What is a best practice approach?

KPI Example #1: % of closed incidents without a Configuration Item (CI) Identify the processes/activities/domains of what needs to be managed Incident Management Articulate the goals to be met An increasing trend of incidents being closed with being related to a CI, whether component or Business Service, is critical for building a history of ‘what’ the incidents are associated with As a result of improving on this KPI, an IT organization will be capable of understanding and reporting on trends as they relate to CI’s – which is a key input into the Problem Management process Define the metrics/indicators that measures if goals are being met Number of closed incidents without a CI / Number of closed incidents * 100 Suggested improvement actions Although CI’s may not always be identifiable at ticket creation, set configuration item field to be a mandatory field upon setting the incident state to resolved or closed Consider using coaching loops or weekly exception reporting to identify undesired behavior in assigning CI’s to incidents What do the ITIL books say? Relating incidents to CI’s is essential for quality reporting Measuring MTBF, MTTR, and MTBSI are all dependent on being able to correlate incidents being logged against devices/services (CIs) When customers ask if they should assign a CI to an Incident record, what do you say? What is a best practice approach?

KPI Example #1: % of closed incidents without a Configuration Item (CI) Identify the processes/activities/domains of what needs to be managed Incident Management Articulate the goals to be met An increasing trend of incidents being closed with being related to a CI, whether component or Business Service, is critical for building a history of ‘what’ the incidents are associated with As a result of improving on this KPI, an IT organization will be capable of understanding and reporting on trends as they relate to CI’s – which is a key input into the Problem Management process Define the metrics/indicators that measures if goals are being met Number of closed incidents without a CI / Number of closed incidents * 100 Suggested improvement actions Although CI’s may not always be identifiable at ticket creation, set configuration item field to be a mandatory field upon setting the incident state to resolved or closed Consider using coaching loops or weekly exception reporting to identify undesired behavior in assigning CI’s to incidents What do the ITIL books say? Relating incidents to CI’s is essential for quality reporting Measuring MTBF, MTTR, and MTBSI are all dependent on being able to correlate incidents being logged against devices/services (CIs) When customers ask if they should assign a CI to an Incident record, what do you say? What is a best practice approach?

KPI Example #1: % of closed incidents without a Configuration Item (CI) Identify the processes/activities/domains of what needs to be managed Incident Management Articulate the goals to be met An increasing trend of incidents being closed with being related to a CI, whether component or Business Service, is critical for building a history of ‘what’ the incidents are associated with As a result of improving on this KPI, an IT organization will be capable of understanding and reporting on trends as they relate to CI’s – which is a key input into the Problem Management process Define the metrics/indicators that measures if goals are being met Number of closed incidents without a CI / Number of closed incidents * 100 Suggested improvement actions Although CI’s may not always be identifiable at ticket creation, set configuration item field to be a mandatory field upon setting the incident state to resolved or closed Consider using coaching loops or weekly exception reporting to identify undesired behavior in assigning CI’s to incidents What do the ITIL books say? Relating incidents to CI’s is essential for quality reporting Measuring MTBF, MTTR, and MTBSI are all dependent on being able to correlate incidents being logged against devices/services (CIs) When customers ask if they should assign a CI to an Incident record, what do you say? What is a best practice approach?

KPI Example #1: % of closed incidents without a Configuration Item (CI) Identify the processes/activities/domains of what needs to be managed Incident Management Articulate the goals to be met An increasing trend of incidents being closed with being related to a CI, whether component or Business Service, is critical for building a history of ‘what’ the incidents are associated with As a result of improving on this KPI, an IT organization will be capable of understanding and reporting on trends as they relate to CI’s – which is a key input into the Problem Management process Define the metrics/indicators that measures if goals are being met Number of closed incidents without a CI / Number of closed incidents * 100 Suggested improvement actions Although CI’s may not always be identifiable at ticket creation, set configuration item field to be a mandatory field upon setting the incident state to resolved or closed Consider using coaching loops or weekly exception reporting to identify undesired behavior in assigning CI’s to incidents What do the ITIL books say? Relating incidents to CI’s is essential for quality reporting Measuring MTBF, MTTR, and MTBSI are all dependent on being able to correlate incidents being logged against devices/services (CIs) When customers ask if they should assign a CI to an Incident record, what do you say? What is a best practice approach?

More Information Public KPI Library ServiceNow Community Barry Brooks Sr. Advisory SC 630.688.3616 Barry.brooks@servicenow.com LinkedIn - barrybrooks55 Twitter - @barry_brooks