Major Incident Process @ Progressive

Own lifecycle of all Major Incidents Enable quick restoration of service and minimize impact for all key incidents via centralized communication, collaboration, facilitation, and coordination. Major Incident Facilitation When the IMT takes ownership of an incident record they will provide direction over an AT&T bridge. The bridge will be used for bringing support groups together to make decisions for service restoration and normalization. This will allow technicians to collaborate and troubleshoot the incident. Research potential causes by utilizing tools available to the team including researching change requests, dashboards and related Remedy tickets that maybe related to the incident. Engage Support Groups / Technicians during the course of a Major Incident as needed Document actions from the Major Incident in an Incident Ticket Regularly communicate to stakeholders and interested parties Monitor if the Major Incident meets any Disaster Recovery Triggers Own lifecycle of all Major Incidents Successful restoration and normalization of Service for Major Incidents Remedy Ticket Handling and Documentation Creation of Problem Investigation; Emergency Change Requests, & CI Unavailability Forms Send Incident Notifications and Escalations Pages, texts, and emails regarding a Major Incident Maintenance of the Incident Management Notifications site MyQumas documentation, training Direct-line for Vendors, Partners, & Customers Lifecycle Owner Leadership, Direction, Standards & Practices Incident Task Force For elongated incidents where there is no immediate resolution, the IMT will schedule and facilitate meetings to investigate root cause and normalization of service Platform Owner Planning, Road-mapping, & Supporting Support Groups / Technical Staff Investigation & Resolution of Major Incidents Participate in Problem Reviews The Problem Management team schedules and invites the IMT to review the Key Incident. The IMT is needed to collaborate on details and assist in determining root cause of the incident handled by IMT. The IMT will also give feedback in how well the incident process was executed. Problem COE Communicate & Cooperate Business Inform on Major Incidents Service Desk Coordination during a Major Incident Emergency Change Management When working a Key Incident and a change is required within 24 hours or less to restore service, the Incident Management Team will be engaged to initiate the Emergency Change process. Service Management teams Consult with other SMO process teams for consistency

What is a Major Incident? A Major Incident will be an Incident that meets any of the below criteria.

Today’s Major Incident Model

Issues with current Root Cause Investigation process Lack of focus Unclear Ownership Longer Investigation times Inconsistent Prioritization Finger-pointing Hot Potato Afraid to get ‘dinged’ Lack of customer perspective Do we have the right skill set performing the right functionality?

Focus on Quality Call Auditing:

Communication Incident Management Notifications

Future: Major Incident Process Goal: decrease IMT engagement Time Pilot to begin where we dissect a critical application: understand current monitoring strategy, enhance the monitors Have monitors directly notify IMT Build pre-defined communication / escalation scripts per monitor Some Stats: The average time to engage IMT for 2013 which is 16:59:46 (for 700 total Incidents).  There were a few outliers that caused this number to shoot to just under 17hrs to engage the IMT.  If I take out the top 5, this number drops to 7:37:35. In terms of engaging the IMT, we use 20 minutes as a high-water mark: In 2012, for 82% of Incidents handled by the IMT, took longer than 20 mins to engage the IMT In 2013, this number dropped to 59%. One of our issues today is not so much as monitoring, but easily identifying what is critical. Our current model is for support teams to engage the IMT after they have determined that a Major Incident occurred. With this model, we have seen a big variance of engagement time based on the support groups and issue.

Questions?