1. Incident Management
ISD Division
Office of State Finance

2. Contents Purpose of Incident Management Process Escalation Guidelines
Office of State Finance
Purpose of Incident Management
Process
Escalation Guidelines
Policy 2

3. Contents
Office of State Finance
Purpose of Incident Management 3

4. Terms (Service Operations)
Office of State Finance 1
Something is broken or about to break.
Repair as quickly as possible.
Open incidents are measured against SLAs.
Incident 2
Underlying “root-cause.”
Remains open until underlying issue is resolved.
Open Problems are not measured against SLAs.
Creates “work-arounds.”
Problem 3
Routine pleas for help – but nothing is broken. These include items like “can you tell me Joe Blow’s phone number”, “can you unlock my account”, and “will you reset my password.”
Service Request
ITIL Training

5. Incident Guidelines Office of State Finance
Since an incident is always related to a production process or product that is not functioning as it once was, it is critical to perform triage and resolve the most serious incidents as quickly as possible, leaving the less serious ones to be resolved as time allows.
Incidents do not need prior approvals to be worked and resolved. They only need to be prioritized to work the most critical incidents first.
In most cases incidents should be worked and resolved before time is allocated to fulfill service requests. High priority service requests may be worked before low priority incidents.

6. Incident Prioritization
Office of State Finance
Before an incident can be prioritized, the impact and severity must be assessed.
Incident Priority
Impact: People & Service
Severity: Time
Severity
3-Low
User cannot perform a portion of their duties.
2-Medium
User cannot perform critical time sensitive functions
1-High
Major portion of a critical service is unavailable
Impact
One or two personnel
Degraded Service Levels but still processing within SLA constraints
Multiple personnel in one physical location
Degraded Service Levels at or below SLA constraints
Cause of incident falls across multiple functional areas
All users of a specific service
Personnel from multiple agencies are affected
Public facing service is unavailable
Any item listed in the Crisis Response tables

7. Incident Service Level Targets
Priority
Target
Response
Resolve
3 - Low
90% - 24 hours
90% - 7 days*
2 - Medium
90% - 2 hours
90% - 4 hours
1 - High
95% - 15 minutes
Although desirable, it is not always possible to resolve all incidents within the target time frame. For this reason, for SLA purposes, the success rate to resolve incidents by the target time frame is not expected to be 100%

8. Contents
Office of State Finance
Process 8

9. Explanation of chart on following pages
Process Flow
Explanation of chart on following pages

10. Process Flow - Identification
Service Desk will collect all available information about the issue.
The information will be logged.
Before continuing the Service Desk will verify the related service having the issue and correlate it with an existing service and SLA.
If the case is not actually an incident, but a service request, the Service Desk will continue the case under the Service Request Process guidelines.
After incident verification, Service Desk will attempt to establish incident severity and impact which define priority.

11. Process Flow – Analyze & Resolve
If this is a major incident, the Service Desk will notify senior management immediately.
Information on how to resolve many incidents is contained in the Knowledge Base within CRM. If the information is present, the Service Desk will attempt to resolve.
The Service Provider Group will continue the diagnosis until a solution or workaround is available and verify solution with customer.
If customer is satisfied, the Service Desk will close the case.

12. Contents
Office of State Finance
Escalation Guidelines 12

13. Purpose of Escalation Process
By “escalation” we mean:
The Service Desk broadens awareness and status of incident, particularly to management and customers
Management & Customers
Service Desk
Incident Handling

14. Purpose of Escalation Process
Office of State Finance
Establish a standard method of escalation and notification across all services, platforms, and functions
Restore normal service operations as quickly as possible
Communicate incident status to the affected customer
Minimize the adverse impact on business operations

15. Components of Incident Escalation
Concept
Office of State Finance
Incident: Something is broken or is about to break.
Impact: Users and/or services affected
Severity: How much time can elapse before the incident creates a significant business impediment?
Components of Incident Escalation

16. Concept – Always an Exception
Office of State Finance
The following should always be escalated with a high priority:
Agency executive level person (or their representative)
Governor’s / Lt. Governor’s office
Exec branch of Oklahoma State government
Cabinet Secretaries
Agency directors (or deputy agency directors representing the agency director)
The following should always be escalated with a medium/high priority (CTO and ISD Managers are notified via ):
Security breach

17. Escalation Process

18. Time Guidelines Priority Time Limit before Escalation by Service Desk
3-Low
3 business days
Manager
2-Medium
4 hours
If on-call contact cannot be reached during non-business hours
If neither on-call contact or their manager cannot be reached during non-business hours
CTO & ISD Mgt
48 hours
1-High
Immediate
Senior Mgt

19. Log Incident with the Service Desk!
Guidelines
Office of State Finance
Critical starting point!
Log the case with as much detail as is available.
Notifications
(Service Desk)
Escalation
Log Incident with the Service Desk!

20. Escalation Notifications
Office of State Finance
The default mechanism for notification will be by unless otherwise specifically stated.
Whenever escalation or notification by phone is indicated, the Service Desk will utilize all known numbers for contact, leaving voice mail on each until person is contacted.

21. Escalation Notifications
Office of State Finance
Senior management notification will include CIO, CTO, and all functional managers.
Escalation of a case does not remove the assignment from an individual. The provider group manager will make certain the right personnel are assigned.
Any time a case is escalated, the case will be updated to reflect the escalation and the following notifications will be performed by the Service Desk:

22. Notifications - 2-Medium
Office of State Finance
Escalation Priority
Severity
3-Low
2-Medium
1-High
Impact
Upon discovery of the incident, the Service Desk will notify the Director, CTO, and managers of the Information Services Team within 30 minutes by using CRM’s interested parties function.
The person assigned to the case or the functional manager will update notes on the case at least once daily.
The customer will be notified via interested parties of the current status at least once daily using a case note with visibility set to “all”.
The managers of all functional groups involved in resolving the case will be notified each time the case is updated through the CRM interested parties section.

23. Notifications - 2-Medium
Office of State Finance
Escalation Priority
Severity
3-Low
2-Medium
1-High
Impact
Upon discovery of the incident, the CTO and managers of the Information Services Team will be notified within 30 minutes by using CRM’s interested parties function.
The case will be updated with resolution status at least once daily by person assigned to the case or the functional manager.
The customer will be notified via interested parties of the current status at least once daily.
The managers of all functional groups involved in resolving the case will be notified each time the case is updated through the CRM interested parties section.

24. Notifications – High Escalation Priority Severity 3-Low 2-Medium
Office of State Finance
Escalation Priority
Severity
3-Low
2-Medium
1-High
Impact
Upon discovery of an incident of high priority…Take time to log the incident in with the Service Desk. The Service Desk will notify the Director and CTO of the Information Services Division using CRM’s interested parties function and by phone.
The managers will also be notified by using CRM’s interested parties function simultaneously with the Director and CTO notifications.
Each time the case is updated, the senior management notification will be performed, again using the interested parties function of CRM.

25. Notifications – 1-High Escalation Priority Severity 3-Low 2-Medium
Office of State Finance
Escalation Priority
Severity
3-Low
2-Medium
1-High
Impact
Case updates with current status will be made a minimum of every four hours between 8 AM & 5 PM. CRM will then forward the updated status to the interested parties on the case.
If the incident was reported by a customer, the person reporting the incident will be notified by or phone as requested by the customer.
If the incident was not reported by the customer, or if other agencies are impacted, notification should utilize the contact information on record at the Service Desk to notify the potentially impacted customers via method that is still functional.

26. Escalation Notifications
Office of State Finance
Update the case!
When new info is discovered, a plan of action is devised, or resolved, update the case!
Notifications
(Service Desk)
Escalation
Update Incident Case in CRM

27. Contents
Office of State Finance
Policy 27

28. Incident Policy Office of State Finance
Incidents must be logged with the Service Desk before they are worked.
Accept the assignment of the Case from the Service Desk before working the case.
Update the case notes regularly as new information is uncovered.
On the case note, set visibility to “All” and set the note type appropriately.

29. Incident Policy Office of State Finance
The case assignee is responsible for updating the case status on the appropriate schedule.
Escalation notifications to the customer will be performed by the Service Desk unless otherwise determined by CTO.
Upon resolution of the incident, all persons previously notified of an escalation will be notified by CRM via the interested parties function.

30. Incident Policy Office of State Finance
In the event of a High escalation, all available means of communication will be used by the Service Desk to make successful contact with support personnel and senior management.
Medium and High escalations will occur regardless of business hours using the appropriate methods of communication.
The escalation process does not remove the responsibility from the assigned resource to seek necessary assistance in resolving an incident.