Presentation is loading. Please wait.

Presentation is loading. Please wait.

Wendy Tegart, Provincial Director Service Management Jill Robert, IT Strategic Partner e-Health 2013 Challenges & Opportunities TEC Talk Service Interrupted…

Similar presentations

Presentation on theme: "Wendy Tegart, Provincial Director Service Management Jill Robert, IT Strategic Partner e-Health 2013 Challenges & Opportunities TEC Talk Service Interrupted…"— Presentation transcript:

1 Wendy Tegart, Provincial Director Service Management Jill Robert, IT Strategic Partner e-Health 2013 Challenges & Opportunities TEC Talk Service Interrupted… AHS Experience with IT Major Incidents & Clinical Involvement

2 2 Service Interrupted… TEC Talk Faculty/Presenter Disclosure Faculty: Wendy Tegart & Jill Robert Relationships with commercial interests: –Grants/Research Support: Not applicable –Speakers Bureau/Honoraria: Not applicable –Consulting Fees: Not applicable –Other: Employees of Alberta Health Services Nothing to Disclose

3 3 Service Interrupted… TEC Talk Alberta Health Services Overview 1 1 Major Incident Process Communication Approach Clinical Involvement Next Steps Questions Major Incident Roles 2 2 4 4 5 5 6 6 7 7 3 3 Agenda

4 4 Service Interrupted… TEC Talk Responsible for delivering health services to the 3.8 million people living in Alberta, over 661,848 square kilometers served Annual Service Volumes (2011-12) Alberta Health Services (AHS) Acute Care 2,029,191 Emergency Department Visits 376,115 Hospital Discharges 2,602,384 Total Hospital Days 50,099 Births 99 Acute care hospitals and 5 stand-alone psychiatric facilities Primary Care 104,704 Home Care Clients 766,146 Health Link calls 393,964 EMS Calls/Events Alberta Health Service Overview

5 5 Service Interrupted… TEC Talk Largest Employer in Alberta, 5 th largest in Canada ◦ 100,000 employees ◦ 7,000 physicians ◦ 120,000 network IDs Scope of AHS-IT ◦ 1,514 production apps (163 critical) ◦ 34 data centers ◦ 4,721 servers (physical and virtual) ◦ 75,000 workstations ◦ 48,000 tickets generated monthly ◦ 550 concurrent users in ITSM tool ◦ 1,300 IT Staff (+ outsourced partners) AHS Scale of Effort Alberta Health Service Overview

6 6 Service Interrupted… TEC Talk Context to Current Realities Complexities of Electronic Health Record in Alberta Local vs Provincial IT service delivery Given the complexities of the AHS IT landscape, aging and varied technical infrastructure and critical service requirements to support patient care... “Downtimes happen...” How do we minimize organizational and clinical impact and provide robust support when the technology fails? Major Incident Process

7 7 Service Interrupted… TEC Talk Super Bowl 2013 – infamous power outage Major Incident Process

8 8 Service Interrupted… TEC Talk What is a Major Incident (MI)?  IT has a provincial Incident Management Process to manage all Incidents. When an Incident is of a certain scale, scope, or impact, a “Major” Incident is launched.  The goal of the Incident process is to return an IT Service to operational status.  Throughout AHS-IT, we employ this common process to ensure that major IT service issues are quickly identified and appropriately responded to.  The purpose of the MI process is to supplement the Incident process with additional resources, escalation, communication and record keeping. Major Incident Process

9 9 Service Interrupted… TEC Talk Is this a “Critical” Incident? Urgency and Impact must both be High to create a critical incident. Critical Incidents must be escalated to the IROC immediately. Critical incidents are: –a major outage affecting a large number of customers –an essential service and/or a business unit where there is no available resolution or work around to provide a return to business operations Must also consider: –Patient safety may be at risk or reduced effectiveness of patient care –The safety of AHS staff and personnel –Impact to confidentiality of data, or reliability of data –Degradation of a service including data, applications, or infrastructure. –A Senior Admin from the business is requesting a Major Incident be declared (requires immediate escalation to IROC) Major Incident Process

10 10 Service Interrupted… TEC Talk Is this a “Critical” Incident? Urgency Major Incident Process

11 11 Service Interrupted… TEC Talk Is this a “Critical” Incident? Impact Major Incident Process

12 12 Service Interrupted… TEC Talk Priority Major Incident Process

13 13 Service Interrupted… TEC Talk Major Incidents by Month Major Incident Process

14 14 Service Interrupted… TEC Talk IT Major Incident Roles An IT service Incident is typically managed by the IT Service Desk and/or a specific IT Service team. When an MI is initiated, some additional resources brought in include: IT Incident Response On Call (IROC) This is a group of IT Directors who share an On Call responsibility for MI’s. Once contacted, the IROC is responsible for managing the MI Process so the Service Desk and Service team can concentrate on resolving the Incident. IT Security & Compliance On Call On Call IT Security staff to respond to MI’s with a security component. IT Senior Leader On Call This group of IT senior leaders is available to provide additional guidance and authority if/as required by the particular MI. Problem Manager Chair and facilitate communication bridge meetings. Notify IT staff of updates. Major Incident Roles

15 15 Service Interrupted… TEC Talk Clinical Roles Not all MIs require the engagement of clinical experts, but when required these roles provide context to clinical impact and urgency Clinical Informatics This is a group of Physicians and non-physicians Clinical Operations Administrator On-call On Call AHS leaders including Executive Directors and Site Administrators. May provide front line resources to support in downtime and reconciliation efforts Senior Leadership On-call This group of AHS Senior leaders include Facility Medical directors and VPS Health Information Management Health Record Management experts with data and record integrity expertise Zonal Emergency Operations Centres (ZEOCs) Tied into Emergency Preparedness Major Incident Roles

16 16 Service Interrupted… TEC Talk Bridges Types (conference calls) Technical Bridge Part of the Incident Management process, as is initiated independently of MI process Opened when collaboration by several parties is required during incident resolution activities Communications Bridge Launched by IROC to bring the right stakeholders together to identify the problem and direct its resolution. Problem manager assists by recording chronology, participants, decisions and results Directs communications within IT and the user community Clinical Bridge Usually chaired by a Clinical Informatics physician Communication Approach

17 17 Service Interrupted… TEC Talk MI Heads Up Notification Communication Approach

18 18 Service Interrupted… TEC Talk Communications to Customers IT - Service Issue Information Message may be sent to users of IT Services. In relation to unexpected/unplanned service issues. Say who is this information is intended for / pertains to. Speak in terms the customer will understand. Briefly and directly tell users what is happening and what impact they will experience. Note that IT teams are working to resolve the issue and restore Services. Acknowledge the issue/inconvenience and provide contact information for the relevant zone/FHE Service Desk. If appropriate, state that an update will be provided within a specific timeframe. Replace all text in this section with pertinent information. Review the notifications guide if there are questions on when to use this format. Impact Summary Clearly state what, from the users perspective, is not working. Also set out the specific locations affected by this Service Issue. NOTE: Any exclusions or caveats to what you've stated above regarding this Service Issue. Replace all text in this section with pertinent information. Review the notifications guide if there are questions on when to use this format. Communication Approach

19 19 Service Interrupted… TEC Talk MI Root Cause Code Definitions Cause CodeSummary Application/Software BugThe failure is caused by a problem within the packed software itself. CommunicationFailure is caused by a missed communication. DataUnexpected or corrupted data elements caused the failure. EnvironmentThe failure is caused by an uncontrolled element of the physical world where redundancy would not have reasonably mitigated the effect. EquipmentFailure due to age, malfunction or fault in the physical equipment where redundancy would not have reasonably mitigated the effect.. IT Third Party VendorRoot cause lies with the vendor providing a service. ProcessMissing or undeveloped process caused the failure. There was an oversight in the process; a branch of the process isn’t properly developed or missed entirely. SecurityAn IT Security failure caused the issue. TrainingThe failure was caused by lack of understanding, incorrect qualification or insufficient training. OtherA mistake was made where existing process, if followed correctly, should have avoided the failure. UnknownRoot cause undetermined. Communication Approach

20 20 Service Interrupted… TEC Talk Clinical Support During a MI Transparency Communication –Understanding and translating the clinical impact –Timely and frequent “clinical speak” communication about the incident and immediate risk mitigation measures Support –Robust downtime procedures owned by clinical operations –Bedside to boardroom engagement and support Clinical Involvement

21 21 Service Interrupted… TEC Talk More than clinical involvement... it’s about relationships, partnerships and supporting safe patient care! Clinical Involvement

22 22 Service Interrupted… TEC Talk Next Steps Continuous improvement per incident review Develop service improvement plans overall driven by business requirements Examine different scales of MIs and support requirements Leveraging the successes of the MI process to other risk areas Continually examine clinical business risk tolerance/value and architecture of information systems Simplify – application consolidation, migration to a provincial patient care platform and large scale reliability/redundancy Higdon’s Law 1.Good judgement comes from bad experience. 2.Experience comes from bad judgement. Next Steps

23 23 Service Interrupted… TEC Talk Insanity: Doing the same thing over and over again and expecting different results. ~ Albert Einstein Questions? Comments / Questions

Download ppt "Wendy Tegart, Provincial Director Service Management Jill Robert, IT Strategic Partner e-Health 2013 Challenges & Opportunities TEC Talk Service Interrupted…"

Similar presentations

Ads by Google