Presentation is loading. Please wait.

Presentation is loading. Please wait.

August 14, 2003 Blackout Summary Based on Interim Report of the United States – Canada Power Outage Task Force November 19, 2003.

Similar presentations

Presentation on theme: "August 14, 2003 Blackout Summary Based on Interim Report of the United States – Canada Power Outage Task Force November 19, 2003."— Presentation transcript:

1 August 14, 2003 Blackout Summary Based on Interim Report of the United States – Canada Power Outage Task Force November 19, 2003

2 2 U.S.-Canada Interim Report Released November 19, 2003 Result of an exhaustive bi-national investigation Working groups on electric system, nuclear plant performance and security Hundreds of professionals on investigation teams performed extensive analysis Interim report produced by the teams and accepted by the bi-national Task Force

3 3 Overview Overview of power system and reliability Pre-outage conditions on August 14 Trigger events and start of cascade Wide area cascade Root causes Next steps

4 4 Power System Overview

5 5 Reliability Overview Balance generation and demand Balance reactive power supply and demand Monitor flows and observe thermal limits Observe power and voltage stability limits Operate for unplanned contingencies Plan, design and maintain a reliable system Prepare for emergencies Reliably operate the system you have!

6 6 3 Interconnections / 10 NERC Regions

7 7 NERC Control Areas

8 8 NERC Reliability Coordinators

9 9 Footprints of Reliability Coordinators in Midwest

10 10 NERC Immediate Response to Blackout First hours Worked closely with NERC Reliability Coordinators Identified what had tripped and extent of outage Assessed restoration efforts Maintained open line with DOE/FERC Communicated with DHS, White House, and NRC First days Assigned project manager Established Steering Group with industry executive experts Began organizing investigation teams 90 + volunteers + entire NERC staff

11 11 Investigation Organization Overview Steering Group MAAC/ECAR/NPCC Coordinating Group MAAC ECAR NPCC MEN Study Group Project Planning and Support Sequence of Events Data Requests and Management Investigation Team Lead System Modeling and Simulation Analysis NERC & Regional Standards/Procedures & Compliance Transmission System Performance, Protection, Control Maintenance & Damage Operations - Tools, SCADA/EMS Communications Op Planning System Planning, Design, & Studies Root Cause Analysis Cooper Systems Generator Performance, Protection, Controls Maintenance & Damage U.S – Canada Task Force Vegetation/ROW Management Frequency/ACE Restoration Investigation Process Review

12 12 Data Gathering and Analysis Three fact-finding meetings August 22 September 8-9 October 1-3 Onsite interviews and inspections Secure database of outage information Extensive corroboration of data to determine facts Analysis by teams of technical experts

13 13 Root Cause Analysis Logical structure for investigating complex problems Identifies changes, conditions, actions, or inactions at each causal step Starts with final event and drills back through each branch of causal tree. Asks why? at each step. Accurate, reliable, defensible understanding of the root causes. Successfully used to investigate root causes of PJM voltage stability condition in July 1999 and established history in nuclear and defense industries.

14 14 Root Cause Analysis Phases Pre-Existing Conditions E.g. voltages, wide- area transfers, line and generator outages, etc. Sammis – Star Star – South Canton Hanna – Juniper Chamberlin - Harding BLACKOUT 16:15 16:06 15:05 Initial Focus

15 15 August 14 Conditions Prior to Blackout Planned outages Cook 2, Davis Besse nuclear plants East Lake 4, and Monroe 1 Transfers high to northeast U.S. + Ontario Not unusually so and not above transfer limits Critical voltage day Voltages within limits Operators taking action to boost voltages Frequency Typical for a summer day System was within limits prior to 15:05, on both actual and contingency basis

16 16 Warm But Not Unusual for August

17 17 August 14 Imports to Northeast-Central Compared to 6/1 to 8/13/2003

18 18 Voltages Prior to 15:05 EDT August 14

19 19 Frequency Typical for Summer Day

20 20 Blackout was NOT Caused by Heavy wide-area transfers Low voltages, voltage collapse Lack of IPP voltage/reactive support Frequency anomalies Cinergy outages starting at 12:08 East Lake 5 trip at 13:31 Contributing factor to later events, but not by itself causal to the blackout DPL Stuart-Atlanta trip at 14:02 Contributing factor to loss of MISO real-time monitoring, but not electrically significant

21 21 Outage Sequence of Events Transmission Map Key

22 22 East Lake 5 Trip: 1:31:34 PM ONTARIO 2 1

23 23 East Lake 5 Exciter Failure Causes Trip

24 24 Stuart Atlanta Trip: 2:02 PM

25 25 MISO State Estimator and Reliability Analysis MISO state estimator and contingency analysis ineffective from 12:37 to 16:04 State estimator not solving due to missing information on lines out in Cinergy then DPL Human error in not resetting SE automatic trigger Using Flowgate Monitoring tool to monitor conditions on previously identified critical flowgates

26 26 FirstEnergy Computer Failures 14:14 Alarm logger fails and operators are not aware No further alarms to FE operators 14:20 Several remote consoles fail 14:41 EMS server hosting alarm processor and other functions fails to backup 14:54 Backup server fails EMS continues to function but with very degraded performance (59 second refresh) FE system data passed normally to others: MISO and AEP AGC function degraded and strip charts flat-lined 15:08 IT warm reboot of EMS appears to work but alarm process not tested and still in failed condition No contingency analysis of events during the day including loss of East Lake 5 and subsequent line trips

27 27 Phone Calls to FirstEnergy FE received calls from MISO, AEP, and PJM indicating problems on the FE system but did not recognize evolving emergency 14:32 AEP calls regarding trip and reclose of Star-S. Canton 15:19 AEP calls again confirming Star-S. Canton trip and reclose 15:35 Calls received about spikes seen on system 15:36 MISO calls FE regarding contingency overload on Star-Juniper for loss of Hanna-Juniper 15:45 FE tree trimming crew calls in regarding Hanna-Juniper flashover to a tree PJM called MISO at 15:48 and FE at 15:56 regarding overloads on FE system

28 28 Chamberlin-Harding (3:05:41)

29 29 Chamberlin-Harding Indication of Ground Fault Due to Tree Contact as Measured by DFR at Juniper

30 30 (3:05:41) Hanna-Juniper (3:32:03)

31 31 Hanna Juniper Confirmed as Tree Contact at Less than Emergency Ratings of Line

32 32 Effects of Ambient Conditions on Ratings

33 33 (3:05:41) (3:32:03) Star- S. Canton (3:41:35)

34 34 Situation after Initial Trips 3:05:41 – 3:41:35 ONTARIO

35 35 Canton Central – Tidd (3:45:41)

36 kV Lines Overload and Cascade Near Akron

37 37 15:05:41 EDT15:32:03 EDT15:41:35 EDT15:51:41 EDT 16:05:55 EDT 138 kV Cascade Contributes Further to Overload of Sammis-Star

38 38 Sammis-Star (4:05:57.5)

39 39 Sammis-Star Zone 3 Relay Operates on Steady State Overload

40 40 Actual Loading on Critical Lines

41 41 Actual Voltages Leading to Sammis-Star

42 42 Major Path to Cleveland Blocked after Loss of Sammis-Star 4:05:57.5 PM Remaining Paths

43 kV Lines Trip Across Ohio to West ONTARIO

44 44 Generation Trips 4:09:08 – 4:10:27 PM ONTARIO

45 kV Transmission Cascade Moves North into Michigan 4:10:36 – 4:10:37 PM

46 46 Northern Ohio and Eastern Michigan Served Only from Ontario after 4:10:37.5 – 4:10:38.6 PM

47 47 Power Transfers Shift at 4:10:38.6 PM

48 48 Eastern Eastern Michigan (Detroit) Unstable Voltage and Frequency Collapse and Pole Slipping Ontario – Michigan Interface Flow and Voltages Beginning 16:10:38

49 49 Generator Trips to 16:10:38

50 50 Generator Trips – Next 7 Seconds

51 51 Overloads on PJM – NY Ties 4:10:39 PM

52 52 PJM – NY Separating 4:10:44 PM

53 53 Cleveland – Toledo Island 4:10:39 - 4:10:46 PM Cleveland Blacks Out

54 54 Northeast Completes Separation from Eastern Interconnection 4:10:43 – 4:10:45 PM North of Lake Superior

55 55 Conditions at Niagara Indicate Progressively Worsening Stability Conditions with Prior Events

56 56 Island Breaks Up: 4:10:46 – 4:13 PM

57 57 Frequency in Ontario and New York during Breakup Niagara Generation Stays with Western NY

58 58 Generator Trips – After 16:10:44

59 59 Areas Affected by the Blackout Service maintained in some area Some Local Load Interrupted End of the Cascade

60 60 Blackout Root Cause Group 1 FE Situational Awareness FE did not ensure a reliable system after contingencies occurred because it did not have an effective contingency analysis capability FE did not have effective procedures to ensure operators were aware of the status of critical monitoring tools FE did not have effective procedures to test monitoring tools after repairs FE did not have additional high level monitoring tools after alarm system failed

61 61 Blackout Out Root Cause Group 2 Vegetation Management FE did not adequately manage tree growth in its transmission rights of way

62 62 Blackout Cause Group 3 Reliability Coordinator Diagnostics MISOs state estimator failed due to a data error. MISOs flowgate monitoring tool didnt have real-time line information to detect growing overloads MISO operators couldnt easily link breaker status to line status to understand changing conditions. PJM and MISO ineffective procedures and wide grid visibility to coordinate problems affecting their common boundaries

63 63 Voltage support/reactive supply Reliability communications Computer failure response & notifications Emergency action plans and capabilities Operator training for emergencies Vegetation management Near-Term Industry Actions Responses from Control Areas and Reliability Coordinators Due December 15

64 64 Next Steps U.S./Canada Power Outage TF hearings Public hearings to allow comment on report and input on recommendations December 4 December 5 December 8 – Toronto Industry technical conference December 10 – Philadelphia NERC next steps NERC executive committees December 11 NERC committees meet January Continue investigation Near term analysis and recommendations in support of U.S. Canada Task Force Long term analysis and recommendations for NERC

Download ppt "August 14, 2003 Blackout Summary Based on Interim Report of the United States – Canada Power Outage Task Force November 19, 2003."

Similar presentations

Ads by Google