1. The Mathematics of the Great U.S. Blackout August 14, 2003 Ralph Fehr, P.E. Engineering Consultant

2. The Mathematics of the Great U.S. Blackout August 14, 2003 Ralph Fehr, P.E. Engineering Consultant Note: The photograph on the preceding slide has been circulating on the Internet since shortly after August 14, 2003. It is impressive, dramatic, and FAKE. But it does attempt to indicate the huge impact and expanse of the August 14 event, and most importantly, it makes for a cool title slide!

3. North American Electric Reliability Council (NERC) NERC is a not-for-profit company formed after the 1965 Northeast Blackout to promote the reliability of the bulk electrical system that serves North America.

4. North American Electric Reliability Council (NERC) NERC is divided into 10 regional reliability councils. Each council monitors utilities within its geographic area.

5. North American Electric Reliability Council (NERC)

6. 3 Interconnections / 10 NERC Regions

7. NERC Control Areas

8. NERC Reliability Coordinators

9. Footprints of Reliability Coordinators in Midwest

10. The August 14, 2003 Blackout was caused by a cascading series of events. What started the ball rolling?

11. A wire attached at two points sags under its own weight.

12. The amount of sag is a function of tension. whereS = span length T H = horizontal component of tension w = conductor weight per unit length Tension is a function of wire temperature and weight. Temperature is a function of several variables – but a major contributor is electrical loading. Temperature increases with electrical loading SQUARED. As the wire temperature INCREASES, the tension DECREASES. For a level span, the maximum sag D is the y-coordinate at the midpoint of the line, and is given by:

13. Effects of Ambient Conditions on Ratings

14. x y y = cosh(x) Mathematical Analysis of Root Cause Alternate Mathematical Analysis of Root Cause Cost maintenance > 0

15. Outage Sequence of Events Transmission Map Key

16. East Lake 5 Trip: 1:31:34 PM ONTARIO 2 1

17. Stuart Atlanta Trip: 2:02 PM

18. MISO State Estimator and Reliability Analysis MISO state estimator and contingency analysis ineffective from 12:37 to 16:04 –State estimator not solving due to missing information on lines out in Cinergy then DPL –Human error in not resetting SE automatic trigger Using Flowgate Monitoring tool to monitor conditions on previously identified critical flowgates

19. FirstEnergy Computer Failures 14:14 Alarm logger fails and operators are not aware –No further alarms to FE operators 14:20 Several remote consoles fail 14:41 EMS server hosting alarm processor and other functions fails to backup 14:54 Backup server fails –EMS continues to function but with very degraded performance (59 second refresh) –FE system data passed normally to others: MISO and AEP –AGC function degraded and strip charts flat-lined 15:08 IT warm reboot of EMS appears to work but alarm process not tested and still in failed condition No contingency analysis of events during the day including loss of East Lake 5 and subsequent line trips

20. Phone Calls to FirstEnergy FE received calls from MISO, AEP, and PJM indicating problems on the FE system but did not recognize evolving emergency –14:32 AEP calls regarding trip and reclose of Star-S. Canton –15:19 AEP calls again confirming Star-S. Canton trip and reclose –15:35 Calls received about “spikes” seen on system –15:36 MISO calls FE regarding contingency overload on Star-Juniper for loss of Hanna-Juniper –15:45 FE tree trimming crew calls in regarding Hanna- Juniper flashover to a tree –PJM called MISO at 15:48 and FE at 15:56 regarding overloads on FE system

21. The Chamberlin - Harding 345 kV line sags into a tree at 3:05:41. Contact with tree causes a ground fault which results in very high current. The protective relays on the Chamberlin – Harding line sense the high current and trip (de-energize) the line. A Digital Fault Recorder (DFR) at nearby Juniper Substation recorded the fault current.

22. Chamberlin-Harding (3:05:41)

23. Chamberlin-Harding Indication of Ground Fault Due to Tree Contact as Measured by DFR at Juniper y = e x sin x NOT STABLE should be y = e -x sin x

24. (3:05:41) Hanna-Juniper (3:32:03)

25. Hanna Juniper Confirmed as Tree Contact at Less than Emergency Ratings of Line

26. (3:05:41) (3:32:03) Star- S. Canton (3:41:35)

27. Situation after Initial Trips 3:05:41 – 3:41:35 ONTARIO

28. Canton Central – Tidd (3:45:41)

29. Anatomy of a Cascading Outage SourceLoad 20%

30. Anatomy of a Cascading Outage SourceLoad 0% 25%

31. Anatomy of a Cascading Outage SourceLoad 0% 33%

32. Anatomy of a Cascading Outage SourceLoad 0% 50%

33. Anatomy of a Cascading Outage SourceLoad 0% 100%

34. 138 kV Lines Overload and Cascade Near Akron

35. 15:05:41 EDT15:32:03 EDT15:41:35 EDT15:51:41 EDT 16:05:55 EDT 138 kV Cascade Contributes Further to Overload of Sammis-Star

36. Sammis-Star (4:05:57.5)

37. Sammis-Star Zone 3 Relay Operates on Steady State Overload Operating point must lie below blue curve, or line will trip. As loading on line increases, operating point moves up and to the left.

38. Actual Loading on Critical Lines

39. Actual Voltages Leading to Sammis-Star

40. Major Path to Cleveland Blocked after Loss of Sammis-Star 4:05:57.5 PM Remaining Paths

41. 345 kV Lines Trip Across Ohio to West ONTARIO

42. Generation Trips 4:09:08 – 4:10:27 PM ONTARIO

43. 345 kV Transmission Cascade Moves North into Michigan 4:10:36 – 4:10:37 PM

44. Northern Ohio and Eastern Michigan Served Only from Ontario after 4:10:37.5 – 4:10:38.6 PM

45. Power Transfers Shift at 4:10:38.6 PM

46. Eastern Eastern Michigan (Detroit) Unstable Voltage and Frequency Collapse and Pole Slipping Ontario – Michigan Interface Flow and Voltages Beginning 16:10:38

47. Generator Trips to 16:10:38

48. Generator Trips – Next 7 Seconds

49. Overloads on PJM – NY Ties 4:10:39 PM

50. PJM – NY Separating 4:10:44 PM

51. Cleveland – Toledo Island 4:10:39 - 4:10:46 PM Cleveland Blacks Out

52. Northeast Completes Separation from Eastern Interconnection 4:10:43 – 4:10:45 PM

53. Conditions at Niagara Indicate Progressively Worsening Stability Conditions with Prior Events

54. Island Breaks Up: 4:10:46 – 4:13 PM

55. Frequency in Ontario and New York during Breakup Niagara Generation Stays with Western NY

56. Generator Trips – After 16:10:44

57. Areas Affected by the Blackout Service maintained in some area Some Local Load Interrupted End of the Cascade

58. Lessons Learned Better maintenance practices Better training for system operators Better communications between utilities

59. Can it happen again? What do you think?

60. Thank you! Manhattan skyline with only emergency lighting – August 14, 2003