1. Scalable Coupled ICT and Power Grid Simulation - High-performance Coupled Transmission, Distribution, and Communication Simulation Tool 15PESGM2794 Liang Min Lawrence Livermore National Laboratory HPC for Power System Planning Panel Discussion July 29, Denver CO 1 This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes. LLNL-PRES-673302

2. Decoupled transmission and distribution simulations are insufficient for complex smart grid systems 2500-house distribution system model 2500-house distribution system model with high penetration of PV Renewables and electric energy storage in the distribution network will have significant impacts throughout the entire network LLNL-PRES-673302 2

3. We have built a HPC architecture to couple T&D simulation to address pending complex smart grid systems 3 f, g and h are all functions of V 1, V 2,, and The grid simulation assumption was made that the time-scales on the distribution network do not impact the transmission system. LLNL-PRES-673302

4. We have developed our own transmission simulator and enhanced the distribution simulator for HPC We have built our own transmission simulator prototype. It is written in C++ and use SUNDIALS/IDA and KINSOL as the solvers. Software components are independent, reusable, and replaceable. Distribution simulator (GridLab-D) was modified and enhanced for HPC. LLNL-PRES-673302

5. Test Case – Coupled Transmission and Distribution System Model PG&E Feeder Model IEEE 13 Feeder Model WECC Transmission Model Assigned one distribution feeder simulation to one core, the whole transmission simulation to one core. Increased load on Bus 140 by connecting more distribution feeders to that transmission bus and monitored what happens to the bus voltage. Goal is to run larger distribution problem in same amount of time LLNL-PRES-673302

6. As CAISO studied, after the SONGS retirement, voltage stability collapse became the limiting constraint in LA basin. The urgency to scale-up demand response is high to maintain a reliable electric system, particularly in Southern California, in the absence of the San Onofre Nuclear Generating Station (SONGS). Test Case – Using scaled-up demand response at the distribution level to offset the need for load shedding to avoid voltage collapse LLNL-PRES-673302

7.  We need a power systems and communication co-simulation to answer very important questions:  What will happen to the electric grid: - If data is dropped or delayed? - If data is modified in transit? - …  With this capability, we can help utilities better design their wide- area control schemes and ensure system security and reliability. NIST Smart Grid Reference Diagram The interdependencies of communication and power systems are becoming increasingly important  Smart grid = Electricity infrastructure + “Intelligence” infrastructure. Grid transformation also needs coupled power systems and communication network simulations LLNL-PRES-673302

8. Time advancement is key for achieving correctness in federated systems LLNL-PRES-673302

9. Small toolkit for coupling continuous and discrete time simulations. It provides – Time control for advancing state of federated simulators – Communication between objects in federated simulators Designed for HPC – Asynchronous API design – MPI used as communication layer – Parallel conservative granted time window synchronization algorithm We have developed a FSKIT (Federated Simulation Toolkit) for high-fidelity and cross-domain modeling and analysis LLNL-PRES-673302 B. M. Kelley, P. Top, S. G. Smith, C. S. Woodward, and L. Min, “A federated simulation toolkit for electric power grid and communication network co-simulation “,2015 Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES), Seattle, WA, USA, April 13, 2015.Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES),

10. Original Single Line Diagram IEEE 39-bus System Resultant Communication System Model Test Case - Coupled Transmission and Communication Network Model LLNL-PRES-673302

11. We could vary the line latency and the throughput to assess different control schemes. As the WAN latency increases, trip times increase, which affects system voltage recovery. Supervisory (master agent) wide area control scenario: fault at Bus 4 at t=0.2s and clear the fault at t=0.35s Ad-hoc (peer to peer) protection relay systems scenario: Bus fault at Bus 4 at t=0.2s and clear the fault at t=0.25s Delayed Voltage Recovery due to long fault clear time Test Case – Using the co-simulator to validate different protection control schemes LLNL-PRES-673302

12. To address the need of a grid connected testing facility for large-scale smart grid hardware and software testing, we are interfacing the scaled-up coupled transmission and distribution simulator at LLNL and the PG&E’s Distribution Test Yard to form a close-loop validation environment. We are extending this capability to support industry to build large-scale HIL for smart grid technologies testing LLNL-PRES-673302

13. Thank You 13 LLNL-PRES-673302