1. IEEE LCN SGNI, Denver, CO, October 2010 1 Open Cyber-Architecture for Electrical Energy Markets M. Yuksel, K. Bekris, C. Y. Evrenosoglu, M. H. Gunes, S. Fadali, M. Etezadi-Amoli, and F. Harris {yuksem, bekris}@cse.unr.edu, cevrenosoglu@unr.edu, mgunes@cse.unr.edu, {fadali,etezadi}@unr.edu, fredh@cse.unr.edu University of Nevada, Reno

2. IEEE LCN SGNI, Denver, CO, October 2010 US Energy Market Map Energy markets are governed by Independent System Operators (ISOs) in different regions of the North America 2      

3. IEEE LCN SGNI, Denver, CO, October 2010 3 Power Grid Today Need for decentralization of SCADA The amount of data being collected and processed by SCADA systems is too large Hard to implement distributed control functions with centralized SCADA operation Need for extensive information exchange Inter-ISO information exchange is daunting and prone to human error Market and business motivations constrain the extent of inter- ISO information exchange Need for more responsive operations to major failures Lack of automated inter-ISO information exchange causes failures to cascade

4. IEEE LCN SGNI, Denver, CO, October 2010 2003 Blackout in Northeastern US A transmission line failure in the Midwest ISO was not detected due to a malfunctioning topology processor further caused other failures eventually disseminating through the Northeastern part of the interconnection “The NYISO had received no notifications or advisories from other control areas and thus, had no awareness of the precursors to the blackout” [NY ISO 2005] 4

5. IEEE LCN SGNI, Denver, CO, October 2010 5 Multi-Owner Large-Scale Infrastructure Systems Most of these are problems extensible to large-scale infrastructure systems: with multiple owners with trust boundaries with market constraints with fate-sharing Information sharing regarding system state is important for fate-sharing systems activities of individual components may affect the whole system Physical infrastructure is hard to adapt So, a “cyber-architecture” enabling information exchange and openness is key to fighting against major failures.

6. IEEE LCN SGNI, Denver, CO, October 2010 6 Open Cyber-Architecture … to provide the means to increase information sharing through more regulated means and essentially make it part of the physical system itself even to the extent that the domain owners may not be able to avoid sharing of some of the market related information.

7. IEEE LCN SGNI, Denver, CO, October 2010 7 Cyber-Architecture: Open vs. Closed Open Cyber-ArchitectureClosed Cyber-Architecture distributed decision-making at smart subsystems centralized decision-making at operation centers integrated, device-level, open communications disparate, domain-level, proprietary communications increased automated controlmanual, human-driven control reactive human involvement only in emergencies proactive human involvement with online operator

8. IEEE LCN SGNI, Denver, CO, October 2010 8 Existing Power Grid: A Closed Cyber-Architecture View Regional Operation Center A Regional Operation Center B Regional Operation Center C Regional Operation Center D Communication and Control Lines Sensing and Control Subsystems Communication SCADA & Energy Management Systems Information sharing between different regions of the power grid is limited to minimum levels

9. IEEE LCN SGNI, Denver, CO, October 2010 9 Power Grid: An Open Cyber-Architecture View

10. IEEE LCN SGNI, Denver, CO, October 2010 10 OCA: Key Components Integrated Secure Communication to provide the means to share information among subsystems (or components) of the infrastructure. Self-Healing via Automated Control that can use shared information while safeguarding market constraints and can handle large amounts of information in crises at speeds beyond human capabilities. Distributed Planning via Smart Subsystems to provide individual components with the planning and learning capability required for a robust infrastructure than can respond to unexpected events. Effective Human Interface including visualization tools, that will allow human operators to effectively utilize the available data to implement business policies or deal with emergencies.

11. IEEE LCN SGNI, Denver, CO, October 2010 11 Secure, Reliable, and Scalable Communication Infrastructure Reliable Delivery of Critical Infrastructure State Information communication protocols secure by design In-Network Aggregation and Filtering of Intra-ISO State reduce the amount of state to be sent to other ISO domains filter highly proprietary data

12. IEEE LCN SGNI, Denver, CO, October 2010 12 Importance-Based Network Protocols Timely and efficient routing and dissemination of data proactive flooding of the minimum state data required to detect risk of an important event E.g., voltage and current levels of major power transmission lines E.g. failure of a power transmission line reactive on-demand transfer of detailed state data following detection of a risk of a major event. Flash crowds from peer-to-peer literature multicast

13. IEEE LCN SGNI, Denver, CO, October 2010 13 Mitigating Cascading Events Distributed smart decision-making – Distributed AI Establish a joint coordinated plan when possible But yet, be autonomous if left alone

14. IEEE LCN SGNI, Denver, CO, October 2010 Securing Inter-ISO Communication Blind Processing establish a secure communication channel between trusted processes concealed from rest of system including root processes hence system administrators enable exchange of sensitive data between processes in different systems with enhanced privacy improve information sharing between (potentially) competing entities 14

15. IEEE LCN SGNI, Denver, CO, October 2010 Blind Processing Idea Sensitive data is transmitted via secured channel to processes running in an isolated environment 15 P1 A1 P2 A2 Concealed Open Domain-ADomain-C P3 C1

16. IEEE LCN SGNI, Denver, CO, October 2010 Blind Processing Blind processing consists of blind communication blind execution Traditional security mechanisms: protect transmission channel and processing environment from third parties Blind processing: also, protect the data from root processes of the system processing the data 16

17. IEEE LCN SGNI, Denver, CO, October 2010 Blind Processing Mechanisms We need a mechanism to attest a remote system to have a “well behavior” A host identity certificate does not guarantee that admins are not interfering with data Software cannot be directly trusted Kernel itself is not trustworthy 17

18. IEEE LCN SGNI, Denver, CO, October 2010 Blind Processing Mechanisms We need an immutable root to trust Hardware that cannot be easily tampered with Trusted computing mechanisms: utilized for blind processing Trusted Computing Group: an industry-led initiative to provide security primitives that can be utilized to establish trust relationships between systems or components of a system 18

19. IEEE LCN SGNI, Denver, CO, October 2010 Blind Processing Model 19 Hardware Layer Legacy O.S. App Legacy O.S. App Critical Application Configuration Software Layer Security Kernel Trusted Computing Support Software Interaction and Monitoring Layer Hardware Resource Management Layer TPM

20. IEEE LCN SGNI, Denver, CO, October 2010 20 Distributed Control under Market Rules Market constraints can be ambiguous and conflicting with each other owner A: “accept the information as correct if it is coming from X” owner X: “try to cooperate with A more since it is cheaper to buy from A” IF (Premise) THEN (Formula) Distributed – delay between subsytems’ control operations How to achieve stable and efficient control under such distributed fuzzy rules and constraints?

21. IEEE LCN SGNI, Denver, CO, October 2010 21 Summary Power Grid reliability and efficiency requires more inter-ISO information sharing A cyber-architecture that enables open sharing of intra-ISO state Importance-based network protocols Blind processing for motivating more sharing of proprietary information Distributed planning Fuzzy control rules and constraints to capture market dynamics

22. IEEE LCN SGNI, Denver, CO, October 2010 22 Thank you! THE END