1. Ravi K. Iyer Information Trust Institute Coordinated Science Laboratory University of Illinois at Urbana-Champaign A Configurable Hardware Framework for a Trusted Computing Base: Application to the Power Grid 1

2. 2 Objectives Develop enabling technology to provide customizable level of trust to a significant critical infrastructure as exemplified by the Power Grid computing and communication systems. The focus is on design methods and runtime techniques to achieve application-specific level of reliability and security, while delivering optimal and timely performance.

3. Secure and Reliable Computing Base TCIP- NSF Cyber Trust Center-Scale Project- Trustworthy Cyber Infrastructure for Power www.iti.uiuc.edu Address technical challenges motivated by domain specific problems in Ubiquitous exposed infrastructure Real-time data monitoring and control Wide area information coordination and information sharing By developing science in Trustworthy infrastructure for data collection and control Wide-Area Trustworthy Information Exchange Quantitative Validation

4. Present Day Power Grid Cyber Infrastructure Control Area Coordinator - 1000’s of RTU/IEDs - Monitor and control generation and transmission equipment 10’s of control areas feed data to coordinator - State estimator creates model from RTU/IED data - Peer coordinators may exchange information for broad model - Degree of sharing may change over time Photos courtesy of John D. McDonald, KEMA Inc.

5. U.S. Power Grid Cyber Infrastructure Complexity Copyright 2003 Edison Electric Institute. Source: POWERmap, © Platts, a Division of the McGraw Hill Companies. CAISO RTO WEST ERCOT MISO TVA GRID FLORIDA GRID SOUTH PJM NYSO ISO-NE

6. 6 Increased Power Grid Trustworthiness via Secure and Reliable Computing Base Application Specific Architectures Customizable Reconfiguration New Types of Platforms

7. Approach Explore processor/OS/Application level solutions to achieve low-cost, high-performance, scalable security and reliability checking in the same framework Provide small footprint solutions that not require large amount of extra hardware or software Provide solutions that can coexist with the current generation of critical infrastructure Ensure timely detection and recovery to prevent loss of service or damage to critical infrastructure 7

8. Approach Vision: Systematically transform the computing base for application-level security and reliability Main idea: Derive application-centric checks embed them in the HW access them with OS/middleware support validate them in power-grid cyber infrastructure Considering: Both COTS and new architectures technical challenges raised by deployment/management

9. Current Generation of Low-end Devices (1) New generation devices for substation automation: Network Terminal Units (NTUs) –Combines traditional RTU (remote terminal unit) functionality with superior processing power and advanced tools for data collection and control. Capabilities –innovative data management –easy-to-use configuration program, employs Windows interface and drag-and-drop database editing –preservation of legacy components, subsystems, and data concentration applications assemble multiple databases from the available data points. ability to monitor real-time transfer of raw data to and from your NTU –support for reconfiguration or expansion for changes and additions –grow from a basic site to one with many IEDs, IED configuration and on- site setup –upgrade of RTUs to NTU capabilities with minimal difficulty. OpEn Connect NTU-7500 Advanced Control Systems

10. Current Generation of Low-end Devices (2) NTU-Substation Controller –high-performance –large database capacity –data concentrator and protocol converter applications –ability to process a large amount of data from IEDs, –interface a large number of discrete data acquisition and control devices in the substation. –use of virtual RTUs sort incoming IED and RTU data into discrete databases can be based on data type, e.g., critical and non-critical can configure Virtual RTUs to provide appropriate data subsets Design Features –distributed processing architecture; –multiple 32-bit microprocessors, –linked using a peer-to-peer type network –multiple IED isolated serial communication interfaces Operating Systems –IED: RTOS, e.g., Thread X –NTU/RTU: Linux, XP

11. Achieving Secure and Reliable Computing Base Reliability and Security Microkernel (RSM) Reliability and Security Engine (RSE) Reconfigurable processor-level hardware framework to support security and reliability Current features On-core approach Framework and modules implemented on an FPGA Available modules Malicious attack detection - Pointer taintedness detection - Information-flow signatures Transparent hang/crash detection for OS and applications Reconfigurable operating system-level kernel module to support OS/application aware security and reliability services Current features Two level hierarchy: - low-level pins interfacing with OS and hardware - high-level modules providing application-specific security and reliability techniques Available modules Application/OS hang/crash detection Transparent application checkpoint

12. Automated Design Flow Application Source Code Dynamic Execution Profile Application code annotated with critical variables Application code instrumented with interface to invoke H/W checks Path-tracking state machines Checking Expressions General-purpose Processor Reliability and Security Engine (RSE) RSE Inter- face profilingFanouts analysis Regular compilerVHDL Translation & synthesis Reliability Augmented Compiler

13. Path Tracking Static-Checking Register File PATH_CHECK Instruction Committed Write Buffer EXPR_CHECK Instruction Committed Main Memory Runtime Path DLX Superscalar with RSE Error Detected Static-Detector Module Hardware Implementation: RSE Module

14. 14 Security Partitioned Applications Intelligent Electronic Device - SEL 3351 Data Aggregator Main Processor Secure Coprocessor Protected with RSE Database Application Streaming Video Distribution Application Access Control Functions Secured Code Kernels Secure Data

15. 15 Example of Security Partitioned Applications Intelligent Electronic Device - SEL 3351 Data Aggregator Main Processor Secure Coprocessor Protected with RSE Database Application Streaming Video Distribution Application Access Control Functions Secured Code Kernels Secure Data Main application runs on COTS processor.

16. 16 Security Partitioned Applications Intelligent Electronic Device - SEL 3351 Data Aggregator Main Processor Secure Coprocessor Protected with RSE Database Application Streaming Video Distribution Application Access Control Functions Secured Code Kernels Secure Data Coprocessor allows utilization of custom checking hardware.

17. 17 Security Partitioned Applications Intelligent Electronic Device - SEL 3351 Data Aggregator Main Processor Secure Coprocessor Protected with RSE Database Application Streaming Video Distribution Application Access Control Functions Secured Code Kernels Secure Data Secure kernels provide high levels of trust

18. 18 Security Partitioned Applications Intelligent Electronic Device - SEL 3351 Data Aggregator Main Processor Secure Coprocessor Protected with RSE Database Application Streaming Video Distribution Application Access Control Functions Secured Code Kernels Secure Data Clearly Defined Interfaces Support Development

19. 19 Security Partitioned Applications Intelligent Electronic Device - SEL 3351 Data Aggregator Main Processor Secure Coprocessor Protected with RSE Database Application Streaming Video Distribution Application Access Control Functions Secured Code Kernels Secure Data Secured Data Remains on Coprocessor

20. 20 Coprocessor Integration within the Testbed Augment SEL 3351 with FPGA- based coprocessor. Nallatech FPGA Card available in PC-104+ Demonstrate Coprocessor in various applications: –Undervoltage Relay –Video Streaming Distribution FPGA Coprocessor PC-104+ Computer SEL SEL 3351 Data Aggregator

21. 21 Applications Undervoltage Load Shedding Relay: Monitor critical power data and take corrective action before system is affected. Protect against accidental and malicious data corruption using RSE. Integrate FPGA coprocessor into SEL-3351 Data Aggregator using PC-104+ interface. FPGA Coprocessor with RSE will execute security critical kernels within protected application with a high degree of Trust. Streaming Video Distribution: Provide protection for distributed distribution of Substation security camera video. Prevent malicious tampering with video feed due to bandwidth strangling by limiting each individual users bandwidth usage. RSE FPGA Coprocessor will protect small security critical kernels within the streaming application.

22. Nallatech DIME-II with Xilinx FPGA Schweitzer SEL-3351 Data Aggregator Schweitzer SEL-421 Relay Xilinx JTAG Debug Cable TCIP Testbed: Synchrophaser Relay Protected with RSE Coprocessor Integration within the Testbed

23. Backups 23

24. Approach Vision: Systematically transform the computing base for application-level security and reliability Main idea: Derive application-centric checks embed them in the HW access them with OS/middleware support validate them in power-grid cyber infrastructure Considering: Both COTS and new architectures technical challenges raised by deployment/management

25. Research and Engineering Issues Application-Oriented ThreatsGeneral Platform Threats Exploring HW support and use cases for “Tunable Trust” Between components of substation Between RTUs and the rest of the system Goal: Restrictions in bandwidth and processing power prevent security from being applied in current system Setting up testbed to experiment with emerging hardware approaches: Physical security attacks Goal: Increase assurance that IEDs/RTUs can maintain secrets despite physical attack Explore ways to update crypto and sw for devices that must last decades, unmanned HW mechanisms to enable trustworthy sharing of computation on private data “Tiny Trusted Third Parties” Draws on secure coprocessing, oblivious RAM, blinded circuits Prototyped on IBM 4758 Implementing smaller, cheaper, faster version on FPGAs Application-aware security and reliability services Setting up testbed to experiment with emerging hardware/software approaches: RSE (reliability and security engine) AEGIS RSM (reliability and security microkernel) Multicore Goal: Increase availability and resilience for general-purpose platforms exposed to network attacks Substations Control Centers and above

26. Automated Design Flow Application Source Code Dynamic Execution Profile Application code annotated with critical variables Application code instrumented with interface to invoke H/W checks Path-tracking state machines Checking Expressions General-purpose Processor Reliability and Security Engine (RSE) RSE Inter- face profilingFanouts analysis Regular compilerVHDL Translation & synthesis Reliability Augmented Compiler

27. Path Tracking Static-Checking Register File PATH_CHECK Instruction Committed Write Buffer EXPR_CHECK Instruction Committed Main Memory Runtime Path DLX Superscalar with RSE Error Detected Static-Detector Module Hardware Implementation: RSE Module