1. Context-Dependent Network Agents Specific technology goals of funded effort: distributed computation and control applications of synchronized sampling collaboration techniques Accomplishments distributed rolling horizon strategies protocols for dynamic collaboration Markov modeling and multi-mode learning context-dependent FACTS and PSS controls dependable & secure protective relaying strategies distributed power flow evaluation of reactive power control in deregulated markets remote-access real-time control emulator Next steps context detection and identification integrated CDNA strategies CDNA modeling, simulation and validation extensions to computer, traffic, biological, and C 2 networks Goals/Progress/Directions Concept Programmatic Information PI: Bruce H. Krogh Dept. of ECE, Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213-3890 ph. +1 412 268 2472 fax -3890 e-mail: krogh@ece.cmu.edu Contract Number: WO8333-05 Start and duration of funded effort: Jan. 1, 1999 through Dec. 31, 2003 Objective: Improve agility and robustness (survivability) of large dynamic networks through agents that are: widely distributed context-dependent semi-autonomous collaborative multi-modal self-improving local in sensing & influence multi-objective Physical Network G G G Agent Network Context-Dependence Collaboration Sensing & Control Learning multi-objective hybrid strategies learning, diagnostics and adaptation real-time infrastructure SKIP

2. CDNA: Progress by Subtask Task 1: Agent Templates and Modules Subtask 1.1: CDN Agent Template Implemented and studied structures for realizing various agent behaviors and capabilities for specific network control scenarios (CMU, RPI, TAMU, UM) Subtask 1.2: Module Specifications Created initial design of control agent modules separated from power system simulation program for real-time system emulation (RPI, UIUC) Subtask 1.3: Interface Design Designed and implemented input-output interfaces between MATLAB power system simulator and real-time system emulator (RPI, UIUC) Subtask 1.4: Tools for CDN Agent Construction Evaluated various algorithmic tools for constructing agent capabilities (CMU, RPI, TAMU, UM) Task 2: Restructured Power System Modeling Subtask 2.1: Definition of Operating Modes Examples of operating modes for TCSC control for voltage and stability transients (RPI, TAMU) Use of Markov decision models for defining operating modes (CMU) New method for evaluating safe transient-stability operating regimes (CMU)

3. Subtask 2.2: Online Identification of Operating Modes Analysis of data from phasor measurement units (PMU) to identify signatures of disturbances (RPI) Use of neural nets and synchronized sampling for improved dependability/security (TAMU) Subtask 2.3: Decomposition and Aggregation Convergence results for new distributed load flow computations (UM) Task 3: Agent Coordination and Learning Subtask 3.1: Development of Collaboration Strategies New system structures for distributed model predictive control (CMU) Neighbor-coordination schemes in C-Nets (collaborative networks) (CMU) Subtask 3.2: Learning Algorithms for Coordination Game-theoretic formulations and learning for multi-agent control (CMU) Multi-objective coordination (TAMU) Subtask 3.3: Local Control Strategies Multimode control of Markov decision processes (CMU) Congestion control strategies for voltage and stability transients (RPI, TAMU) Impacts of deregulation market on reactive control (TAMU) Subtask 3.4: Robust Hybrid Dynamics Conditions for certainty equivalents in switching control strategies for Markov decision processes (CMU) CDNA: Progress by Subtask (cont'd.)

4. Task 4: Real Time Infrastructure Subtask 4.1: Real-time Environment Completed demonstration system (Telelab) for remote access to the Simplex infrastructure and MATLAB Power System Toolbox (RPI, UIUC) Subtask 4.2: Robustness Features Implemented Telelab for multiple users to download and test code (UIUC) Task 5: Tests and Demonstrations Subtask 5.1: Demonstration Scenarios Multi-area scenarios for TCSC voltage and transient stability (RPI, TAMU) Scenarios for distributed multi-agent model predictive control (CMU) Multi-area load flow computation scenario (UM) Subtask 5.2: Application Simulator MATLAB Power System Toolbox-Simplex Telelab (RPI, UIUC) Subtask 5.3: Visualization Tools Evaluated power system simulation tools for protective relaying scenario presentation (MATLAB and EUROSTAG) (TAMU) Task 6: The Virtual Institute Subtask 6.1: Customization of Lire Upgraded LIRE for faster access and e-mail notification services (CMU) Subtask 6.2: Computer-Based Collaboration Web-based distribution of project reports and results (All participants) CDNA: Progress by Subtask (cont'd.)

5. Overview of CDNA Accomplishments Physical Network G G G Agent Network - collaboration techniques - distributed computation distributed control applications of synchronized sampling multi-objective hybrid strategies real-time infrastructure - learning - diagnostics - adaptation

6. Specific CDNA Accomplishments B C A distributed - rolling horizon strategies - power flow protocols for dynamic collaboration - Markov modeling - multi-mode learning context-dependent FACTS controls context-dependent PSS controls dependable & secure protective relaying strategies remote-access real-time control emulator evaluation of reactive power control in deregulated markets

7. CDNA Universities and Principal Investigators

8. Current CDNA Collaborations 1. U of Minn 3. TAMU 2. CMU 4. RPI 5. UIUC agents arch. & collaboration ABC system switching control implementation Telelab-MATLAB real-time emulator ABC system voltage studies synchronous data modeling economic markets decentralized comp. biological networks distributed power flow stability region comp.

9. Selected Results

10. Transmission System Security Analysis using Network Agents Security analysis is done by running power flows We are seeking methods of solving distributed power flows using agents (computer systems) in multiple control systems We would like to eliminate the idea of a “security center” approach. University of Minnesota

11. ISO Trends –Getting larger –Standard data formats –Less functionality in regional systems

12. Networked Control Systems –Region can be any size –Can extend to any number of regions –Regions retain original functionality –Aggregate has same functionality as large area control system

13. Poor Results from Multiple Processors solving One Power Flow Divide power system into several areas Solve each area on a separate processor Communicate results of each processor with other processors Communication time is greater than time saved by using multiple processors Try to minimize data that must be sent between processors University of Minnesota

14. Security Analysis and multiple processors Security analysis requires solving multiple power flows, one for each contingency case When calculation on one case is completed, start communication While communication is being done, start calculation on next case University of Minnesota

15. Multiple processors solving multiple outage cases – calculation overlaps communications Greatly increased speed University of Minnesota

16. Methods Tested Gauss-Seidel Method –Filtered Solution –Block Border Gauss Method Conjugate Gradient Methods Reduced Orthogonal Subspaces Diakoptics University of Minnesota

17. Results of research at the University of Minnesota Communication is the bottleneck Methods with only neighbor to neighbor communications require too many iterations to solve Methods that exchange ‘sensitivities’ require fewer iterations  Some entity must calculate the sensitivities We have reduced the sensitivity data that must be exchanged to a minimum without sacrificing speed University of Minnesota return

18. COORDINATION OF DISTRIBUTED, AUTONOMOUS AGENTS Sarosh Talukdar, Eduardo Camponogara, Haoyu Zhou ACCOMPLISHMENTS Extension of Model Predictive Control (the Rolling Horizon Strategy) to serve as a coordination framework for autonomous, distributed agents. Development of a test-bed for coordination and learning strategies in networks of stationary and mobile, autonomous, distributed agents Carnegie Mellon University

19. EXTENSION OF MODEL PREDICTIVE CONTROL TO AUTONOMOUS, DISTRIBUTED AGENTS The communication links between agents define a set of overlapping neighborhoods. Neighbors of an agent = adjacent agents For each agent, the system’s variables are divided into three sets: X: proximate variables (those variables the agent can sense or control) Y: neighborhood variables (those variables the agent’s neighbors can sense or control) Z: remote variables (all other variables) Carnegie Mellon University

20. SUFFICIENT CONDITIONS (for the successfull extension of model predictive control to distributed, autonomous agents) If: the overall-system-problem is feasible the overall-system-problem is convex the overall-system-problem is decomposed into sub-problems for the agents, such that each sub-problem matches its agent exactly (Z is empty for each agent) each agent uses an iterative, interior point method to solve its sub-problem each agent communicates the results of each iteration to its neighbors the agents in each neighborhood work serially (one after the other) Then:the agents’ iterations will converge to an optimal solution of the overall-system-problem Question: are these conditions necessary? Carnegie Mellon University

21. COORDINATION HEURISTICS There are at least two families of heuristics by which the conditions on: exact matchings of agents to sub-problems problem-convexity communication frequency within each neighborhood serial work within each neighborhood can be demonstrated to be unnecessary for representative networks. These families are based on: 1.tightening the resource constraints by the inclusion of “resource margins” 2.learning models by which each agent can predict the actions of its neighbors These heuristics allow the agents to work asynchronously (in parallel, each at its own speed) on realistic (non-convex) control tasks. Carnegie Mellon University

23. Experiments A Heuristic: Constraint Margins When is the C-Net Context-Dependent? The agents modify the tasks, {(P m )}, to make up for the varying context C-Net Penalty (%) Number of Pendulums  APE with mutual help Pendulums are randomly disturbed Agents adjust {(  m )}  Asynchronousprox. exch. (P m ): G m (X m,U m,Y m )  -  m 1) Implement Constraint Margins 2) Collaboration Protocols: 3) Context Dependency APE APE M-Help Carnegie Mellon University

24. Context Dependent Switching and Learning Contexts: different buyers and sellers (decision-makers) with the same Objective: to develop bidding strategies for their own profits. So many uncertainties for a decision-maker, G1, for example, –Unobservable infinitely many possible combinations of bidding from G2, L1, L2. –Transmission line capacity variations. G1 L1 Zone 1 G2 L2 Zone 2 Transmission Line An application in the deregulated power market Carnegie Mellon University

25. Switching: –Using finite number of modes to describe the infinitely many possibilities. –Designing optimal strategy for each mode and switching between these optimal strategies. Learning: –Performance measurement for the switching among the current set of strategies. –When the performance is not satisfactory, a new mode will be identified and corresponding optimal strategy will be designed. Carnegie Mellon University

26. Multi-Mode Markov Decision Process Model Markov System X k, k = 0, 1, …, state space S. System Mode  k, k = 0, 1, …,  = {1, 2, …, |  |}. Action set U and action subset U(s)  U for each s  S. A (stationary) policy L is a mapping from S to U such that L(s)  U(s) for each s  S. At epoch k, after an action u  U(X k ) is applied, –Transition to s with probability –Reward incurred –Mode jumps to  k+1. Objective: find optimal policy sequence L 0, L 1, … to maximize performance Carnegie Mellon University

27. Switching Based on Certainty Equivalence (CE) Let L  * be the optimal policy when  k is a constant . Suppose  k is a Markov process with transition matrix Q. CE Switching Strategy: Apply L  * when  k = . When is the CE strategy optimal? || I - Q ||   (1 -  ) B /(2  A), A and B computable How well does CE switching do in general? ||J CE - J * ||   2  A ||I - Q||  / (1 -  ) 2 J CE : performance under CE switching J * : optimal performance. Carnegie Mellon University

28. CE switching: Use the MLE of  k When is CE switching optimal? 2(1 - max  p(  ))  (1 -  ) B /(2  A) How well does CE switching do? ||J CE - J * ||   4  A(1- max  p(  )) / (1 -  ) 2 J CE : performance under CE switching J * : optimal performance. CE Strategy for Unobservable Modes Carnegie Mellon University

29. Simple Example Stationary policies for each mode: –If G2 always bids $14, $19 and $25, G1 bids $10, $15 and $20. Case 1: –G2 bids randomly with prob. 0.2, 0.2, 0.8 –G1’s optimal bidding strategy: Always bid $20 - CE strategy! Case 2: –G2 bids randomly with prob. 0.3, 0.3, 0.4 –G1’s optimal bidding strategy: Always bid $15 - not a CE strategy! G1 L G2 G2 Capacity: 1000MW G2’s possible bids: $14, $19, $25/MW G1 Capacity: 500MW G2’s possible bids: $10, $15, $20/MW 3 possible load demand levels: 500MW, 800MW, 1200MW with probabilities 0.3, 0.4, 0.3 Interest rate: 0.1%  = 1/(1+0.1%) Carnegie Mellon University return

30. Research at TAMU: Objectives Survivability and Protection of the system for Transient voltage, angle, oscillation, long term voltage stability crises. Overflow problems. Protective relaying Responsibility evaluations of Loop flow Problems. Market Efficiency for Generation Dispatch Problems. All the problems are coupled, objectives are sometimes conflicting. Texas A&M University

31. CDNA Interpretations Detection agents to detect transient angle stability, voltage stability, oscillation, long term voltage crises using acceleration angle velocity, line flows, voltage profiles, etc. (Security Margin Monitoring) Stabilizing agents, congestion control agents, auction agents, protection agents, performance control agents (to compromise different objectives.) Need to know contexts to switch among agents for survivability, protection and market efficiency. CDNA activates the needed control for best performance. Texas A&M University

32. Accomplishments Transient angle stabilizing controls using TCSC, SMES, SVC, Braking resistors. Stabilizing controls for transient voltage stability using TCSC. Generation dispatch using auction agents. Flow Decompositions of bilateral trades for responsibility evaluations. Demonstrate interactions between market policies and reactive power controls: Stable financial systems imposed on a stable engineering system may cause overall instability. Bad incentives and misconceptions. Texas A&M University SKIP

33. Work to be done in 2000 Detection Agents for Transient voltage, angle, long term voltage stability crises. Security Margin Monitoring for long term and short voltage problems. Responsibility evaluations of loop flows using Flow Decompositions. Congestion controls using FACTs. Demonstrate the use of Protection Relays as Structural Controls to avoid cascading failures.. Texas A&M University

34. Some Highlights Key misconception on reactive power controls are identified. We demonstrate using a simple BPA system why these concepts are wrong. Six questions are clarified using simulations. Pricing based on these wrong concepts may lead to system instability. Financial incentives should be based on solid engineering foundations. Texas A&M University

35. Q1: Is Voltage Control Effect of Generators Local by nature? What are the impacts on reducing the Var reserve? No. It is system-wide. And reduced Var reserve will have system-wide impact. Reduced reserve will also cause voltage transient stability, which collapsed in seconds. Q2: How ULTC affects Voltage Stability? In many cases they harm the security. Texas A&M University

36. No, definitely not. Q3: Can Intensive Use of Smart Shunt Banks at Load Areas Replace Dynamic VAR Reserves of Generators? Q4: What is the impact of Real Loads on Voltage Stability? They have substantial impacts. Texas A&M University

37. Q5: How do Load Characteristics Impact on Security Margin? They have substantial impact. Q6: Will a stable financial system imposed on a stable engineering system destabilize the whole system? Yes, definitely. Wrong incentives and misconception of reactive power system can destabilize the whole system. Interactions between financial system and engineering systems need to be investigated. Texas A&M University

38. Conclusions Not all VARs are created equal. Misconceptions on voltage stability are demonstrated. New findings will enable us to accurately evaluate reactive power provisions from generators and other devices in a deregulated power market, such as power pool market, bilateral trade market or compatible market. Texas A&M University

39. Identified needs: Reduce dependency on setting inaccuracy Improve selectivity between permanent and temporary faults Improve security/dependability Introduce Coordination between Control and Protection Protective Relaying Texas A&M University

40. Protective Relaying Defined New Protective Relaying Agents: #1 Neural Net (NN) Algorithm for Fault detection and classification #2 Synchronized Sampling (SS) algorithm for fault location #3 Coordination Between NN and SS #4 Coordination Between NN, SS and Control Texas A&M University

41. Protective Relaying Developed Context Dependent Approach: Learning (training) for NN Agents Line Model (on-line parameter estimation) for Synchronized Sampling Agents Texas A&M University

42. Protective Relaying Introduced New Performance Benefits: Better relaying (dependability/security) Better reclosing (recognition of permanent vs temporary faults) Better control (preventing cascading outages) Texas A&M University

43. Protective Relaying Introduced New Evaluation Approach: Definition of future use of modeling and simulation tools (local and system events) Use of Matlab customized software for evaluation of individual protective relaying agents Use of Eurostag software for evaluation of system-wide interaction among agents Texas A&M University return

44. Real-Time System Emulation Based on inputs from UIUC, developed a preliminary power system simulator with external control from a remote computer Simulator is MATLAB based; communication protocol is SOCKET Demonstrated with the ABC system; external control switch between several control options Rensselaer Polytechnic Institute

45. Power System Dynamic Monitoring Worked with ISO-NE, NYISO, and NYPA to obtain monitored power system disturbance transient data from about a dozen Dynamic Recording Devices (from several vendors) Developed a rule-based Event Identifier for classifying system disturbances; next step will be the development of an advanced identifier using detection filters In the process of using data obtained from several different monitors for the same event to analyze interarea oscillations Rensselaer Polytechnic Institute

46. Control Design Proposed alternative controller structure using remote measurements as feedback signals Controller structure to handle communication delay Further development of linear matrix inequality techniques to control systems with parametric dependence Rensselaer Polytechnic Institute

47. Scenarios Introduce two contingency scenarios for the ABC system, in addition to the normal operating condition, and design controls for the contingencies; will continue to develop additional scenarios for the system Rensselaer Polytechnic Institute return

48. Real Time Infrastructure How to support the deployment of control agents in real time reliably without shutting down the normal operations is an important concern. Telelab integrates WWW service with a fault tolerant dynamic real time architecture, the Simplex architecture. Telelab architecture gives you the ability –to add or replace application software components on the fly without shutting down its operation. –to protect the system operation and the integrity of equipment from bugs that could be introduced by changes. University of Illinois at Urbana Champaign

49. Telelab: Remote Lab Interface Win98/NT * important Win98/NT * important Win98/NT * important Win98/NT * important LynxOS Simplex annotated, pre-recorded presentation (e.g. HTML) (in case of communication failures) CORBA A/V Streams CORBA A/V Streams Demo available at www-drii.cs.uiuc.edu University of Illinois at Urbana Champaign SKIP

50. Next Step: A Sample Power Network G G G G G G Load load Circuit Breaker University of Illinois at Urbana Champaign

51. “Sympathetic” Relay Tripping: A Model Problem –Background: Short circuit and temporary overload are very different. But they are treated as if they are the same problem due to the lack of coordination. Local response could lead to cascading failures that bring down a large portion of network. –Coordination Context: Triggering event: a relay open followed by neighboring relay open Event network: SS sample locate the fault, inform overload relays to hold and related nodes to do power rerouting/load shedding Data network: continue monitoring and report the overload situation on the relays in the holding mode Control network: agents change from normal control to overload management to bring the relays from holding mode to normal mode University of Illinois at Urbana Champaign

52. “Sympathetic” Relay Tripping: Model Problem - cont’d For each chain of events there should be a coordinated response. Value of information: for each fault event there can be two solutions. The CDNA solution using information or throwing resources at it. This allows us to compute the resource equivalence of CDNA. Research on cascaded failures (network system instability): do we have parallels in Internet or other forms of network reactions, where a coordinated response could have prevented cascading failures. Will any ideas in Internet congestion control useful for power networks? University of Illinois at Urbana Champaign

53. A Sample Power Network Failure G3 G4 G6 G5 G2 G1 Load load Circuit Breaker overload G1 or G2 could become unstable unless controllers are switched The open of the overload lines could propagate the failure to the entire region We need to stabilize G1 and G2 controllers and re-adjust G3,4,5, 6 and normalize the overloaded lines quickly University of Illinois at Urbana Champaign

54. CDNA Simulation –Contingency management and agent based control testing cases to implement agent and context management. Getting RPI’s sample system implemented. Getting CMU’s agent sample system implemented. G1, 2 (MATLAB) Agents for G1, 2 (Simplex) G3, 4,5, 6Network Agents for Load Management Shared contexts Telnet Net Back Door TeleLab inteface TeleLab interface Agents G3,4,5,6 University of Illinois at Urbana Champaign return