Presentation is loading. Please wait.

Presentation is loading. Please wait.

Presented by Members: Tom Botello - Southern California Edison Terry Baker - Platte River Power Authority Reliability Coordination Task Force Report to.

Similar presentations


Presentation on theme: "Presented by Members: Tom Botello - Southern California Edison Terry Baker - Platte River Power Authority Reliability Coordination Task Force Report to."— Presentation transcript:

1 Presented by Members: Tom Botello - Southern California Edison Terry Baker - Platte River Power Authority Reliability Coordination Task Force Report to Finance and Audit Committee December 5, 2012

2 2 FAC Assignment Summary of the Issue/Finding/Conclusions Real-time Operations Operations Support Western Interconnection Synchrophasor Program (WISP) Compliance Training Recommendations Report Agenda

3 3 Terry Baker Tom Botello John Cupparo Graham Edwards John Meyer Reliability Coordination Task Force (RCTF) Members

4 4 July 18 Kick-off meeting in Portland, Oregon August 15 In-person meeting in Denver August 24 Meeting by Webinar October 22 In-person meeting at the Vancouver RC November 20 Meeting by Webinar November 23 Meeting by Webinar November 27 Meeting by Webinar December 4 In-person meeting in Salt Lake City RCTF 2012 Working Sessions

5 5 RCTF to examine the following: o Original scope of RC duties and expenses o Current scope of duties including expanded duties since 2009 o Future scope of duties and expenses Analyze key drivers for operational and budgetary change, WECC’s response to those drivers, and provide recommendations FAC Assignment

6 Operations Planning Current - day/Real-time Support Next-day Study Data Validation and Mapping Seasonal SOL Coordination and Validation Emergency Plan Coordination Real-time Operations Monitor and analyze real-time system conditions Coordinate system restoration activities Proactively respond to and mitigate potential adverse impacts to the BES Communicate with and direct corrective actions of registered entities during actual SOL exceedances Identify and mitigate potential/actual IROLs in real- time operations horizon Compliance Document and Evidence Management Audit Preparation Event Analysis Standards Development Compliance Reporting Disturbance Reporting Operations Support Application Architects Application Support DTS Support EMS Modeling RAS Engineer Architecture Support Communications IT Infrastructure Systems Administrator WISP Broaden deployment of phasor measurement units throughout the Western Interconnection Implement telecommunication network necessary to support a large- scale production synchrophasor system Install synchrophasor applications for Western Interconnection Increase ability to integrate variable generation Enhance power system analysis capability/model validation Training Develop/Deliver in-house training to new RC trainees and current RC staff to ensure compliance with mandatory standards and retention of NERC RC System Operator certification Conduct just-in- time training for new tools, applications, procedures, and processes Update job task analysis Develop/deliver NERC-certified restoration training to BAs, TOPs, and GOs Other Direction and Oversight Administrative Support WECC and NERC Committee Work WECCRC.org Development Membership Request Management Existing WECC RC Functions and Current Staffing Current FTE – Filled Current FTE Approved – Unfilled Totals

7 7 FERC/NERC Requirements o Highest level of authority responsible for the reliable operation of the BES o Authority to prevent or mitigate emergencies in next-day analysis/real-time o Wide Area view of BES (situational awareness) o Tools, processes, and procedures o Broad purview to enable IROL calculation(s) that may be based on parameters of transmission systems beyond any TOPs vision Summary of the Issue

8 8 Reliability Coordination in the Western Interconnection o Past – Reliability through collaboration  Hosted by utilities – CAISO, BPA, WAPA  Use of facilities, tools, and personnel o Present – WECC’s Responsibility  Independent facilities, tools, and responsibilities  Compliance obligations questioned  Reliability vulnerabilities exist o Future – Implement RCTF Recommendations  Increase staffing in critical areas  Meet regulatory compliance requirements  Reduce reliability vulnerabilities Summary of the Issue

9 9 Summary of RCTF Findings Additional personnel are recommended in the following areas: o Real-time operations  Broad purview to enable IROL calculations (granularity, specific focus)  Additional real-time tools, processes, procedures o Operations support  Creation of operating plans, procedures (seasonal and next-day studies)  System Modeling and model validation  IT application support requirements o Compliance support o Training

10 10 Summary Conclusions WECC has a high level of responsibility and is at risk for reliability and compliance issues o Staffing is a major factor o Tools are a major factor Emergent tools and Standards will add to these risks o WISP-related displays, data, training o PER-005 and EOP-005

11 11 Real-Time Operations RC Contingency Analysis Four RC System Operators on shift on September 8, 2011 o Prior to loss of North Gila-Hassayampa 500-kV line, there were 287 harmful contingencies identified by RTCA o RTCA run following line loss identified 632 harmful contingencies o Between start of event and SONGS units tripping (less than 11 minutes) 1,185 alarms were generated

12 12 Real-Time Operations Similar events to SW Outage reveal RC real- time staffing concerns Ability to monitor current system specifics questioned by regulators o Increase in required monitoring of RAS o Emergent tools such as WISP o Sub-100-kV facilities Specific interconnection area knowledge by RC System Operators needed to meet requirements RC phone calls per quarter have increased to over 37,000

13 13 Real-Time Operations

14

15 Loveland RC Total Load: MW Total BAs: 16 Total TOPs: 25 Vancouver RC Total Load: MW Total BAs: 21 Total TOPs: 29

16 Vancouver RC Total Load: MW Total BAs: 16 Total TOPs: 17 Loveland RC Total Load: MW Total BAs: 15 Total TOPs: 24 CA/MX RC Total Load: MW Total BAs: 6 Total TOPs: 13

17 17 The RC must create and maintain operating plans, procedures, and processes o This documentation must:  Identify RC System Operator actions taken to address specific system conditions and events  Reflect compliance to mandatory NERC, FERC, and Regional requirements, policies, and procedures Currently no specific position to perform this function Real-Time Operations - Documentation

18 18 Additional personnel are recommended in order to: o Monitor the Western Interconnection to the level of granularity expected by FERC, NERC, and the NPCC o Respond effectively to the volume of alarms and RTCA results during an event to determine appropriate corrective action o Create and maintain Operating Plans, Procedures, and Processes as required by NERC Standards Real-Time Operations − Findings

19 Real-Time Operations Monitor and analyze real-time system conditions Coordinate system restoration activities Proactively respond to and mitigate potential adverse impacts to the BES Communicate with and direct corrective actions of registered entities during actual SOL exceedances Identify and mitigate potential/actual IROLs in real-time operations horizon FTE Totals Current FTE – Filled Current FTE Approved – Unfilled Future FTE Projection WECC RC Function and Recommended FTE Increase

20 20 SW Outage event and regulatory findings revealed seasonal planning gaps o Not all WECC paths studied seasonally o TOPs coordination weak o Incomplete WECC contingency analysis Mitigating reliability vulnerabilities o With an enhanced seasonal process and more thorough assessments, the event may have been predicted Operational Planning (Seasonal)

21 21 Operational Planning (Next-Day) Why did the RC next-day study not see the problem that caused SW Outage event? o Only study next-day peak o Study automation incomplete o Need to assess sub-100-kV facilities o Forecast data quality inputs need improvement and are too numerous  Number of scheduled outages continues to increase  A million rows of unit commitment data daily

22 22 Additional personnel are recommended to perform the following activities: o Seasonal Planning:  Take lead in implementation of an enhanced process with more thorough assessments  Participate in the resolution of identified gaps in the WECC Regional process o Next-Day Study:  Complete study automation  Assess sub-100-kV facilities  Ensure quality of data inputs Operations Planning - RCTF Findings

23 Operations Planning Current -day/Real-time Support Next-day Study Data Validation and Mapping Seasonal SOL Coordination and Validation Emergency Plan Coordination FTE Totals Current FTE – Filled Current FTE Approved – Unfilled Future FTE Projection WECC RC Function and Recommended FTE Increase

24 24 Model Impacts to Situational Awareness Three ways to improve accuracy of RTCA: o Increase accuracy of facility ratings and SOLs o Increase number of RAS modeled o Expand modeling of lower-voltage networks that impact the BES  Sub-100-kV systems that parallel higher voltage networks  Sub-100-kV systems with both automatic and manually-switched reactive devices Operations Support – System Modeling

25 25 Operations Support – System Modeling RAS Engineer and EMS Modeling – WECC RC System Operators are required to improve awareness of the impacts of RAS Continuing increase in the number of WECC RAS A key component of improved awareness – RAS modeling in the EMS and real-time contingency analysis (RTCA) Only 45 of 206 RAS modeled in RTCA

26 26 Operations Support – System Modeling

27 27 Operations Support – System Modeling

28 28 Addressing Model Deficiencies WECC RC needs to be more proactive in the modeling process by increasing the frequency of model reviews with TOPs o West-wide System Model and Base Case Reconciliation Task Force (WBRTF) o Develop process for periodic review and sign off of model accuracy o Add more power system measurements Share data and application results Operations Support – System Modeling

29 29 Applications and IT areas of responsibility include, but are not limited to: o coordinated outage scheduling o EMS system support o load and interchange forecasting o operating system maintenance o change management o cyber security administration o networking Operations Support - Applications and IT

30 30 Application Architects and Support areas of responsibility include, but are not limited to: o support planning of IT initiatives to evaluate and determine new tools, technology, and integration with external drivers such as SAFNR or data requests o attend working groups o review new software releases o set system direction o upgrade paths o ensure a timely response for application support o ensure continued compliance with new CIP Standards Operations Support - Applications and IT

31 31 IT and Applications increases since 2009: o Number of applications increased from seven to 16 o Number of users increased from 15 to 34 As a cost savings measure, RC tool applications have been developed in-house o Creating a burden: volume of custom applications requiring on-going maintenance have increased o Number of developed applications increased from two to 53 Support staff increased from five to six Operations Support - Applications and IT

32 32 Operations Support - IT Infrastructure

33 33 Operations Support - IT Infrastructure In 2009 average was 40 servers/IT employee Today, it is over 112 servers/IT employee

34 34 WISP − IT Infrastructure WISP is cutting edge technology o WECC staff to develop and maintain custom software for the WISP program o There will be a 40 percent increase in the amount of data storage in SQL databases with the synchrophasor systems o Voltage Stability Analysis will be integrated with 1500 points of synchrophasor data that will be maintained and tuned

35 35 Additional personnel needed in the following in order to: o Identify and address model deficiencies o Make necessary improvements to the model o Ensure proper model validation with TOPs o Model all WECC Region RAS and real-time RAS data points in the WECC RC EMS o Provide maintenance and support for an increasing number of users, applications, and servers o Ensure compliance Operations Support − RCTF Findings

36 Operations Support Application Architects Application Support DTS Support EMS Modeling RAS Engineer Architecture Support Communications IT Infrastructure Systems Administrator FTE Totals WECC RC Function and Recommended FTE Increase Current FTE – Filled Current FTE Approved – Unfilled Future FTE Projection

37 37 WECC RC Compliance staffing has experienced high turnover in the last three years As of 11/26/12 only one staff position is dedicated to Compliance o Compliance monitoring and documentation undertaken by upper-level RC management resulting in additional work often outside normal duty hours Compliance

38 38 The WECC RC increased staffing will improve the ability to: o Meet the March 2012 WECC Board Resolution that established clear compliance expectations for WECC o Meet increased compliance and reliability responsibilities o Meet increased regulator expectations o Mitigate increasing regulatory and financial risks Compliance - RCTF Findings

39 Compliance Document and Evidence Management Audit Preparation Event Analysis Standards Development Compliance Reporting Disturbance Reporting 1 11 FTE Totals Current FTE – Filled Current FTE Approved – Unfilled Future FTE Projection WECC RC Function and Recommended FTE Increase

40 40 Reliability Coordination Trainers o Training responsibilities are identified in NERC Reliability Standards DTS Engineers o Support increase in system restoration drills, exercises, and simulations o Assist in developing implementation of new TOP and region-specific restoration training scenarios Training

41 41 NOTE: Includes projected hours for remainder of 2012 Internal -Training Hours

42 42 Additional personnel recommended to: o Provide required training to RC staff o Provide required training to regional reliability entities o Ensure training documentation is maintained as a provider o Ensure compliance with NERC Reliability Standards Training - RCTF Findings

43 Training Develop/Deliver in-house training to new RC trainees and current RC staff to ensure compliance with mandatory standards and retention of NERC RC System Operator certification Conduct just-in-time training for new tools, applications, procedures, and processes Update job task analysis Develop/deliver NERC certified restoration training to BAs, TOPs, and GOs FTE Totals WECC RC Function and Recommended FTE Increase Current FTE – Filled Current FTE Approved – Unfilled Future FTE Projection

44 44 Conclusions WECC has a high level of responsibility and is at risk for reliability and compliance issues o Staffing is a major factor o Tools are a major factor Emergent tools and Standards will add to these risks o WISP related displays, data, training o PER-005 and EOP-005

45 45 Increase monitoring areas from two to three Add 47 WECC RC FTEs to enhance and improve the capability of the RC as follows: o FTEs o FTEs o FTEs Acquire additional tools as identified in the projected budget RCTF Recommendations

46 Current FTE – Filled Current FTE Approved – Unfilled Future FTE Projection Totals Operations Planning Current-day/ real-time Support Next-day Study Data Validation and Mapping Seasonal SOL Coordination and Validation Emergency Plan Coordination 9511 FTE Totals Real-time Operations Monitor and analyze real-time system conditions Coordinate system restoration activities Proactively respond to and mitigate potential adverse impacts to the BES Communicate with and direct corrective actions of registered entities during actual SOL exceedances Identify and mitigate potential/actual IROLs in real- time operations horizon FTE Totals Compliance Document and Evidence Management Audit Preparation Event Analysis Standards Development Compliance Reporting Disturbance Reporting 111 FTE Totals IT Applications, Engineering Support, & IT Support Application Architects Application Support DTS Support EMS Modeling RAS Engineer Architecture Support Communications IT Infrastructure Systems Administrator FTE Totals WISP Broaden deployment of phasor measurement units throughout the Western Interconnection Implement telecommunication network necessary to support a large- scale production synchrophasor system Install synchrophasor applications for Western Interconnection Increase ability to integrate variable generation Enhance power system analysis capability/model validation 029 FTE Totals Training Develop/Deliver in-house training to new RC trainees and current RC staff to ensure compliance with mandatory standards and retention of NERC RC System Operator certification Conduct just-in- time training for new tools, applications, procedures, and processes Update job task analysis Develop/deliver NERC certified restoration training to BAs, TOPs, and GOs 456 FTE Totals Other Direction and Oversight Administrative Support WECC and NERC Committee Work WECCRC.org Development Membership Request Management 504 FTE Totals WECC RC Function and Recommended FTE Increase

47 47 Budget Projections 1 BASE BUDGET Includes WISP, RC tools, approved FTE's and an across the board 3% inflation increase $14,438,927$19,024,411$20,400,136$21,010,320$21,588,266$23,169,891 3 TOTAL DOLLAR INCREASE FROM PREVIOUS YEAR $1,490,803$3,585,484$2,375,725$610,184$577,946$1,581,625 4 PERCENT INCREASE FROM PREVIOUS YEAR 10%19%12%3% 7% 5 FUTURE REQUIREMENTS 6 Approved but unbudgeted FTE salaries, loading and set up cost. Not budgeted, staggered hiring in 2012 but included in 2013 base budget. And, over-runs for Operating Expenses and Fixed Assets.$1,520, unbudgeted requested 2013 FTE salary & loading costs. Staggered hiring in Full cost in 2014 and beyond. Includes 3% inflation increase on salaries. $1,648,480$1,702,759$1,757,038$1,811,317$1,865,596 8 Build out unbudgeted construction cost for 2013 requested 12 FTE. (6 FTE are in real-time control room) $157, requested 2014 FTE salary & loading costs. Staggered hiring in Full cost in 2015 and beyond. Includes 3% inflation increase on salaries. $2,098,286$2,164,627$2,230,968$2,297, Build out construction cost for 2014 requested 14 FTE (8 FTE are in real-time control room) $183, requested 2015 FTE salary & loading cost. Staggered hiring in Full cost in 2016 and beyond. Includes 3% inflation increase on salaries. $854,394$878,518$902, Build out construction cost for 2015 requested 5 FTE (3 FTE are in real time control room) $65, We will need to acquire Sq feet of space in 2013 as it becomes available in LRCC and VRCC so we don't lose the option in the current buildings. This is an added $17,000 per month. Assuming full cost beginning in 2013 $204, Increase each year $1,520,569$2,009,980$4,188,795$5,045,684$5,124,803$5,269, Totals for base & future budgets (line 14 plus line 2) $15,959,496$21,034,391$24,588,931$26,056,004$26,713,069$28,439, PERCENT INCREASE FROM BASE BUDGET 10% 17%19% NOTE DOE expansion of control rooms is $3.5 million dollars, anything outside of the $3.5 million will be a WECC cost. NOTE -- the Enhanced Curtailment Calculator is in 2014 WECC Business Plan but not in the budget because it has not been approved by the WECC OC. NOTE budget is projected to exceed base approved budget by $1,520,569 due to cost overrun in Personnel, Operating Expenses and Fixed Assets. NOTE and 2014 Total Dollar Increase From Previous Year (Line 4) is due to change in WISP funding from DOE to WECC NOTE Total Dollar Increase From Previous Year is due to hardware refresh for WISP support

48 Questions?

49 49 Back Up Slides

50 50 Control Room Facilities Comparison MISO WECC RC Vancouver

51 51 Acronym Guide BA – Balancing Authority BES – Bulk Electric System CIP – Critical Infrastructure Protection DTS – Dispatcher Training Simulator EMS – Energy Management System FTE – Full-time Equivalent IROL – Interconnection Reliability Operating Limit NERC – North American Electric Reliability Coordination RAS – Remedial Action Scheme RC – Reliability Coordinator RTCA – Real-time Contingency Analysis SOL – System Operating Limit SONGS – San Onofre Nuclear Generating Station TOP – Transmission Operator USF – Unscheduled Flow WBRTF – West-wide System Model and Base Case Reconciliation Task Force WISP – Western Interconnection Synchrophasor Project WSM – West-wide System Model

52 52 Real Time Operations Subregional strategic shift from two to three areas: o More RC System Operators monitoring the system o Wide area coordination with specific and targeted increase in focus o Dedicated on-shift engineer in each control room to ensure accuracy of data and provide study support o Improved and more specific regional procedures, operating guides, and corresponding displays

53 53 Operations Support - Applications

54 54 Operations Support - Applications

55 55 Operations Support - Applications

56 Operations Support - RCWorkbook – Monitoring RTCA CONFIDENTIAL Post-contingent branch, interface, and bus voltage limit exceedances presented. Island contingencies as well as unsolved contingencies (potential IROLs) are presented using this WECC RC developed tool.

57 57 External -Training Hours

58 58 Training - External Savings Airfare and car rental are not included in the estimated savings

59 59 Training - Internal Cost Savings Airfare and car rental are not included in the estimated savings

60 60 Real-time Operations Are these numbers by Quarter?

61 61 Real Time Operations

62 Current FTE – Filled Current FTE Approved – Unfilled Totals Operations Planning Current - day/real-time Support Next-day Study Data Validation and Mapping Seasonal SOL Coordination and Validation Emergency Plan Coordination 95 FTE Totals Real-time Operations Monitor and analyze real-time system conditions Coordinate system restoration activities Proactively respond to and mitigate potential adverse impacts to the BES Communicate with and direct corrective actions of registered entities during actual SOL exceedances Identify and mitigate potential/actual IROLs in real- time operations horizon 330 FTE Totals Compliance Document and Evidence Management Audit Preparation Event Analysis Standards Development Compliance Reporting Disturbance Reporting 00 FTE Totals Operations Support Application Architects Application Support DTS Support EMS Modeling RAS Engineer Architecture Support Communications IT Infrastructure Systems Administrator FTE Totals 203 WISP Broaden deployment of phasor measurement units throughout the Western Interconnection Implement telecommunication network necessary to support a large- scale production synchrophasor system Install synchrophasor applications for Western Interconnection Increase ability to integrate variable generation Enhance power system analysis capability/model validation 29 FTE Totals Training Develop/Deliver in-house training to new RC trainees and current RC staff to ensure compliance with mandatory standards and retention of NERC RC System Operator certification Conduct just-in- time training for new tools, applications, procedures, and processes Update job task analysis Develop/deliver NERC certified restoration training to BAs, TOPs, and GOs 56 FTE Totals Other Direction and Oversight Administrative Support WECC and NERC Committee Work WECCRC.org Development Membership Request Management 04 FTE Totals Existing WECC RC Functions and Current Staffing

63 63 Improving Models - Adding Measurements Adding measurements is necessary but translates to more alarms for the RC System Operators. Many measurements are still needed.


Download ppt "Presented by Members: Tom Botello - Southern California Edison Terry Baker - Platte River Power Authority Reliability Coordination Task Force Report to."

Similar presentations


Ads by Google