Presentation on theme: "March 6, 2008 TAC Texas Nodal Market Implementation Program Update Jerry Sullivan."— Presentation transcript:
March 6, 2008 TAC Texas Nodal Market Implementation Program Update Jerry Sullivan
2 2 Texas Nodal Program UpdateMarch 6, 2008 AGENDA Program Status Key Risks IBM Review given to BoD in February Quality Results Schedule and the Program Level “Health Checks” Cost The area of emphasis at today’s TAC Meeting
3 3 Texas Nodal Program UpdateMarch 6, 2008 Program Status: Nodal Delivery Nodal Delivery status remains amber; key checkpoints approaching Cost ScheduleScope / Quality Revised budget approved by ERCOT BOD, however we will be RED until fee filing approved. Revised Fee Filing with the PUC this week. Need to decide on methodology for grayboxing and/or time-boxing the handling of: current protocols,or future NPRRs. Requirements for DAM in MMS are being revised. Key checkpoints in place to ensure ongoing viability of 12/1/08. Planning for the “168 Hour Test,” which is under 200 days away. EDS 2 is Red, but soon may be amber. EDS 4 is Red, will go amber when EMS 4/5 completed. Number of defects discovered in EDS were high, being resolved. Applying increased efforts to identify and resolve defects in pre- FAT and FAT GreenAmber RedAmber Nodal Delivery Legend Red Amber Green Estimate at Complete = <$263m Estimate at Complete = < $263m + 10% Estimate at Complete = >$263m + 10% Go-live = 12/1/08 Go-live = <30 days+ Go-live = >30 days+ Program aligned with current protocols Program aligned to previous protocol version Program not aligned to protocols Current product quality is not sufficient for Nodal launch Most Nodal products currently achieve quality standards All Nodal products currently achieve quality standards
4 4 Texas Nodal Program UpdateMarch 6, 2008 Program Status: ERCOT Readiness (3/15/2008) ERCOT Readiness status remains amber, but a renewed awareness of the need for preparation is apparent Organization, Processes and Procedures, and Training show progress –Organization Completed review of Deep Dive presentations including staffing/ sizing department models for Post-Nodal implementation (2009). Draft Vision, Mission, Core Values and Strategic Plan completed. Transition sponsors are assigned. January Executive Readiness Survey overall indicates Amber. Key Focus Areas to work on: adequate staffing, procedures, transition planning. Transition plans being documented for all key departments. –Processes and Procedures Business Process Model completed to procedure level and ready to leverage for testing and validation during EDS and training. Increased focus on the creation of procedures. –Training Training tracking and tailoring at the department/individual level. Validation of current individual training plans Concerns Remain –Adequate staffing and resources for transition from project teams to employees –Procedures still in flux until delivery of systems –Completion of training in time for EDS Priority for ERCOT Staff is to ensure readiness for the “168 hour test”
5 5 Texas Nodal Program UpdateMarch 6, 2008 Program Status: Market Participants ( 2/28/2008) Registration and Qualification [green] - Resource Asset Registration quality, as measured by metric MP11, has improved. Two QSEs represent resources that are not current with submittals for resource registration: Barton Chapel Wind, Austin Energy Engagement continues to be main area of concern. Engagement remains [amber] for all segments except QSEs with Resources. Continuing to work with MPs to improve their engagement scores. 13 QSEs [red ] - due to Engagement. The following remain red in the scorecard: Citadel, Clearview, Duke Energy Ohio, Econnergy, EnergyCo, JPMorgan Ventures Energy, Juice Energy, Kansas City Power and Light, Liberty Power, LPT, Mirant, Texpo, WCW International Next survey to be released March 17. Negative responses will be treated as red in the next round. Training [green] – First pass of self reported training is green for all segments except LSEs Connectivity and Telemetry [green] – Connectivity metrics complete for most MPs. Outliers are new to the program and need to go through EDS process Overall status* returns to amber * Status current as of 2.28.08
6 6 Texas Nodal Program UpdateMarch 6, 2008 Market Readiness Required by: ERCOT Nodal Transition Plan (5.4.4 (3)) Purpose: Provide an opportunity for market participants to measure and improve the accuracy and stability of the Real Time Sequence (RTS). Accuracy: LMPs are reasonable given the input data, but the input data needs improvement. These are not “settlement grade” LMPs. ERCOT is limited in its capacity to handle all MP questions based on the quality and complexity of the data. Use: Should be used for trends and to assist the EDS team to improve the accuracy of the input data, but not for financial analysis. Actions: ERCOT is continuing to resolve a number of problems known to impact the SCED results: Working with QSE’s to resolve a number of SCED data input issues; similar exercise is in progress with TSP’s to improve the quality of the tele-metered data feeding the State Estimator. Six month posting of LMPs initiated on February 15
7 7 Texas Nodal Program UpdateMarch 6, 2008 Key Risks as 5 March 2008 These are the key, key risk M H M M Likelihood H M H M Impact on Go Live ERCOT working with vendor to rectify defects/outstanding. issues Single entry model must be in place by July to verify go-live model Single entry model could be late NMMS Prioritizing build out of environments and applications based on EDS schedule Planning active / standby EDS environment (Austin / Taylor) to eliminate short-term ITest need Building test environment in Blue Building server room Removing all non-essential equipment from Austin data center Compressing Nodal deployment footprint as much as possible Working with 3 main vendors on multiple iterations of development and validation of CIM import/export Investigating contingency plan to preserve April 1st start of DAM / RUC EDS trials Building interfaces in advance of applications delivery based on available information expecting re-factoring Testing application and integration in available, sub-optimal hardware / software environments Mitigation Limits speed and scale of environment build outs, impacts development projects, integration and EDS trials Infrastructure constraints (power, space, cooling) Uncertainty over ability to integrate CIM models may impact data and system quality and testing CIM (Common Information Model) Lack of source systems and infrastructure limits ability to validate integration points, impacts re- factoring cycle times and costs EIP (Enterprise Integration Project) DescriptionRisk
8 8 Texas Nodal Program UpdateMarch 6, 2008 IBM Review - Dec 07 – Jan 08 - Some issues involve quality processes IssueNodal Team Comments Resource Planning and Deployment. Attrition. More engagement needed from ERCOT. Effort / Engagement will increase. Training necessarily follows delivery of software. Few key losses since October. We are relying on metrics and ownership which will drive behaviors. Risk Management. Minor risks numerous, but duplicative by project. Few High program risks. Relatively known risks. Risk log is comprehensive and quite exhaustive; Focus now is on the 5 or 6 top key risks rather than individual project perspectives. Quality Reviews. Documentation lagging due to priority given to delivery. Protocols covered. Priority given to fixing defects and trials. Documentation of traceability is lagging behind delivery, we acknowledge documentation needs improvement Timeline. Lagging measures in quality, need more Unit Testing visibility (before Pre FAT visibility. High visibility given instead to Pre FAT at vendor site, to FAT at ERCOT sites, and to EDS. We view “Unit Testing” to be less reliable, and/or reporting delayed by most vendors. PMO resources low (due to shifting resources to more critical projects) Moved experienced personnel to other areas to fill needs; program leadership looks to optimize staffing in right places. Now adding staff to PMO and to Delivery Assurance Group (DAG)
9 9 Texas Nodal Program UpdateMarch 6, 2008 Quality - Summary Successes –NMMS has fixed a number of high severity defects (2 fixed of 3) –CRR has reduced the overall number of defects; currently only 3 Severity 2 defects exist (No severity 1 defects currently) Areas of Concern –Defect resolution time –High percentage of EMS defects not detected till EDS Mitigation Measures –Passive data center site with live data feed will allow for earlier defect detection Defect Classification Quality has experienced recent successes but concerns remain Severity 1 – Data Loss/Critical Error Severity 2 – Loss of Functionality w/o workaround Severity 3 – Loss of functionality with workaround Severity 4 – Partial Loss of a feature set Severity 5 – Cosmetic/Documentation Error Severity 6 - Enhancement
10 Texas Nodal Program UpdateMarch 6, 2008 Quality details: Nodal Test Results per Project Since December 2007, the Program undertook a significant documentation and tracking process. Significant visibility into quality status of each project: December 2007: 5 projects tracked March 2008: 14 projects tracked (others are not in the testing phase) Failure rates within acceptable limits, especially with Severity* 1 and 2 defects. * Note: Severity 1 and 2 defects are the most significant of the 4 types. December 17, 2007 March 4, 2008
11 Texas Nodal Program UpdateMarch 6, 2008 Quality Details: Nodal FAT Active Defects by Severity by Project (Note - Does not include Closed and Deferred) Few severity 1 or 2 defects in the Market Management System (MMS) No severity 1 defects in Energy Management System (EMS) Commercial System Registration (COMS) is still early in development. In Pre FAT The Network Model Management System (NMMS) has a few Sev 1 defects Overall, active defects are being tracked by project and by classification. The relative low number of Sev 1 and 2 defects is somewhat encouraging.
12 Texas Nodal Program UpdateMarch 6, 2008 Quality Details: Nodal Active Defects by Severity Trend Sev 3 Sev 4 Sev 1 Sev 2 Severity 3 defect trend is rising due to focus on fixing higher priority severity 1 and 2 defects Severity 2 trend is remaining steady which indicates “defect open rate” is only slightly higher than defect closure rate
13 Texas Nodal Program UpdateMarch 6, 2008 Quality Details : Average Days to Fix Defect by Vendor The larger project vendors are doing fairly well in their turn around time for defect fixing. There are known and understood reasons for the other vendors. What is not shown here is the turnaround by defect classification. Overall, the time to fix defect in various categories gives us a good indication on: how long the product will be in testing what the next release will look like how long will it take to get to market trials. a good indication of ability to meet the 168 hour test.
14 Texas Nodal Program UpdateMarch 6, 2008 Quality Details: Nodal Defects Reopened % (After Fixing by Vendor) Indicates what percentage of fixes delivered by a vendor retested unsuccessfully. ABB and Areva are fixing their products with a good low reopen rate. Good indication that the Energy and the Market Management System are conducting good quality reviews at the vendor site prior to releases being given back to ERCOT and the market. This metric is being tracked to determine trends.
15 Texas Nodal Program UpdateMarch 6, 2008 Quality Details: Defects found in the Early Delivery System Severity 1 – Data Loss/Critical Error Severity 2 – Loss of Functionality w/o workaround Severity 3 – Loss of functionality with workaround Severity 4 – Partial Loss of a feature set Severity 5 – Cosmetic/Documentation Error Severity 6 - Enhancement We released EMS 2 through 3 too early, resulting in defects showing up in the Early Delivery System No severity 1 defects in Energy Management System (EMS) Commercial System Registration (COMS) is still early in development. In Pre FAT The Network Model Management System (NMMS) has a few Sev 1 defects Overall, active defects are being tracked by project and by classification. The relative low number of Sev 1 and 2 defects is somewhat encouraging. Severity 1 Severity 2 Severity 3 Severity 4 Severity 5 Severity 6
16 Texas Nodal Program UpdateMarch 6, 2008 Schedule: “Health Checks” Still targeting a 12/1/08 Go-Live date; critical path runs through EDS activities JanFebMarAprMayJunJulAugSepOctNovDec EDS 1 EDS 2 EDS 3 EDS 4 Today MP Trials & NM Verification SCED & LMP Posting for 6 months ICCP / P2P SE Tuning E1 R2 E2 R3 E2 R4 E3 R5 E3 R6 LFC Testing E3 R7 CRR Testing E4 R9 CMM, DAM & RT Settlements, Verifiable Costs & Disputes DAM, SASM & RUC execution, DAM Stability Test, EECP, DST, Compliance E4 R8 Performance Health Checks – measure of schedule viability Dictated by PRR 727: Market Readiness Certification 168 Hr TestEDS 4Go Live Sequence CIM Integration Defect Go-Live (Go/No-Go) SeptOctNov TAC Meeting: 10/2 ERCOT BoD: 10/21 First Market Notice: 10/31Second Market Notice: 11/21168 Hr Test Start 32 Days to complete Test 3 2 1 Business Processes
17 Texas Nodal Program UpdateMarch 6, 2008 Schedule: This is the metric status for each of the Early Delivery Systems EDS activities are monitored and measured by 59 metrics (xx) – Number of metrics associated with a particular Release EDS1 – Complete EDS2 – Overall Red resulting from ongoing issues with State Estimator EDS3 – Yellow resulting from LMP performance and delays in LFC / ACE testing EDS4 – Mostly unmeasured at this point, but inheriting Red status from recently moved Alarms Processing metric. Commercial Ops metrics being measured are Green Overall Readiness – Yellow due to open issues on telemetry, modeling, and CIM JanFebMarAprMayJunJulAugSepOctNovDec EDS 1 EDS 2 EDS 3 EDS 4 Today MP Trials & NM Verification SCED & LMP Posting for 6 months ICCP / P2P SE Tuning E1 R2 E2 R3 E2 R4 E3 R5 E3 R6 LFC Testing E3 R7 CRR Testing E4 R9 CMM, DAM & RT Settlements, Verifiable Costs & Disputes DAM, SASM & RUC execution, DAM Stability Test, EECP, DST, Compliance E4 R8 168 Hr TestGo Live Sequence (1 EDS, 2 MP) (3 EDS) (5 EDS, 1 MP) (2 EDS, 1 MP) (3 MP) (2 EDS, 2 MP) (12 EDS, 2 MP) (5 EDS, 3 MP) Overall Readiness: (15 metrics)
18 Texas Nodal Program UpdateMarch 6, 2008 Outage Scheduler - Need for periodic duel entry of outages ERCOT will develop scripts to alleviate duel entry during most periods: –Between minor tests –Between the 168 hr. test and go-live There will be times, especially during the 168 hour test where zonal and nodal systems will require MPs to enter data into two systems Solid Bars below indicate duel entry requirement, estimated to be a total of 45 days December November October September August Dual Entry to Support 168 Hour Test Dual Entry just prior to Go – Live. Minor tests * (hashed bar if 168 hour test extends to the full month
19 Texas Nodal Program UpdateMarch 6, 2008 Cost: Element Summary EAC Revised Budget and Forecast remain constant; spending in January expected to be lower and should reverse much of the overrun through Dec. January actuals are still unavailable as of 5 March