Parallel Ops / Go-Live Checkpoint Weekly Update – Friday Meeting January 31, 2014

Agenda IDT Debrief 9:00A – 9:20A IDT Schedule and Postings IDT Report Review: IDT 1/21 and IDT 1/27 Analysis and Announcements Updates 9:20A – 9:40A Announcements MP Behavior / Strategy during Parallel Ops Scenario Tracker and Scenario Results Look-Ahead 9:40A – 10:00A February Schedule Weekly Updates: Release Scorecard, Migration Report, Known Issues Log Commitment Notifications Presentation 10:00A – 10:15A Settlements Update 10:15A – 10:30A Readiness Update 10:30A – 11:30A

Remaining IDTs IDT Date IDT Times Conditions Notes 2/25/14 8:30A-4:30P 6hr IDT begins at 9:30A 2/28/14 7:00P 4hr IDT begins at 8:00P Cutover for March 1 Go-Live Ginny Watson

IDT Debrief slides on spp.org here IDT 01/21 Report IDT reports posted to spp.org here 7 calendar days after each IDT spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test IDT 01/27 Report IDT report posted early to spp.org here IDT Debrief slides on spp.org here spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test For your reference: 01/27/14 IDT Preliminary Debrief 01/21/14 Reserve Selection IDT 01/21/14 IDT Preliminary Debrief Brian Strickland

Announcements Please continue to submit dual tags through February Even though these days will not be “Bid to Bill” days, MPs should continue entering dual tags in February Thank you for your participation! Shari Brown

Announcements RMS Change As Parallel Ops ends 1/31, there will no longer be a Parallel Ops QuickPick. Continue to submit routine questions to the default “General Inquiry” For specific “Go Live” questions or February IDT questions, choose the new “Go Live” QuickPick Ginny Watson

Announcements Extended Maintenance Window Wed Feb 5 – Fri Feb 21: 24/7 availability cannot be guaranteed Current functionality and availability will be maintained until Feb 5 Complete any extensive/critical testing prior to the start of this maintenance period Nothing will be settled during the maintenance period Systems will be fully available from 8A-9A CPT daily during the maintenance window Intermittent maintenance will occur outside of 8A-9A CPT System outages, errors, and performance issues may occur Notifications will only be sent for outages that impact systems for more than 24 hours or during the 9A-9A CPT availability window Markets Systems bids/offers can be submitted during the 8A – 9A CPT window On Feb 22, MPs may begin submitting Bilateral Settlements Schedules and bids/offers for OD March 1 Settling will begin OD March 1 with Meter Agent submittals beginning March 2 and initial postings on March 10 Ginny Watson

Market Participant Parallel Ops Behavior Concerns Poor Meter Data Quality: Several of the Market Participants’ Meter Agents are submitting very low quality Meter Data. Inconsistently Poor – it is hard to predict which specific MPs will submit poor data on any given day. There are few consistent poor performers. There are A LOT OF GOOD performers. Don’t believe this is due to lack of understanding or ability. Likely due to Parallel Ops ‘burn out’. This is likely a Parallel Ops issue that will go away when the market goes live. SPP is reaching out to MPs that appear to have consistently high Meter Data Calibration and assure they are capable to improving the quality of their submissions. Low MP Submission of Interchange Schedules: Many (Most?) Market Participants are not ‘dual’ submitting their SPP Interchange Schedules. This affects MPs individually and globally: Individual MPs will not observe their true settlement Revenue/Costs due to the Non- Settlement of the missing Interchange Schedules. Globally the missing Interchange Schedules create an artificially LARGE Inadvertent Energy Settlement. SPP is reaching out to MPs that have lowest Interchange Schedule participation to assure they are capable and comfortable with the submission process. Casey Cathey

Market Participant Parallel Ops Behavior Concerns Day Ahead Load Bids un-aligned with Load Forecasts: Some MP’s appear to be over scheduling (over bidding) their Load in the DA Market. The difference between their cleared DA Load the SPP Load forecast is very large – as high as 1,000 MWs per hour. These delta MWs impact the DA solution and DA LMPs. They indirectly result in RT Asset Settlement being poor indicator of expected Market Results. SPP will reach out to the MPs with large DA Schedule vs Load Forecast deltas to assure they are aware of the differences and have the ability to schedule more ‘accurately’. Some MPs showing consistent Revenue Losses: There are a few MPs that are consistently showing net losses. SPP recognizes that most MPs are not attempting optimal operations at this time. This data is accurate, but SPP is cautious using it as a stand alone indicator of an MPs ability to be profitable/successful in SPPs Market. SPP will reach out to the MPs with the appearance of poor market results to assure they are aware of their overall results and have a good understanding of the activities required at go-live to achieve optimal outcomes. Casey Cathey

DA Shortage & Excess Capacity Scenarios SPP is currently awaiting some more fixes for existing defects in the Day-Ahead emergency logic. Many of the defects were taken care of in the R1.9.3 patch, but a few more remain to be delivered Fixes are due in a patch with R1.9.3 SPP will retest Scenarios in lower environments as soon as available Detailed results will be presented to MPs. Jarrett Friddle

Scenario Tracker as of Jan 31 Summary – Scenarios in Parallel Ops: 11 Scenarios, 2 Partial Scenarios, Dispatch Simulator Test successful: Scenario 7 Must Offer Scenario 28a Scenario 15 Commitment Mismatch Part 2 Scenario 24 Scenario 13 DDR w/ Settlements Scenario 29 RSG Scenario 31 BDR w/ Settlements Scenario 36 GFA Scenario 11 EDR Mitigation Scenarios 16.1, 16.2, 16.3 Non-Functional Test Scenario Dispatch Simulator Test Note: Dispatch Simulator Test occurred 12/6 – 12/9 and again 1/15 – 1/17. Summary – Retests Pending: 2 individual retest requests by 1 MP are scheduled for Jan 31 & Feb 4 Shortage & Excess Capacity Scenarios still being retested in lower environments Ginny Watson

Scenario 31 BDR w/ Settlements Results On the Markets side, this Scenario passed Market Participant data was correct. Resource behaved as designed in RTBM and EMS. Dispatch instructions were observed via XML and Markets UI. BDR Settlements Statements were posted on Jan 28 Jarrett Friddle

Non-Functional Testing Results Successfully tested January 23rd Result: EMS failed over correctly; functioned as expected Result: ICCP failed over correctly, functioned as expected Result: Marketplace Portal and API failed over correctly. Users may have experienced brief intermittent connectivity when SPP’s primary facility was taken offline but no issues were reported. Result: Web Services (RTO_SS) Remained available during the exercise, as expected Result: No RTBM studies were missed during the exercise and ID-RUC studies completed successfully Result: Email was unavailable for approximately 16 minutes, as expected. After the exercise, failover back to SPP’s primary facility was successful although some monitoring applications had some small glitches that were identified and fixed within the hour. CJ Brown

Scenario 11 EDR Results SPP successfully executed the EDR scenario during the Jan 27 IDT From an RTBM perspective, the scenario was a success. SPP sent the expected dispatch via Markets UI, XML, and ICCP. MP requested to retest their resolved issues on Jan 31 Jarrett Friddle

Mitigation Scenarios Summary Commitment 348 Resources submitted at least one hour in ‘Market’ commit status 264 Resources committed (‘Market’ and ‘Self’) 37 Resources with ‘Market’ commit status failed the Start-Up Offer Conduct Test 19 of these Resources were committed 45 Resources with ‘Market’ commit status failed the No-Load Offer conduct test 18 of these Resources were committed Dispatch 41 committed Resources failed the Energy Offer Conduct Test and were mitigated for at least one hour 169 Resources cleared for Operating Reserve Product MWs 20 of these Resources failed the Operating Reserve conduct test and were mitigated for at least one hour Jarrett Friddle

16.1 DA Market Mitigation The Day Ahead Market for OD 01/29 was completed with Mitigation enforced. 5 MPs submitted specific resources to test Mitigation with Scenario 16.1 3 of the 5 were mitigated The other 2 submitted mitigation and DAMKT offer curves the same and did not fail the behavior test. There were 62 resources ultimately mitigated in the DA Market Commitment including those explicitly identified in RMS tickets. Market Monitoring has reviewed cases and agrees that the Mitigation is working appropriately. In order to obtain mitigation examples in the entire footprint, Reserve Zone minimum requirements were set to a level to ensure each Reserve Zone was binding. In order to do that, all Reserve Zone minimum limits were raised to an unattainable level. This did effectively mitigate all the resources we had hoped but it also resulted in the Demand Curves for the Upward reserve Products being enforced and thus, all of the UP AS Product MCPs were $1100 for all hours. Jarrett Friddle

16.2 DA RUC Mitigation The Day Ahead RUC for OD 01/29 was completed with Mitigation enforced. 6 MPs submitted specific resources to test Mitigation with Scenario 16.2 All 6 were successfully mitigated and committed through the DA RUC process. All entities should have been able to see the appropriate Mitigation indicators in the Market UI and API messages. There were several additional resources ultimately mitigated besides those explicitly identified in RMS tickets. 14 Resource were mitigated for Commitment. Jarrett Friddle

Scenario 16.3 RTBM Mitigation Results Scenario results from 1/28 and 1/28 A total of 104 Resources experienced energy mitigation in at least 1 RTBM interval MPs submitted 8 RMS tickets identifying Resources that were to participate in this Scenario All 8 Resources experienced mitigation over the 48 hour test period During business hours, mitigation tuning parameters were tightened in an attempt to force more Resources to mitigate Mitigation did occur around the clock, with the protocol parameters Jarrett Friddle

Look-Ahead: Jan 19 – Jan 31 Parallel Ops Calendar on spp.org spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents Settlements details on the Settlements Calendar marketplace.spp.org > Public > Settlements > Calendars Feb 1 – Mar 1 Schedule: Wed Feb 5: SPP Maintenance Window Begins. Fri Feb 7: Weekly Go-Live Checkpoint - Register on spp.org. Same Dial In and WebEx Wed Feb 12: CWG Thurs Feb 13: CWG Fri Feb 14: No Regular Friday Meeting Mon Feb 17: SPP Holiday Fri Feb 21: Weekly Go-Live Checkpoint Mon Feb 24: IDT Prep Call 3P Tues Feb 25: IDT Call 8:30A – 4:30P, 6 hour IDT begins at 9:30A Wed Feb 26: IDT Debrief Call 3P Thurs Feb 27: IDT Prep Call 3P Fri Feb 28: IDT Call 7P, Go-Live cutover IDT begins at 8P Ginny Watson

Migration and Maintenance Report Migration and Maintenance Report posted to spp.org spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents Note: Discontinued after 1/31 Ginny Watson

Release Scorecard Release Scorecard posted to spp.org: spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents Note: Discontinued after 1/31 Ginny Watson

Known Issues Log Please review the Market Trials Known Issues Log on spp.org spp.org > Integrated Marketplace > Market Trials > Structured/Unstructured Test > Integrated Marketplace Market Trials Project Documentation Folder Note: Will continue after Parallel Ops ends Issue # Component Synopsis Severity 903 Markets Systems Gen Resource with scadastatus = NOTFOLLOWDISPATCH getting CMODE = 1; the CMODE should be = 3 Severity 2: High 904 1st interval, the cmode is 3 (and for DVERs follow dispatch flag is false). Under these conditions, MCE does not calculate an achievable MW. 2nd interval, the cmode switches to 1 or 2 and/or the follow dispatch flag is true. Since MCE does not have an achievable MW, it uses the initial MW and applies the ramp rate over 10 minutes to calculate the dispatch. For the first interval a resource is in a control mode of 1 or 2 and/or follow dispatch flag is true, MCE should use the initial MW and apply the ramp rate over 5 minutes for the dispatch MW. . Severity 3: Medium Jodi Woods

Go-Live Useful Resources Readiness Center Integrated Deployment Test spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test IDT MP Guide Parallel Ops Calendar Additional documents: Settlements Calendar marketplace.spp.org > Public > Settlements > Calendars Change Tracker spp.org > Integrated Marketplace > Marketplace Documents

Commitment Notifications General Commitment Timelines: (When does this happen?) Day-Ahead RUC notifications issued around 20:00 hrs. It could be earlier or later If it is late, there should be a Market Message indicating the DA RUC will be late for the day Intra-Day RUC/Real-Time notifications can happen at any time Notification: (How do I navigate to see it?) Via an XML notification (PostEnergyCommitmentSet) Via Markets UI Markets User Interface Guide for more information Jim Gonzalez

Commitment Notifications by Asset Owner Query Resource Commitment Schedules by Asset Owner Step Action Comments 1 Access the Marketplace Portal The Portal Address is: https://marketplace.spp.org 2 Select the Resource Offers tab. 3 In the Navigation tree, select the appropriate Asset Owner. Navigation tree is located on the left-hand side of the Markets UI. 4 Select a date. 5 Select Market Results. 6 Choose Commitments, and select View. Jim Gonzalez

Commitment Notifications by Asset Owner Query Historical Resource Commitment Schedules by Asset Owner Step Action Comments 1 Access the Marketplace Portal The Portal Address is: https://marketplace.spp.org 2 Select the Resource Offers tab. 3 In the Navigation tree, select the appropriate Asset Owner. Navigation tree is located on the left-hand side of the Markets UI. 4 Select a date. 5 Select Market Results. 6 Choose Commitment History, and select View. Jim Gonzalez

Commitment Notifications by Resource Query Resource Commitment Schedules by Resource Step Action Comments 1 Access the Marketplace Portal The Portal Address is: https://marketplace.spp.org 2 Select the Resource Offers tab. 3 In the Navigation tree, select the appropriate Asset Owner. Navigation tree is located on the left-hand side of the Markets UI. 4 Select a date. 5 Select Resources. 6 Select a resource. 7 Choose Commitments, and select View. Jim Gonzalez

Commitment Notifications by Resource Query Historical Resource Commitment Schedules by Resource Step Action Comments 1 Access the Marketplace Portal The Portal Address is: https://marketplace.spp.org 2 Select the Resource Offers tab. 3 In the Navigation tree, select the appropriate Asset Owner. Navigation tree is located on the left-hand side of the Markets UI. 4 Select a date. 5 Select Resources. 6 Select a resource. 7 Choose Commitment History, and select View. Jim Gonzalez

Commitment Notifications via API Notification comes via XML Executing the following commands to access commitment schedules: GetEnergyCommitmentSet GetEnergyCommitmentSetByDay GetEnergyCommitmentHistorySet GetEnergyCommitmentHistorySetByDay Follow Up Item: Do these commands differentiate between asset owner and resource? Or are those separate commands? Jim Gonzalez

Commitment Notifications via API XML Listener Server can be established to notify MPs of any commitment notifications. Execute the command below: PostEnergyCommitmentSet notification Jim Gonzalez

Commitment Notifications Follow Up SPP is not planning on following up with RUC Commitment notifications via phone call ID RUC Process Commitments (both as a result of ID RUC Study Suggestions and as a result of RUC Operator Manual Commitment) can occur at any time after the conclusion of the DA RUC Process All Resource Commitment Notifications (MUI and API) are consistent in format Jim Gonzalez

Brent Wilcox Settlements Update Brent Wilcox

Settlements Update - February Last statements (1/31 I) will be posted Feb 3rd. Please see highlighted section of February calendar for details Statements will remain on portal until noon on Feb 5th Settlement system cutover begins the afternoon of Feb 5th. Due to the extended maintenance period any meter or bilateral settlement schedules submitted between Feb 5th and Feb 21st may go to ‘Pending’ or error out. This is to be expected. Brent Wilcox

Settlements Update – February Cont’d Market Participants can start to submit their Bilateral Settlement schedules on Feb 22nd. Any Bilateral Settlement schedules submitted prior to February 22nd will be deleted as a part of the system cutover Any Bilateral Settlement schedules submitted February 22nd will be live production data. Meter windows will open on OD+1 (3/2/14).  Any meter data submitted prior to March 2nd will be deleted to ensure test data is not mixed with live data. Brent Wilcox

Settlements Outreach – Announcement SPP Settlements will be offering two informational sessions next month. The topics to be covered are: Metering Bilateral Settlement Schedules Transition from EIS to Integrated Marketplace and the overlap between the two  Q/A The sessions are tentatively scheduled for Feb 11 and Feb 18 from 2-3CST. Please look for a formal announcement in the next few days and sign up for one of the sessions if interested. Brent Wilcox

Erin Jester READINESS Update Erin Jester

Update Market Offer Data Markets Test Data vs. Production Level Data Market Offer data will NOT be purged from the Markets Systems prior to 3/1 Includes existing offer data submitted since June 2013 Market Trials start -- even MP offer data that was seeded by SPP for testing purposes Market Participant test offer data MAY become financially binding on 3/1 and beyond if preventative measures aren’t taken before 2/22 Erin Jester

Update Market Offer Data MP Required Action for Updating Market Offer Data Complete Acknowledgement Letter by 2/14 Market Trials Liaisons received RMS ticket on 1/17 requesting acknowledgement of responsibility to update Market offer data and understanding of risks associated with having non-production ready offer data in the Market Systems after 2/22 Acknowledgement to be completed via RMS no later than 2/14 Escalation will occur for MPs not completing acknowledgement by 2/14 Update all Market Offer Data in the Markets System by 2/22 To avoid test data becoming financially binding, update offer and parameter data for DA Resource Offers, RTBM Resource Offers and Mitigated Offers Market offer data MUST be updated prior to 2/22 to ensure good, production-ready data for Go-Live Erin Jester

Production Credit Management System Market Participants can begin submitting Offers/Bids, including Virtual Offers/Bids, for a 3/1 Operating Day on 2/22. The MPs can also continue to submit for Operating Days prior to 3/1 Beginning 2/22, the production Credit Management System will interface with the production Market system and any virtual transaction submitted will be validated by the production credit system. The MPs credit limit will be impacted by these transactions. Beginning 2/22, virtual transactions should not be submitted for an Operating Day prior to 3/1. If a virtual transaction is submitted, the MP will be asked to cancel the transaction. If the transaction is not cancelled, the MPs credit limit will reflect the existing transaction. Jodi Woods

IDT URD Report Beginning in January, SPP sends a report to each MP that failed a URD test in at least one applicable interval during the previous IDT The market trials liaison for the applicable MPs received an email with the subject: SPP Action Requested: Review Resource Response during Jan XX IDT The following slides illustrate aggregate-level URD trends over the Dec/Jan IDTs Casey Cathey

IDT URD Report Top 3 MPs with failed URDs as a percentage of total SETPOINT-SCADA difference by MP Casey Cathey

IDT URD Report Top 3 MPs with failed URDs as a percentage of total SETPOINT-SCADA difference by MP Casey Cathey

IDT URD Report Top 3 MPs with failed URDs as a percentage of total SETPOINT-SCADA difference by MP Casey Cathey

IDT URD Report Percentage of intervals to pass URD tests, and average SETPOINT-SCADA difference of URD failures Casey Cathey

Readiness Metric Update Updated Readiness metrics status will be posted on iDashboard by EOD today (1/31) Readiness metrics will be continued to be tracked up to Go-Live and some will continue after Detailed plans for Feb/Mar will be presented at the Feb CWG meeting Metrics measuring MPs (i.e. CROW, ICCP) will be tracked during February with the understanding that it is during the maintenance window Some of these metrics will not be ‘green’ prior to Go-Live but are not considered an issue for Go-Live Potential financial impacts were discussed at the January CWG Erin Jester

CROW Metric Update (OPS-15) Metric ID Metric Name Status OPS-15 CROW Outage Submission Red 1/19-1/25 12/15-12/21 43% 4% 53% Percentage out of 100 OPS-15 is still reported as ‘red’ (through 1/25), but SPP has seen a very positive trend towards timely submissions in CROW over the past two weeks. Erin Jester

ICCP Metric Update (OPS-18) Metric ID Metric Name Status OPS-18 ICCP Control Status Tracking Red 11/18-11/22 1/18-1/25 Average percentage of control modes submitted correctly Percentage of MPs with 100% of control modes submitted correctly Erin Jester

Go-Live Assessment Update Updates on criteria status and critical outstanding items are in the following slides Minor Changes to report: Added formal CWG recommendation Updated test data for GL-04 (SAT completion), GL-06 (FIT completion), and GL-08 (NFT completion) Updated Markets and Settlements defects in GL-09 (Critical defects) and GL-10 (High defects) Moved GL-20 (Parallel Ops exit criteria) from ‘at risk’ to ‘not met’ Erin Jester

Go-Live Criteria Update 21 Go-Live Criteria Total 7 complete 11 tracking on time GL-18 (FERC approval order) moved from ‘at risk’ 1 at risk 2 not met GL-20 (Parallel Ops Criteria met) moved from ‘at risk’ to ‘not met’) Latest status of criteria will be posted in the Go/No- Go Folder on SPP.org by EOD 1/31 Erin Jester

Go-Live Assessment Update GL-20: Completion of approved open Parallel Ops Exit Criteria Parallel Ops Exit Criteria that were not met by 1/31: Updates on the status of these criteria will be provided during the weekly CWG Checkpoints Erin Jester

Go-Live Assessment Update Outstanding items that need to be resolved prior to Go-Live: Critical Markets functionality scheduled for Markets Operations System (MOS) Release 1.9.3 Market Control and Market Database (MDB) performance improvements Mitigation and Market Clearing Engine (MCE) Updates Frequently Congested Areas (FCA) and External Dynamic Resource (EDR) logic Markets UI/API defects addressing MP requests MCE certification NERC certification of the SPP BA (no change in status) FERC approval Erin Jester

Markets System Release Update MOS System Release Plan 1/21 – Delivery of MCE and RRC Patches from Alstom 1/23 – Delivery of MCE and MDB Patches from Alstom 1/28 – Delivery of MCE, MDB and MUI Patches from Alstom 1/29 – Delivery of MDB Patch from Alstom 1/30 – Delivery of MCE, MDB, and External Interface Patches from Alstom 2/3 – Promotion of R1.9.3, and incremental patches to Production Functionality Available in Incremental Patches MCE Functionality External Interface functionality RRC Increased functionality System Performance Improvements Erin Jester

Markets System Release Risks Currently one outstanding defect required for Go-Live relating to Markets UI/API Performance Main Risks for Markets System: Required Go-Live Functionality failing in most recent incremental patch with shortened testing timeline Continuing to find new defects required for Go-Live Continuing to experience performance and stability issues Erin Jester

Planned Install and Test Timeline Erin Jester

FERC Order Update On 1/29/2014 SPP received 2 FERC orders which conditionally approve SPP’s proposed tariff provisions Subject to SPP filing additional revisions and explanations Filed no later than 2/28/2014 Will include effective date of 3/1/2014 – to allow SPP to Go-Live on 3/1 Erin Jester

Go-Live Decision Timeline A final Go-Live Recommendation report will be provided to the CWG & Go-Live Team on Monday, February 24 Status of criteria and the outstanding items that were listed as the conditions in the recommendation will be updated at the weekly CWG checkpoints The final Go-Live Team checkpoint is scheduled for Wednesday, February 26 Erin Jester

Upcoming Go-Live Checkpoints Friday, February 7 (9-11am) Cancelled – Friday, February 14 Weekly update will be part of the February CWG meeting Friday, February 21 (9-11am) Erin Jester