1. Parallel Ops / Go-Live Checkpoint
Weekly Update –
Friday Meeting
January 31, 2014

2. Agenda IDT Debrief 9:00A – 9:20A
IDT Schedule and Postings
IDT Report Review: IDT 1/21 and IDT 1/27
Analysis and Announcements Updates 9:20A – 9:40A
Announcements
MP Behavior / Strategy during Parallel Ops
Scenario Tracker and Scenario Results
Look-Ahead 9:40A – 10:00A
February Schedule
Weekly Updates: Release Scorecard, Migration Report, Known Issues Log
Commitment Notifications Presentation 10:00A – 10:15A
Settlements Update 10:15A – 10:30A
Readiness Update 10:30A – 11:30A

3. Remaining IDTs IDT Date IDT Times Conditions Notes 2/25/14 8:30A-4:30P
6hr IDT begins at 9:30A
2/28/14
7:00P
4hr IDT begins at 8:00P
Cutover for March 1 Go-Live
Ginny Watson

4. IDT Debrief slides on spp.org here
IDT 01/21 Report
IDT reports posted to spp.org here 7 calendar days after each IDT
spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test
IDT 01/27 Report
IDT report posted early to spp.org here
IDT Debrief slides on spp.org here
spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test
For your reference:
01/27/14 IDT Preliminary Debrief
01/21/14 Reserve Selection IDT
01/21/14 IDT Preliminary Debrief
Brian Strickland

5. Announcements Please continue to submit dual tags through February
Even though these days will not be “Bid to Bill” days, MPs should continue entering dual tags in February
Thank you for your participation!
Shari Brown

6. Announcements RMS Change
As Parallel Ops ends 1/31, there will no longer be a Parallel Ops QuickPick.
Continue to submit routine questions to the default “General Inquiry”
For specific “Go Live” questions or February IDT questions, choose the new “Go Live” QuickPick
Ginny Watson

7. Announcements Extended Maintenance Window
Wed Feb 5 – Fri Feb 21: 24/7 availability cannot be guaranteed
Current functionality and availability will be maintained until Feb 5
Complete any extensive/critical testing prior to the start of this maintenance period
Nothing will be settled during the maintenance period
Systems will be fully available from 8A-9A CPT daily during the maintenance window
Intermittent maintenance will occur outside of 8A-9A CPT
System outages, errors, and performance issues may occur
Notifications will only be sent for outages that impact systems for more than 24 hours or during the 9A-9A CPT availability window
Markets Systems bids/offers can be submitted during the 8A – 9A CPT window
On Feb 22, MPs may begin submitting Bilateral Settlements Schedules and bids/offers for OD March 1
Settling will begin OD March 1 with Meter Agent submittals beginning March 2 and initial postings on March 10
Ginny Watson

8. Market Participant Parallel Ops Behavior Concerns
Poor Meter Data Quality: Several of the Market Participants’ Meter Agents are submitting very low quality Meter Data.
Inconsistently Poor – it is hard to predict which specific MPs will submit poor data on any given day. There are few consistent poor performers. There are A LOT OF GOOD performers.
Don’t believe this is due to lack of understanding or ability. Likely due to Parallel Ops ‘burn out’. This is likely a Parallel Ops issue that will go away when the market goes live.
SPP is reaching out to MPs that appear to have consistently high Meter Data Calibration and assure they are capable to improving the quality of their submissions.
Low MP Submission of Interchange Schedules: Many (Most?) Market Participants are not ‘dual’ submitting their SPP Interchange Schedules.
This affects MPs individually and globally:
Individual MPs will not observe their true settlement Revenue/Costs due to the Non- Settlement of the missing Interchange Schedules. Globally the missing Interchange Schedules create an artificially LARGE Inadvertent Energy Settlement.
SPP is reaching out to MPs that have lowest Interchange Schedule participation to assure they are capable and comfortable with the submission process.
Casey Cathey

9. Market Participant Parallel Ops Behavior Concerns
Day Ahead Load Bids un-aligned with Load Forecasts: Some MP’s appear to be over scheduling (over bidding) their Load in the DA Market.
The difference between their cleared DA Load the SPP Load forecast is very large – as high as 1,000 MWs per hour.
These delta MWs impact the DA solution and DA LMPs. They indirectly result in RT Asset Settlement being poor indicator of expected Market Results.
SPP will reach out to the MPs with large DA Schedule vs Load Forecast deltas to assure they are aware of the differences and have the ability to schedule more ‘accurately’.
Some MPs showing consistent Revenue Losses: There are a few MPs that are consistently showing net losses.
SPP recognizes that most MPs are not attempting optimal operations at this time. This data is accurate, but SPP is cautious using it as a stand alone indicator of an MPs ability to be profitable/successful in SPPs Market.
SPP will reach out to the MPs with the appearance of poor market results to assure they are aware of their overall results and have a good understanding of the activities required at go-live to achieve optimal outcomes.
Casey Cathey

10. DA Shortage & Excess Capacity Scenarios
SPP is currently awaiting some more fixes for existing defects in the Day-Ahead emergency logic.
Many of the defects were taken care of in the R1.9.3 patch, but a few more remain to be delivered
Fixes are due in a patch with R1.9.3
SPP will retest Scenarios in lower environments as soon as available
Detailed results will be presented to MPs.
Jarrett Friddle

11. Scenario Tracker as of Jan 31
Summary – Scenarios in Parallel Ops:
11 Scenarios, 2 Partial Scenarios, Dispatch Simulator Test successful:
Scenario 7 Must Offer
Scenario 28a
Scenario 15 Commitment Mismatch Part 2
Scenario 24
Scenario 13 DDR w/ Settlements
Scenario 29 RSG
Scenario 31 BDR w/ Settlements
Scenario 36 GFA
Scenario 11 EDR
Mitigation Scenarios 16.1, 16.2, 16.3
Non-Functional Test Scenario
Dispatch Simulator Test
Note: Dispatch Simulator Test occurred 12/6 – 12/9 and again 1/15 – 1/17.
Summary – Retests Pending:
2 individual retest requests by 1 MP are scheduled for Jan 31 & Feb 4
Shortage & Excess Capacity Scenarios still being retested in lower environments
Ginny Watson

12. Scenario 31 BDR w/ Settlements
Results
On the Markets side, this Scenario passed
Market Participant data was correct.
Resource behaved as designed in RTBM and EMS.
Dispatch instructions were observed via XML and Markets UI.
BDR Settlements Statements were posted on Jan 28
Jarrett Friddle

13. Non-Functional Testing
Results
Successfully tested January 23rd
Result: EMS failed over correctly; functioned as expected
Result: ICCP failed over correctly, functioned as expected
Result: Marketplace Portal and API failed over correctly. Users may have experienced brief intermittent connectivity when SPP’s primary facility was taken offline but no issues were reported.
Result: Web Services (RTO_SS) Remained available during the exercise, as expected
Result: No RTBM studies were missed during the exercise and ID-RUC studies completed successfully
Result: was unavailable for approximately 16 minutes, as expected.
After the exercise, failover back to SPP’s primary facility was successful although some monitoring applications had some small glitches that were identified and fixed within the hour.
CJ Brown

14. Scenario 11 EDR
Results
SPP successfully executed the EDR scenario during the Jan 27 IDT
From an RTBM perspective, the scenario was a success. SPP sent the expected dispatch via Markets UI, XML, and ICCP.
MP requested to retest their resolved issues on Jan 31
Jarrett Friddle

15. Mitigation Scenarios Summary
Commitment
348 Resources submitted at least one hour in ‘Market’ commit status
264 Resources committed (‘Market’ and ‘Self’)
37 Resources with ‘Market’ commit status failed the Start-Up Offer Conduct Test
19 of these Resources were committed
45 Resources with ‘Market’ commit status failed the No-Load Offer conduct test
18 of these Resources were committed
Dispatch
41 committed Resources failed the Energy Offer Conduct Test and were mitigated for at least one hour
169 Resources cleared for Operating Reserve Product MWs
20 of these Resources failed the Operating Reserve conduct test and were mitigated for at least one hour
Jarrett Friddle

16. 16.1 DA Market Mitigation
The Day Ahead Market for OD 01/29 was completed with Mitigation enforced.
5 MPs submitted specific resources to test Mitigation with Scenario 16.1
3 of the 5 were mitigated
The other 2 submitted mitigation and DAMKT offer curves the same and did not fail the behavior test.
There were 62 resources ultimately mitigated in the DA Market Commitment including those explicitly identified in RMS tickets.
Market Monitoring has reviewed cases and agrees that the Mitigation is working appropriately.
In order to obtain mitigation examples in the entire footprint, Reserve Zone minimum requirements were set to a level to ensure each Reserve Zone was binding.
In order to do that, all Reserve Zone minimum limits were raised to an unattainable level. This did effectively mitigate all the resources we had hoped but it also resulted in the Demand Curves for the Upward reserve Products being enforced and thus, all of the UP AS Product MCPs were $1100 for all hours.
Jarrett Friddle

17. 16.2 DA RUC Mitigation
The Day Ahead RUC for OD 01/29 was completed with Mitigation enforced.
6 MPs submitted specific resources to test Mitigation with Scenario 16.2
All 6 were successfully mitigated and committed through the DA RUC process.
All entities should have been able to see the appropriate Mitigation indicators in the Market UI and API messages.
There were several additional resources ultimately mitigated besides those explicitly identified in RMS tickets.
14 Resource were mitigated for Commitment.
Jarrett Friddle

18. Scenario 16.3 RTBM Mitigation
Results
Scenario results from 1/28 and 1/28
A total of 104 Resources experienced energy mitigation in at least 1 RTBM interval
MPs submitted 8 RMS tickets identifying Resources that were to participate in this Scenario
All 8 Resources experienced mitigation over the 48 hour test period
During business hours, mitigation tuning parameters were tightened in an attempt to force more Resources to mitigate
Mitigation did occur around the clock, with the protocol parameters
Jarrett Friddle

19. Look-Ahead: Jan 19 – Jan 31 Parallel Ops Calendar on spp.org
spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents
Settlements details on the Settlements Calendar
marketplace.spp.org > Public > Settlements > Calendars
Feb 1 – Mar 1 Schedule:
Wed Feb 5: SPP Maintenance Window Begins.
Fri Feb 7: Weekly Go-Live Checkpoint - Register on spp.org. Same Dial In and WebEx
Wed Feb 12: CWG
Thurs Feb 13: CWG
Fri Feb 14: No Regular Friday Meeting
Mon Feb 17: SPP Holiday
Fri Feb 21: Weekly Go-Live Checkpoint
Mon Feb 24: IDT Prep Call 3P
Tues Feb 25: IDT Call 8:30A – 4:30P, 6 hour IDT begins at 9:30A
Wed Feb 26: IDT Debrief Call 3P
Thurs Feb 27: IDT Prep Call 3P
Fri Feb 28: IDT Call 7P, Go-Live cutover IDT begins at 8P
Ginny Watson

20. Migration and Maintenance Report
Migration and Maintenance Report posted to spp.org
spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents
Note: Discontinued after 1/31
Ginny Watson

21. Release Scorecard Release Scorecard posted to spp.org:
spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents
Note: Discontinued after 1/31
Ginny Watson

22. Known Issues Log
Please review the Market Trials Known Issues Log on spp.org
spp.org > Integrated Marketplace > Market Trials > Structured/Unstructured Test > Integrated Marketplace Market Trials Project Documentation Folder
Note: Will continue after Parallel Ops ends
Issue #
Component
Synopsis
Severity
903
Markets Systems
Gen Resource with scadastatus = NOTFOLLOWDISPATCH getting CMODE = 1; the CMODE should be = 3
Severity 2: High
904
1st interval, the cmode is 3 (and for DVERs follow dispatch flag is false). Under these conditions, MCE does not calculate an achievable MW.
2nd interval, the cmode switches to 1 or 2 and/or the follow dispatch flag is true. Since MCE does not have an achievable MW, it uses the initial MW and applies the ramp rate over 10 minutes to calculate the dispatch.
For the first interval a resource is in a control mode of 1 or 2 and/or follow dispatch flag is true, MCE should use the initial MW and apply the ramp rate over 5 minutes for the dispatch MW. .
Severity 3: Medium
Jodi Woods

23. Go-Live Useful Resources
Readiness Center
Integrated Deployment Test
spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test
IDT MP Guide
Parallel Ops Calendar
Additional documents:
Settlements Calendar marketplace.spp.org > Public > Settlements > Calendars
Change Tracker spp.org > Integrated Marketplace > Marketplace Documents

24. Commitment Notifications
General Commitment Timelines: (When does this happen?)
Day-Ahead RUC notifications issued around 20:00 hrs.
It could be earlier or later
If it is late, there should be a Market Message indicating the DA RUC will be late for the day
Intra-Day RUC/Real-Time notifications can happen at any time
Notification: (How do I navigate to see it?)
Via an XML notification (PostEnergyCommitmentSet)
Via Markets UI
Markets User Interface Guide for more information
Jim Gonzalez

25. Commitment Notifications by Asset Owner
Query Resource Commitment Schedules by Asset Owner
Step
Action
Comments 1
Access the Marketplace Portal
The Portal Address is: 2
Select the Resource Offers tab. 3
In the Navigation tree, select the appropriate
Asset Owner.
Navigation tree is located on the left-hand side of the Markets UI. 4
Select a date. 5
Select Market Results. 6
Choose Commitments, and select View.
Jim Gonzalez

26. Commitment Notifications by Asset Owner
Query Historical Resource Commitment Schedules by Asset Owner
Step
Action
Comments 1
Access the Marketplace Portal
The Portal Address is: 2
Select the Resource Offers tab. 3
In the Navigation tree, select the appropriate
Asset Owner.
Navigation tree is located on the left-hand side of the Markets UI. 4
Select a date. 5
Select Market Results. 6
Choose Commitment History, and select View.
Jim Gonzalez

27. Commitment Notifications by Resource
Query Resource Commitment Schedules by Resource
Step
Action
Comments 1
Access the Marketplace Portal
The Portal Address is: 2
Select the Resource Offers tab. 3
In the Navigation tree, select the appropriate
Asset Owner.
Navigation tree is located on the left-hand side of the Markets UI. 4
Select a date. 5
Select Resources. 6
Select a resource. 7
Choose Commitments, and select View.
Jim Gonzalez

28. Commitment Notifications by Resource
Query Historical Resource Commitment Schedules by Resource
Step
Action
Comments 1
Access the Marketplace Portal
The Portal Address is: 2
Select the Resource Offers tab. 3
In the Navigation tree, select the appropriate
Asset Owner.
Navigation tree is located on the left-hand side of the Markets UI. 4
Select a date. 5
Select Resources. 6
Select a resource. 7
Choose Commitment History, and select View.
Jim Gonzalez

29. Commitment Notifications via API
Notification comes via XML
Executing the following commands to access commitment schedules:
GetEnergyCommitmentSet
GetEnergyCommitmentSetByDay
GetEnergyCommitmentHistorySet
GetEnergyCommitmentHistorySetByDay
Follow Up Item: Do these commands differentiate between asset owner and resource? Or are those separate commands?
Jim Gonzalez

30. Commitment Notifications via API
XML Listener Server can be established to notify MPs of any commitment notifications. Execute the command below:
PostEnergyCommitmentSet notification
Jim Gonzalez

31. Commitment Notifications Follow Up
SPP is not planning on following up with RUC Commitment notifications via phone call
ID RUC Process Commitments (both as a result of ID RUC Study Suggestions and as a result of RUC Operator Manual Commitment) can occur at any time after the conclusion of the DA RUC Process
All Resource Commitment Notifications (MUI and API) are consistent in format
Jim Gonzalez

32. Brent Wilcox
Settlements Update
Brent Wilcox

33. Settlements Update - February
Last statements (1/31 I) will be posted Feb 3rd.
Please see highlighted section of February calendar for details
Statements will remain on portal until noon on Feb 5th
Settlement system cutover begins the afternoon of Feb 5th.
Due to the extended maintenance period any meter or bilateral settlement schedules submitted between Feb 5th and Feb 21st may go to ‘Pending’ or error out. This is to be expected.
Brent Wilcox

34. Settlements Update – February Cont’d
Market Participants can start to submit their Bilateral Settlement schedules on Feb 22nd.
Any Bilateral Settlement schedules submitted prior to February 22nd will be deleted as a part of the system cutover
Any Bilateral Settlement schedules submitted February 22nd will be live production data.
Meter windows will open on OD+1 (3/2/14). 
Any meter data submitted prior to March 2nd will be deleted to ensure test data is not mixed with live data.
Brent Wilcox

35. Settlements Outreach – Announcement
SPP Settlements will be offering two informational sessions next month. The topics to be covered are:
Metering
Bilateral Settlement Schedules
Transition from EIS to Integrated Marketplace and the overlap between the two 
Q/A
The sessions are tentatively scheduled for Feb 11 and Feb 18 from 2-3CST. Please look for a formal announcement in the next few days and sign up for one of the sessions if interested.
Brent Wilcox

36. Erin Jester
READINESS Update
Erin Jester

37. Update Market Offer Data
Markets Test Data vs. Production Level Data
Market Offer data will NOT be purged from the Markets Systems prior to 3/1
Includes existing offer data submitted since June 2013 Market Trials start -- even MP offer data that was seeded by SPP for testing purposes
Market Participant test offer data MAY become financially binding on 3/1 and beyond if preventative measures aren’t taken before 2/22
Erin Jester

38. Update Market Offer Data
MP Required Action for Updating Market Offer Data
Complete Acknowledgement Letter by 2/14
Market Trials Liaisons received RMS ticket on 1/17 requesting acknowledgement of responsibility to update Market offer data and understanding of risks associated with having non-production ready offer data in the Market Systems after 2/22
Acknowledgement to be completed via RMS no later than 2/14
Escalation will occur for MPs not completing acknowledgement by 2/14
Update all Market Offer Data in the Markets System by 2/22
To avoid test data becoming financially binding, update offer and parameter data for DA Resource Offers, RTBM Resource Offers and Mitigated Offers
Market offer data MUST be updated prior to 2/22 to ensure good, production-ready data for Go-Live
Erin Jester

39. Production Credit Management System
Market Participants can begin submitting Offers/Bids, including Virtual Offers/Bids, for a 3/1 Operating Day on 2/22. The MPs can also continue to submit for Operating Days prior to 3/1
Beginning 2/22, the production Credit Management System will interface with the production Market system and any virtual transaction submitted will be validated by the production credit system. The MPs credit limit will be impacted by these transactions.
Beginning 2/22, virtual transactions should not be submitted for an Operating Day prior to 3/1.
If a virtual transaction is submitted, the MP will be asked to cancel the transaction. If the transaction is not cancelled, the MPs credit limit will reflect the existing transaction.
Jodi Woods

40. IDT URD Report
Beginning in January, SPP sends a report to each MP that failed a URD test in at least one applicable interval during the previous IDT
The market trials liaison for the applicable MPs received an with the subject: SPP Action Requested: Review Resource Response during Jan XX IDT
The following slides illustrate aggregate-level URD trends over the Dec/Jan IDTs
Casey Cathey

41. IDT URD Report
Top 3 MPs with failed URDs as a percentage of total SETPOINT-SCADA difference by MP
Casey Cathey

42. IDT URD Report
Top 3 MPs with failed URDs as a percentage of total SETPOINT-SCADA difference by MP
Casey Cathey

43. IDT URD Report
Top 3 MPs with failed URDs as a percentage of total SETPOINT-SCADA difference by MP
Casey Cathey

44. IDT URD Report
Percentage of intervals to pass URD tests, and average SETPOINT-SCADA difference of URD failures
Casey Cathey

45. Readiness Metric Update
Updated Readiness metrics status will be posted on iDashboard by EOD today (1/31)
Readiness metrics will be continued to be tracked up to Go-Live and some will continue after
Detailed plans for Feb/Mar will be presented at the Feb CWG meeting
Metrics measuring MPs (i.e. CROW, ICCP) will be tracked during February with the understanding that it is during the maintenance window
Some of these metrics will not be ‘green’ prior to Go-Live but are not considered an issue for Go-Live
Potential financial impacts were discussed at the January CWG
Erin Jester

46. CROW Metric Update (OPS-15)
Metric ID
Metric Name
Status
OPS-15
CROW Outage Submission
Red
1/19-1/25
12/15-12/21
43% 4%
53%
Percentage out of 100
OPS-15 is still reported as ‘red’ (through 1/25), but SPP has seen a very positive trend towards timely submissions in CROW over the past two weeks.
Erin Jester

47. ICCP Metric Update (OPS-18)
Metric ID
Metric Name
Status
OPS-18
ICCP Control Status Tracking
Red
11/18-11/22
1/18-1/25
Average percentage of control modes submitted correctly
Percentage of MPs with 100% of control modes submitted correctly
Erin Jester

48. Go-Live Assessment Update
Updates on criteria status and critical outstanding items are in the following slides
Minor Changes to report:
Added formal CWG recommendation
Updated test data for GL-04 (SAT completion), GL-06 (FIT completion), and GL-08 (NFT completion)
Updated Markets and Settlements defects in GL-09 (Critical defects) and GL-10 (High defects)
Moved GL-20 (Parallel Ops exit criteria) from ‘at risk’ to ‘not met’
Erin Jester

49. Go-Live Criteria Update
21 Go-Live Criteria Total
7 complete
11 tracking on time
GL-18 (FERC approval order) moved from ‘at risk’
1 at risk
2 not met
GL-20 (Parallel Ops Criteria met) moved from ‘at risk’ to ‘not met’)
Latest status of criteria will be posted in the Go/No- Go Folder on SPP.org by EOD 1/31
Erin Jester

50. Go-Live Assessment Update
GL-20: Completion of approved open Parallel Ops Exit Criteria
Parallel Ops Exit Criteria that were not met by 1/31:
Updates on the status of these criteria will be provided during the weekly CWG Checkpoints
Erin Jester

51. Go-Live Assessment Update
Outstanding items that need to be resolved prior to Go-Live:
Critical Markets functionality scheduled for Markets Operations System (MOS) Release 1.9.3
Market Control and Market Database (MDB) performance improvements
Mitigation and Market Clearing Engine (MCE) Updates
Frequently Congested Areas (FCA) and External Dynamic Resource (EDR) logic
Markets UI/API defects addressing MP requests
MCE certification
NERC certification of the SPP BA (no change in status)
FERC approval
Erin Jester

52. Markets System Release Update
MOS System Release Plan
1/21 – Delivery of MCE and RRC Patches from Alstom
1/23 – Delivery of MCE and MDB Patches from Alstom
1/28 – Delivery of MCE, MDB and MUI Patches from Alstom
1/29 – Delivery of MDB Patch from Alstom
1/30 – Delivery of MCE, MDB, and External Interface Patches from Alstom
2/3 – Promotion of R1.9.3, and incremental patches to Production
Functionality Available in Incremental Patches
MCE Functionality
External Interface functionality
RRC Increased functionality
System Performance Improvements
Erin Jester

53. Markets System Release Risks
Currently one outstanding defect required for Go-Live relating to Markets UI/API Performance
Main Risks for Markets System:
Required Go-Live Functionality failing in most recent incremental patch with shortened testing timeline
Continuing to find new defects required for Go-Live
Continuing to experience performance and stability issues
Erin Jester

54. Planned Install and Test Timeline
Erin Jester

55. FERC Order Update
On 1/29/2014 SPP received 2 FERC orders which conditionally approve SPP’s proposed tariff provisions
Subject to SPP filing additional revisions and explanations
Filed no later than 2/28/2014
Will include effective date of 3/1/2014 – to allow SPP to Go-Live on 3/1
Erin Jester

56. Go-Live Decision Timeline
A final Go-Live Recommendation report will be provided to the CWG & Go-Live Team on Monday, February 24
Status of criteria and the outstanding items that were listed as the conditions in the recommendation will be updated at the weekly CWG checkpoints
The final Go-Live Team checkpoint is scheduled for Wednesday, February 26
Erin Jester

57. Upcoming Go-Live Checkpoints
Friday, February 7 (9-11am)
Cancelled – Friday, February 14
Weekly update will be part of the February CWG meeting
Friday, February 21 (9-11am)
Erin Jester