1. Project 071103 CIM Test Development Process John Simmins Project Overview 1/27/2011

2. 2 © 2010 Electric Power Research Institute, Inc. All rights reserved. CIM test process overview CIM Test Methodology compatible with –IEC WG 14 –CIM Users Group –OpenSG And possibly MultiSpeak Test process defined, documented and repeatable Vetted with multiple utilities, vendors and user groups Tests designed for multiple implementation environments (test only messages, implementation agnostic) –Legacy point-to-point –SOA implementation –Various ESBs (WebSphere, WebLogic, TIBCO, GlassFish) Tests openly available and published on Open Smart Grid (OpenSG)

3. 3 © 2010 Electric Power Research Institute, Inc. All rights reserved. Project Participants Utility Sponsors Consumers Energy, AEP, SCE, PG&E Industry Open Smart Grid (OpenSG) Project Leadership Electric Power Research Institute (EPRI)/EnerNex

4. 4 © 2010 Electric Power Research Institute, Inc. All rights reserved. Schedule & SOW Task Deliverables

5. 5 © 2010 Electric Power Research Institute, Inc. All rights reserved. Best Practice Test Documentation Test Plan Overall plan defining goals, schedule and expected outcome of the test program Includes stakeholder roles and responsibilities Test Cases Defines test objectives and units under test (UUTs) Includes test requirements and req. traceability matrix (RTM) Test Procedures Detailed step-by-step instructions to execute formal tests Includes test setup, data recording and test data sets Test Report Documents Test Results Lists test discrepancies and status of corrective actions

6. 6 © 2010 Electric Power Research Institute, Inc. All rights reserved. CIM-based Test Overview CIM Tests Abstract Test Cases Conformance Tests Interoperability Tests Concrete Detailed Test Procedures Conformance Tests Interoperability Tests

7. 7 © 2010 Electric Power Research Institute, Inc. All rights reserved. Concrete Test Procedures Detailed step-by-step instructions needed to execute tests in laboratory environment Includes test set-up steps, exact message content, expected results and pass/fail criteria Concrete Detailed Test Procedures Conformance Tests Verifies adherence to IEC 61968 and 61970 standards Validates systems from different vendors can operate together using CIM-formatted messages Interoperability Tests

8. 8 © 2010 Electric Power Research Institute, Inc. All rights reserved. Test Case Development Process

9. 9 © 2010 Electric Power Research Institute, Inc. All rights reserved. Interoperability Test Procedure Development Process

10. 10 © 2010 Electric Power Research Institute, Inc. All rights reserved. Preparation for Formal Test Execution 1. Management Coordination2. Technical Coordination 3. Ensure test readiness by dry running the test procedures

11. 11 © 2010 Electric Power Research Institute, Inc. All rights reserved. 1. Management Coordination of Formal Tests 1.1 Review goals and desired outcomes1.2 Address Intellectual Property Concerns 1.3 Sign MOUs and NDAs as required 1.4 Identify POCs for test coordination

12. 12 © 2010 Electric Power Research Institute, Inc. All rights reserved. 2. Technical Coordination for Formal Tests 2.1 Identify and address IT issues and concerns 2.2 Identify and address security needs and requirements 2.3 Document test equipment setup process including test servers/clients 2.4 Identify and document transport level setup process

13. 13 © 2010 Electric Power Research Institute, Inc. All rights reserved. 3. Ensure Test Readiness/Dry Run Test Procedures 3.1 Coordinate dry run schedule with POCs 3.2 Run through each test procedure to identity errors and ambiguities in the test 3.3 Markup and update test procedure as required. Rerun if required. 3.4 Run through test procedures with each interested party

14. 14 © 2010 Electric Power Research Institute, Inc. All rights reserved. Questions & Contact Informatation John J. Simmins, Ph.D. Senior Project Manager – Smart Grid Electric Power Research Institute (EPRI) 942 Corridor Park Blvd. Knoxville, TN 37932 865-218-8110 (Office) 301-848-5153 (Cell) jsimmins@epri.com www.epri.com