1. Compliance Performance Measure Proposals and Preliminary Trends Greg Pierce – Chair, Performance Measures Task Force Compliance and Certification Committee Meeting March 10-11, 2010

2. 2Topics  PMTF Tasks and Approach  Performance Measure Proposals  Preliminary Trends  Outreach Effort  Next Steps

3. 3 PMTF Tasks  Establish an open, consistent and repeatable metric development process to identify and evaluate metrics  Develop compliance improvement measures at the highest levels to provide insight in terms of a set of compliance principles  Coordinate and work with Regional Entities, RMWG, OC, PC and CIPC to ensure agreement  Recommend how to benefit from the compliance improvement measures

4. 4Approach  Adopt RMWG’s open metric development process  Review data source on compliance and other key input factors  Propose a set of performance measures at high levels  Gather data and validate trends  Solicit feedback and other ideas from stakeholders  Prepare a draft report starting April 2010

5. 5 Proposed Performance Measures  Compliance Culture  Timeliness of CVI/CIQ  Average Duration of Mitigation Plan Completion  Violation Risk Index  Average Duration of Audit Report Completion w/ No Violations  Average Duration of Audit Report Completion w/ Potential Violations  Availability/timeliness of 'lessons-learned’ guidance from audits, operating experience, violations and disturbances  Feedback to standards development  The detailed individual metric definitions and specifications are available at http://www.nerc.com/filez/pmtf.html.http://www.nerc.com/filez/pmtf.html

6. 6 Compliance Culture Measure  Metric – Compare Self-Disclosed Violations to the Total Number of Discovered Violations  Self-Disclosed = Self-report + Self-certification  Indicates the level of trust and cooperation within the “NERC Enterprise” as well as the culture of compliance of the stakeholder community

7. 7 Trend By Discovery Method

8. 8 Compliance Culture Trend

9. 9 Timeliness of CVI/CIQ  Average Duration – for CVI/CIQ with Open status  Total number of CVIs/CIQs for Open and Closed status  Number of lessons learned and trends issued to industry

10. 10 CVI/CIQ Timeliness Measure Duration (Days)CVICIQ OldestAverageOldestAverage NERC644317361242 Region823444126107

11. 11 Triggers for NERC/Regional CVIs/CIQs

12. 12 Average Duration of Mitigation Plan Completion  Duration – from Date plan received to Mitigation close date  Sum of all durations divided by total number of violations discovered in a particular quarter Remove zero duration days

13. 13 Average Number of Days for Completing Mitigation Plans

14. 14 Violation Risk Index (VRI)  Violation Risk Index is defined as a risk factor and severity level weighted average based on unmitigated violations Assess potential consequences of a particular unmitigated violation. Unmitigated violations of higher risk factor requirements have more weighting value than of lower risk factors. Determine the change in reliability levels due to Standard requirement violations.  Weighting factors and an example are included in Appendix A.

15. 15 NERC Unmitigated Violations (6/18/07 to 12/31/2009)

16. 16 Violation Risk Trend

17. 17 Average Duration of Audit Report Completion w/ No Violations  Measure audit process efficiency and ability to promptly advise industry of audit results  These reports do not have potential violations  Starting point: audit ending point  Ending point: report finalized with entity and received by NERC

18. 18 Average Duration of Audit Report Completion w/ No Violations

19. 19 Average Duration of Audit Report Completion w/ Potential Violations  Measure enforcement process efficiency and ability to promptly advise industry of audit results  These reports have potential violations  Starting point: audit ending point  Ending point: report finalized with entity and received by NERC  Data is not available until second quarter of 2010

20. 20 Timeliness of Lessons-Learned  Establish a common and central platform to share lessons-learned  Timeliness of actionable lesson-learned guidance from operating experience, violations and disturbance events Lessons-Learned guidance Recommendations from event analysis Compliance analysis Advisories

21. 21 Feedback to Standards  Timeliness of compliance results/feedback to standards # of SARs (e.g., refine a particular requirement) # of suggestion and comment submittals The Standards input form is available at http://www.nerc.com/files/Standards_Input_Form_Fin al_2008June30.doc http://www.nerc.com/files/Standards_Input_Form_Fin al_2008June30.doc  Direct input to “results-based” standards initiative Focus on risk-based standards, using first assessment triage sheet

22. 22 Outreach Effort  Chair met with and presented to the RMWG on February 18, 2010  Chair presented to the NERC/Regional Entity Compliance Group at its conference call on February 26, 2010  Chair presented to the OC/PC/CIPC at their joint WebEx conference call on March 9, 2010

23. 23 Next Steps  Continue reaching out to seek guidance Members will meet with OC and PC on March 16-17, 2010  Recommend the proposals in June, desiring CCC approval in September 2010  Comments and suggestions are due April 2, 2010 Send to PMTF Chair, Greg Pierce at GPIERC2@entergy.com and NERC at jessica.bian@nerc.net GPIERC2@entergy.com jessica.bian@nerc.net

24. 24 Question & Answer

25. 25 Appendix A  Violation Weighting Factor Table Note: 1. The weighting values are derived by applying similar ratios developed in the Base Penalty Amount Table described in section 4 of the ERO Sanction Guidelines, Appendix 4B to the NERC Rules of Procedure. 2. Reference materials are available in a NERC white paper “Toward Ensuring Reliability: Reliability Performance Metrics”. It can be viewed at: http://www.nerc.com/docs/pc/rmwg/Reliability_Metrics_white_paper.pdfhttp://www.nerc.com/docs/pc/rmwg/Reliability_Metrics_white_paper.pdf.

26. 26 VRI Example