Notice of Compliance Audit Phil O’Donnell Manager, Compliance, Operations and Planning Audits and Investigations My Name is Phil O’Donnell, Manager of Operations and Planning Audits I am going to be discussing the Compliance Audit Notice, Information on formats for submitting your documentation and later discuss types of Audits we conduct and some expectations and advice on audit documentation

Audit Frequency 3 Year Cycle Balancing Authority Transmission Operator Reliability Coordinator All other registered functions Subject to flexibility in the future as part of NERC’s Reliability Assurance Initiative Previously 6 years,

Compliance Audit (on-site vs. off-site) Documentation sent to WECC before audit for preliminary review The audit team reviews evidence during off-site week or the first week of the audit and completes its review during the second week or on-site week Data Requests or DRs Tours to observe facilities In-person interviews for clarification In addition to it being longer Primary difference is the face to face contact at your facilities and tours/inspections

Compliance Audit (on-site vs. off-site) Documentation sent to WECC before audit for preliminary review Data Requests or DRs Entity may be present at audit if desired Telephone interviews for clarification In addition to it being longer Primary difference is the face to face contact at your facilities and tours/inspections

Compliance Audit (on-site vs. off-site) Primary difference is: Location of audit conduct Scope is typically smaller for off site On Site – Required for RC, BA, TOP functions Per NERC Rules of Procedure 403.11.2 Primary reason for on site visits is to observe your control centers and other facilities and conduct interviews

Audit Timeline 145 days 90 days 60 days 30 days 15 days AUDIT CIP v5 Request for Information Notice of Audit Pre-Audit Survey Due Objections to Team Members Evidence Due 15 days to return the CIP RFI Inherent Risk Assessment is taking place 180 days for ICE (Internal Controls Evaluation) 145 days for IRA – you’ll receive a survey in that timeline.

Notice of Audit Packet Notice of Audit Letter ATT D: Audit Scope and WECC RSAWs ATT A: Compliance Monitoring Authority ATT E: Certification Letter Letter ATT F: Pre-Audit Survey ATT B: Audit Team Biographies ATT G: Pre-Audit Data Requests ATT C: Confidentiality Agreements ATT H: Post Audit Feedback Form MORE DETAIL LATER If you were to get an audit notice to day this is what you would get. This information will always be available but we are working on changing the process so some of the static information is made available on our website or on request.

Notice of Audit Letter 90-Day Notice of Audit Letter Details of your specific audit Audit Engagement Dates Audit Period Registered Functions within Audit Scope Audit Team Composition Observers (if applicable) May include FERC/NERC Date/time of proposed Pre-Audit Conference Call Links to reference documents Body of Notice ALSO INCLUDES CONTACT INFORMATION

Attachments A, B and C Attachment A Attachment B Attachment C Explanation of Compliance Monitoring Authority Attachment B Short Biographies of the WECC Audit Staff Attachment C Signed Confidentiality Agreements of the WECC Audit Staff

Attachments D and E Attachment D Attachment E Audit Scope Reliability Standard Audit Worksheets (RSAWs) Attachment E Certification Letter Must be printed on your company letterhead and signed by an Authorized Officer Scope is determined by the Inherent Risk Assessment between the Enforcement and Audit Team Leads. And may be modified by the ICE if applicable RSAW’s have been customized for your audit. Please complete the highlighted areas of the worksheets labeled “(Registered Entity Response Required).” Please reference any outstanding self-reports or mitigation plans in each RSAW, as applicable. Please use only the Compliance RSAWs included with YOUR notice package to prepare for the Audit. Some are WECC Regional Standards Some have Regional Variance sections (Not in NERC RSAW) Some enhanced audit approaches. Keep in Editable Word format! ATT E: Certifies that the information being provided for the Audit is accurate.

Attachment F Attachment F Pre-Audit Survey Verify Registered Functions Audit Logistics Signed by Authorized Officer Please complete all applicable fields May be different if on site audit had previous questionnaire sent. Describe company’s organization and structure Description of your system, provide your RC, BA, TOP, PA, TP and RP. If you are a GO, provide your GOP. When listing your PA, TP and RP please confirm with them or state that you have not confirmed this.

Attachment G Attachment G Pre-Audit Data Requests – Clarifications for Data Submittal One Line Diagram Delegation agreements (if applicable) CCA and non-CCA lists Public Key Encryption This is an evidence checklist to provide assistance when filling out your RSAW’s. It has been customized based on registered functions and the audit scope. Due with your evidence submittal – 30 days Some evidence may apply to more than one Standard One copy is sufficient, but document inventories or “roadmaps” are appreciated WECC strongly recommend using PKE for your Cyber Security Documents. This further increase the security process and adds that extra layer of protection. If you choose to utilize PKE please email your certificate or public key to the CPC. Our email and direct lines are located within Att G

Feedback is encouraged for all phases of audit! Attachment H Attachment H Audit Feedback Sent with initial package Feedback is encouraged for all phases of audit!

Evidence Submittal WECC Enhanced File Transfer (EFT) https://fileupload.wecc.biz Any questions regarding log in or user credentials please contact weccsupport@wecc.biz or call 1-877-937-9722 Audit Data Folder We will upload to the wecc notifications folder

Evidence Submittal File Folder COM COM-001-1 Master folder name is the Reliability Standard Sub-folders for all related standards Additional sub-folders for requirements

Evidence Submittal Adobe Portfolios COM Master folder name is the Reliability Standard Portfolio files for related standards in sub-folders with specific standard name Requirement folders within the PDF portfolio

Audit Approaches We audit to the Requirements of the Standards. General Approaches included in RSAW RSAW may ask specific questions Always includes the section: “Describe, in narrative form, how you meet compliance with this requirement.”

Audit Approaches “Describe, in narrative form, how you meet compliance with this requirement.” Describe here how your company knows it is compliant with this requirement and how you know you have been compliant for the entire period of the audit Your place to describe your internal controls Your evidence should support your narrative No need to duplicate information provided through the ICE

List the evidence provided in the RSAW Audit Approaches List the evidence provided in the RSAW This road map is important Compliance Assessment Approach in RSAW is used as a checklist Data Request (DR) for gaps or samples Document and records review are primary Interviews and observations are usually for corroborating

Sufficient Audit Evidence Sufficiency of Evidence The measure of the quantity of evidence Quantity of evidence is dependent on the scope of the audit Extra quantity does not make up for poor quality Ensure you provide enough evidence to demonstrate compliance for the entire audit period HOW MUCH IS ENOUGH

Sufficient Audit Evidence Sampling is used to limit the amount of detailed evidence provided Normally used in conjunction with summary of a full set of data Sampling used to assess details Reduces the burden on the Audit Team but not really on the Entity Audit Team must select the samples SAMPLING

Appropriate Audit Evidence Appropriateness The measure of the quality of evidence Relevance Validity Reliability WHAT IS GOOD EVIDENCE?

Appropriate Audit Evidence Quality of Evidence Good Internal Controls point to reliable evidence Direct observation is more reliable than indirect observation Examination of original documents is more reliable than examination of copies Testimonial evidence from system experts is more reliable than from personnel with indirect or partial knowledge RELATIVE QUALITY

Types of Evidence Physical Evidence Documentary Evidence Testimonial Evidence Compliance Audits may use all three types but Documentary Evidence is by far the most frequent type of evidence assessed and relied on. FORMS OF EVIDENCE

Testimonial Evidence Attestations of Compliance or Statements of Compliance are generally not accepted as the only available evidence. Attestations may be used to explain minor gaps in documentation or to state if no conditions occurred which are subject to a requirement. Attestor must be knowledgeable and qualified. SIMPLE VERBAL ATTESTATIONS NOT VERY GOOD FORMAL WRITTEN ATTESTATIONS A LITTLE BETTER INTERVIEWS ARE VERY VALUABLE TO COOBORATE AND SUPPORT OTHER EVIDENCE.

Evidence for Procedural Documents The characteristics of a valid procedural or policy document include: Document title Definition or Purpose Revision level Effective dates Authorizing signatures DOCUMENTS - PROCEDURES

Non Applicable Requirements Three instances are acceptable for use of term “Not Applicable” Entity is not registered for the applicable function (only TOP responsible for TOP requirements) Entity does not own, operate or maintain the equipment addressed by the requirement (UVLS, UFLS, SPS etc.) Entity does not use the program or process specified by the requirement (and is not required to… ATC, CBM, etc.) WHAT DOES NOT APPLICABLE MEAN

Evidence for Tasks Performed When the standard calls for a task to be performed it must be documented. Records Logs Reports Work Orders Phone recordings Transcripts of phone recordings Shift Schedules Dates & Times are critical PERFORMANCE EVIDENCE

Evidence of “Coordination” with other entities Typical evidence provided initially is a single email. “…If you have any comments please contact ______” This alone is neither sufficient or appropriate to demonstrate coordination between two or more parties. If emails or correspondence are used Two way communications are needed Better are: Meeting Agendas Meeting Minutes Attendance Lists DOCUMENTS FOR COORDINATION

Evidence of “Distribution” of information Typical evidence provided initially is a single email with a large distribution list. “…please see attached” This alone is typically neither sufficient or appropriate to demonstrate distribution to others. If emails or correspondence are used Need clear identification of the personnel on the distribution list. Even better is corroboration by receipt acknowledgement DISTRIBUTION/POSTING