1. 1 Audit Preparation - Evidence ReliabilityFirst CIP Webinar Thursday, September 30, 2010 Lew Folkerth, Senior Engineer - Compliance

2. Painting the “Compliance Picture” All aspects of each requirement The entire compliance period All appropriate assets Appropriate level of detail to demonstrate compliance During an audit, the picture is painted by the SME The picture is painted with EVIDENCE 2

3. Evidence Preparation What is evidence? What constitutes “good” or “quality” evidence? What is the best way to present evidence? When should evidence be submitted? Can an entity be found compliant with a requirement based solely on pre-audit evidence submitted? 3

4. Evidence Preparation What is evidence? Physical  Obtained by direct inspection or observation Documentary  Processes, procedures, databases, written records Testimonial  Interviews, inquiries, questionnaires, attestations What constitutes “good” or “quality” evidence? What is the best way to present evidence? When should evidence be submitted? Can an entity be found compliant with a requirement based solely on pre-audit evidence submitted? 4

5. Evidence Preparation What is evidence? What constitutes “good” or “quality” evidence? Sufficient  Measure of the quantity of evidence that supports the audit objectives – enough to persuade a knowledgeable person that the findings are reasonable Appropriate  Measure of the quality of evidence that includes relevance, validity and reliability What is the best way to present evidence? When should evidence be submitted? Can an entity be found compliant with a requirement based solely on pre-audit evidence submitted? 5

6. Evidence Preparation What is evidence? What constitutes “good” or “quality” evidence? What is the best way to present evidence? There is no “best” way – whatever fits your evidence organization However, here are some tips on how to organize and present the evidence When should evidence be submitted? Can an entity be found compliant with a requirement based solely on pre-audit evidence submitted? 6

7. Evidence Organization and Presentation PDF documents are greatly preferred PDFs should be searchable If you want to scan pages to show signatures, only scan the signature page and include it after the searchable document 7

8. Suggestions: One PDF per standard (may be large) All versions of compliance evidence applicable to the compliance period Revisions of each document together, in reverse chronological order Two sets of bookmarks:  First set to locate documents  Second set to locate sections within documents which satisfy a requirement 8 Evidence Organization and Presentation

9. 9

10. 10 Evidence Organization and Presentation

11. Evidence Preparation What is evidence? What constitutes “good” or “quality” evidence? What is the best way to present evidence? When should evidence be submitted? Pay careful attention to the audit notification letter Most evidence will be needed 40 days before the start of the audit Provide additional information as requested – sooner is better than later Can an entity be found compliant with a requirement based solely on pre-audit evidence submitted? 11

12. Evidence Preparation What is evidence? What constitutes “good” or “quality” evidence? What is the best way to present evidence? When should evidence be submitted? Can an entity be found compliant with a requirement based solely on pre-audit evidence submitted? YES! At least for some (perhaps many) requirements, although typically not a majority of requirements. But, careful and thorough preparation is required to achieve this. 12

13. Evidence Presentation Keep it simple – the audit team will indicate where more detail is needed Don’t evade questions – “I don’t know” is an acceptable answer, as long as follow- up occurs Don’t provide slide decks to demonstrate compliance – the audit team is interested in evidence 13

14. PDF Bookmarks Demo 14

15. Questions Questions should be emailed to Karen Yoder (karen.yoder@rfirst.org) Subject: “CIP WEBINAR”karen.yoder@rfirst.org Questions will be considered in the order they are received Clarifying questions are welcome and we will do our best to answer during the question period Challenges to a position should be addressed to the presenter and will be taken offline