Company Confidential Registration Management Committee (RMC) 1 Lessons Learned from Accreditation Assessments - ANAB Perspective Boston, MA July 21, 2011 Steve Holladay ANAB, Accreditation Assessor Auditor Workshop Boston, MA July 21-22, 2011

Registration Management Committee (RMC) Boston MA July 21-22, 2011 Objectives Provide a summary of the NCR’s identified from the witness audits Provide a summary of the NCR’s identified from the office audit in Discuss four examples of the most common NCR’s. Rules of engagement. 2

Registration Management Committee (RMC) Boston MA July 21-22, 2011 Goal Provide knowledge/information to the AQMS auditor pool –with the goal to improve auditor/CB performance –that will reduce repeat NCR’s –which will, in turn, add value to the assessment process and the industry as a whole. 3

Registration Management Committee (RMC) Boston MA July 21-22, Overall NCR’s in 40 Witnessed Audits (WA) –1.5 Average per WA –22 Major NCR’s 162 NCR’s in approx 42 Office Assessments (OA) –3.85 Average per OA –21 Major NCRs –22 AS requirements NCR’s in 37 Witnessed Audits (WA) –2.45 Average per WA –19 Major NCR’s 223 NCR’s in approx 37 Office Assessments (OA) –6.03 Average per OA –30 Major NCRs –46 AS requirements

Registration Management Committee (RMC) Boston MA July 21-22, Audit team competency, Cert decision, notification of changes from client, OASIS admin, use of the mark. 1 NCR each. Stage 1 objectives, Stage 2 objectives, Suspension process. 2 NCR’s each. Certification Cycle, corrective action process, use of the Mark. 3 NCR’s each Witness Audit NCR’s

Registration Management Committee (RMC) Boston MA July 21-22, Audit time and report issues. 4 NCR’s each. Certificate errors and lack of verification. 5 NCR’s. Collecting and verifying evidence. 6 NCR’s. NCR recording and proper categorization. 11 NCR’s. Witness Audit NCR’s

Registration Management Committee (RMC) Boston MA July 21-22, Agreement to cover liabilities – 1 NCR. Complaint Process – 2 NCR’s. Certification agreement, certification cycle, satellite offices – 3 NCR’s each. Surv-use of the mark evidence - 4 NCR’s Application errors, monitoring of auditors, NCR recording/category, OASIS Admin -5 NCR’s each. Office Audit NCR’s

Registration Management Committee (RMC) Boston MA July 21-22, Collecting/verifying evidence issues, financial stability evidence, records general – 6 NCR’s each. Cert decision, Impartiality Committee representation, competency of auditors, multisite – 7 NCR’s each. Audit report, NCR process – 9 NCR’s each. Conclusion consistent with report – 12 NCR’s Office Audit NCR’s

Registration Management Committee (RMC) Boston MA July 21-22, Transfers MD 2 issues – 13 NCR’s. Audit time issues and justification – 15 NCR’s. Option 2 conformance issues – 15 NCR’s Certificate issues – 26 NCR’s Office Audit NCR’s

Registration Management Committee (RMC) Boston MA July 21-22, Four Examples of actual audit trails that have been sterilized. The 4 examples represent approx 40% of the findings that occur during a witness audit. Will present the scenario then present some choices the auditor could make. Discuss the choices and select the BEST option. Time management is KEY in this exercise. Its only Lunch after this session. Case Study Examples

Registration Management Committee (RMC) Boston MA July 21-22, While auditing the purchasing process you find that there are external documents (ASTM) that are used in making decisions on what to specify for outsourced plating. The material comes back into the plant with a COA. The verbal answers to your question were all appropriate answers. What are your next steps? Example 1

Registration Management Committee (RMC) Boston MA July 21-22, A.Verify conformance of control of external documents B. Verify conformance of outsourcing C. Verify conformance with process validation and product testing as it relates to the plated product. D. all of the above. E.None of the above. Anecdotal answers were acceptable. Example 1 (best answer)

Registration Management Committee (RMC) Boston MA July 21-22, You observe a tank record of solutions analysis that has an incorrect date for change of tank. The record also had past notations of changes to the dates. Auditee says it’s a typo and changes it. You question why other information had appeared to be changed and was not consistent with what the procedure is required and are told that the company is reconsidering how they are using the form. What are you next steps? Example 2

Registration Management Committee (RMC) Boston MA July 21-22, A.Increase the sample size. B.Accept the explanation and continue the audit. C.Raise a nonconformance. D.Break early for lunch since it’s a HOT lunch today. Example 2 (best answer)

Registration Management Committee (RMC) Boston MA July 21-22, The Audit Plan identify that FAI/Stamp control/configuration management/ Evaluation of Customer Contract Review are to be audited. As the audit proceeds you realize that there would not be enough time to complete the audit for these audit trails. Your next step is? Example 3

Registration Management Committee (RMC) Boston MA July 21-22, A.Ask general questions without verifying all items on the plan in order to complete the plan. B.Just ignore the rest of the plan since they were listed to help the customer understand where you might potentially go in the audit. C.Document in the report or annotate the audit plan as a record what was not audited. D.It’s a process audit. You can make inferences to conformance with what you saw in the doc control audit. Example 3 (best answer)

Registration Management Committee (RMC) Boston MA July 21-22, During the course of the audit you record the following findings. 1.Internal Audits were not conducted on the status of importance when a process had repeat majors. 2.A Supplier CA was lost and never closed nor escalated for incorrect parts being shipped to the supplier. 3.Inaccurate root cause analysis was accepted without comment on 2 of 4 NCRs. What is your next step? Example 4

Registration Management Committee (RMC) Boston MA July 21-22, A.Note these points as OFI since the MR already knew about them. B.Identify the findings as NCR’s. C.Ignore them since you already have your limit of 10 NCR’s. D.Discuss with the company what they should do to fix them so they will not be there for the next audit. Example 4 (best answer)

Registration Management Committee (RMC) Boston MA July 21-22, If findings are NCR’s….raise NCR’s. Ensure the records are clear on what was audited and …..what was not. Pursue Trails to their natural conclusion to determine conformance…and if the trails do not demonstrate conformance…its a nonconformance. Ensure objective evidence is verifiable. Example Take Away

Registration Management Committee (RMC) Boston MA July 21-22, Questions