1. Company Confidential Registration Management Committee 1 Lessons Learned from Accreditation Assessments - ANAB Perspective July 19, 2012 Steve Holladay ANAB, Accreditation Assessor

2. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee Objectives Provide a summary of the NCR’s identified from the witness audits 2011. Provide a summary of the NCR’s identified from the office audit in 2011. Discuss four examples of the most common NCR’s. Rules of engagement. 2

3. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee Goal Provide knowledge/information to the AQMS auditor pool –with the goal to improve auditor/CB performance –that will reduce repeat NCR’s –which will, in turn, add value to the assessment process and the industry as a whole. 3

4. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 4 Overall 2010 61 NCR’s in 40 Witnessed Audits (WA) –1.5 Average per WA –22 Major NCR’s 162 NCR’s in approx 42 Office Assessments (OA) –3.85 Average per OA –21 Major NCRs –22 AS requirements 2011 72 NCR’s in 53 Witnessed Audits (WA) –26 to AQMS requirements –23 Major NCR’s (11 AQMS) 238 NCR’s in approx 46 Office Assessments (OA) –52 to AQMS –37 Major NCRs (13 AQMS)

5. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 5 Audit team competency, Scope/objectives, stage 1 objectives, others. 1 NCR each. CA Process with Client, Min onsite activity. 2 NCR’s each. OASIS Issues, Audit Planning and OER issues. 3 NCR’s each Witness Audit NCR’s

6. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 6 Reaching correct conclusion. 6 NCR’s. Audit Time, Certificate issues and Nonconformance Process. 8 NCR’s each. Collecting and verifying evidence. 9 NCR’s. Witness Audit NCR’s

7. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 7 Surv/recert issues– 1 NCR. Nonconformance Process – 3 NCR’s. Right of Access instructions – 5 NCR’s. Multisite issues and Cert Issues - 4 NCR’s each. No satellite Office issues -5 NCR’s. Conclusion/Reporting consistency issues – 9 NCR’s. Office Audit NCR’s (9104; 32 total)

8. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 8 Pre Audit Planning issues – 3 NCR’s. Nonconforming Reporting issues – 3 NCR’s. Audit report, appendix – 14 NCR’s. Office Audit NCR’s (9101; 20 total)

9. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 9 Four Examples of actual audit trails that have been sanitized. Focused the case studies on the requirements of AS9101D and AS9104/1. Will present the scenario then present some choices the auditor could make. Discuss the choices and select the BEST option. Time management is KEY in this exercise. Lunch will be waiting. Case Study Examples

10. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 10 While preparing for an audit you review the ASD information provided in the pre-audit package. Aerospace: 70% (3 customers) Space: 10% (1 customer) Defense: 20% ( 1 customer) What are your next steps? Example 1

11. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 11 A.Don’t use it. This is a surv audit and doesn’t apply. B. Consider the information during the on-site audit for the audit trail selection. C. Ask what the breakdown of the 3 aerospace customers are to better plan the audit time. D. Plan the time based on the information provided then determine what customers to review once on site. Example 1

12. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 12 You are preparing for an AS9100C audit and have all the required information from the client. You start working on appendix D (Process Matrix Report) and appendix C (PEAR). Which of the following options are acceptable? Example 2

13. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 13 A.Review the previous audit (appendix D) to see what processes were defined then. B.Forward the (appendix D) to the customer and ask them to have it completed before the audit begins. C.Ensure all process (D) that are marked with a section 7 clause has a corresponding PEAR. D.Review the clients Quality Manual to ensure the processes listed in (D) are consistent. Example 2

14. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 14 You are starting a review of the corrective action response to a NCR that was raised at the clients last audit. The NCR summary is: Test equipment (TE) was observed being used in a required inspection process. The TE was found to be out of calibration. Which of the following responses would be acceptable as presented? Example 3

15. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 15 A.Correction: TE was calibrated. B.Root Cause: The TE was new. C.Correction: None needed, the TE was remarked as for reference only. D.Root cause: None needed, investigation found that this was the only TE out of calibration. E.Correction: Told TE manager not to do it again. F.Root Cause: The TE was out of calibration. Example 3

16. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 16 You have just completed an AS9120 audit and are revisiting the certification scope to ensure the audit can support the scope. In your audit you observe the following: The org purchases used aircraft parts and disassembles them and sends some parts out for repair and when returned, stock them for re-sale. They select the repair station based on the repair station FAA designated capabilities. What is your next step? Example 4

17. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 17 A.The activities are acceptable for AS9120. B.Its Questionable since they select the supplier. C.Not suitable for AS9120 since they disassemble. D.Other scenarios. Example 4

18. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 18 Be critical of the review of CA responses. Ensure the records clearly show ASD pre- audit information was utilized in all audits. Ensure the process identified in the 9101 are consistent with the clients manual/process definitions. If the 9120 client does anything other that stocking/distributing…..ask the questions. Example Take Away

19. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee Other Take away Points Make sure decision making person/people are competent in 9101D. Make sure required 9101 documentation is fully completed. Always consider product containment while reviewing NCRs. 19

20. RMC Workshop Minneapolis, MN 19 – 20 July 2012 Registration Management Committee 20 Questions