1. Americas Aerospace Quality Group AAQG Project Management AAQG Project #61 (IAQG REQ-009) IAQG 9101:2009 Revision - Quality Management System Assessment Reported by: Tony Marino Date of Report: 2009-March-11

2. What happened and where is 9101? Released in October 2008 for ballotReleased in October 2008 for ballot –AAQG writing team members did not agree it was ready. –Consulted with AAQG leadership to release or move for sector rejection – decision for sector rejection. Primary IssuesPrimary Issues –Writing was a balanced writing addressing both Conformity and Effectiveness. –Writing structure – too confusing. –Contained guidance material. –Differences of opinion on what could be changed during and after ballot. Document revisions have been made and process of finalization for ballot.Document revisions have been made and process of finalization for ballot.

3. IAQG REQ-009 Working Group Members Hans Luijt – Team LeadASD-EASE - Netherlands Stuart AnthonyRolls-Royce UK Eddy BenattarIAI - Israel Alain BonnelSAFRAN - France Conny FlinkVolvo Aero - Sweden Michel JacquanielloZodiac - France Marie-Laurence PetitAirbus - France Brian Burroughs*LRQA – UK Michel Clément* Bureau Veritas - France Larry BeckLockheed Martin – US (TX) Tom GallagherHamilton Sundstrand (UTC)-US Tony MarinoThe Boeing Company – US (WA) Robert Keys*DNV – US (TX) Paul Dionne*Intertek Masahiro Kawamoto Mitsubishi Heavy Industries – Japan

4. 9101: Revision Objectives Ensure alignment with IAQG strategy (on-time, on-quality performance, Strengthen of Other Party certification)Ensure alignment with IAQG strategy (on-time, on-quality performance, Strengthen of Other Party certification) Support the process approach for quality management systems as described in the 9100-series standardsSupport the process approach for quality management systems as described in the 9100-series standards Incorporate ISO 17021 changes and ICOP resolutionsIncorporate ISO 17021 changes and ICOP resolutions Define requirements for all activities of audit programDefine requirements for all activities of audit program Define adequate audit report that provides useful information to stakeholders on Organization Processes Performance And Effectiveness ( PEAR ) and conformity (Objective Evidence Record) to 9100 series.Define adequate audit report that provides useful information to stakeholders on Organization Processes Performance And Effectiveness ( PEAR ) and conformity (Objective Evidence Record) to 9100 series. Define audit records (objective evidence,…) to support ICOP SchemeDefine audit records (objective evidence,…) to support ICOP Scheme

5. Date of project approval:Project Plate Revision: Solution: Root Causes: Metrics: Deliverables: Success Criteria: Americas Aerospace Quality Group - Project Plate AAQG Project Leader: Champion: Problem Statement and Scope: AAQG P61 - IAQG 9101:2008 Revision - Quality Management System Assessment 2006-Oct-05)6/20/08 Tony Marino Jack Flecher IAQG Gate 0 Approved October 5, 2006 - Seville Spain (REQ-009) 1. 1.Current 9101 does not focus 9100 assessments on process based management 2. 2.Too much emphasis on conformity of assessment – not auditing what is important i.e. performance, continuous improvement, customer satisfaction 3. 3.Linkage to certification: certificate too much of a ‘drivers license, instead of a tool to prevent ‘accidents’. 4. 4.Scoring is complex, time consuming, subjective and purpose of is unclear. 5. 5.Strong relationship of assessment documents (9101, 9111 and 9121). 1. 1.Upon release of ISO9001:2000, assessment tool and methods for 9100 was not updated to assess the process approach. 2. 2.New 9100 release in April 2009 will require update of 9101 1. 1.Update 9101 to be more than an assessment (checklist) document. 2. 2.Publish assessment methods and tools that will provide more meaningful information to the supplier, client and stakeholders 3. 3.Consolidate 9101, 9111 and 9121 into one document 1. 1.Revised 9101 2. 2.Develop position and decision to obsolete 9111 and 9121 3. 3.Aerospace Experienced Auditor training and certification/ recertification recommendations 1. 1.Publication of 9101 2. 2.Auditor efficiency increase 3. 3.Greater fidelity of information in audit reports 1. 1.Project Milestone

6. Project description / benefits: AAQG Champion:Date of last report:Date of this report:Status: Unresolved issues & decisions (if any): Recent accomplishments (since last report): Next steps (30-60-90 day look ahead): Project Leader: Americas Aerospace Quality Group - Project 4-Square P61 - IAQG 9101:2008 Revision - Quality Management System Assessment Kristy Heffernan2008-Sept, 112009-March, 11 Tony Marino Y AAQG SWOT and sector rejection of ballot document Nov 23, 2008. Training specification and competency requirements development Nov 17-20, 2008 – Paris Team F2F to address issues and actions. Huntington Beach CA. Jan. 2009. Proposed 2 nd ballot draft complete Feb 22, 2009. Team comments submitted updated draft. Feb 24, 2009 Completed comment incorporation – March 10, 2009 1. 1.AAQG Briefing F2F Portland. March 10-12, 2009 2. 2.IAQG Release of reballot. March 11, 2009 3. 3.AAQG Release of 2 nd ballot. March 20, 2009 4. 4.AAQG Ballot cycle. March 20-April 20, 2009 5. 5.EAQG Ballot cycle. March 20-May 20, 2009 6. 6.APQG Ballot cycle. March 20-May 20, 2009 7. 7.Target Release July 2009 Project Description Restructure 9101 with the next revision of 9100 to combine the 9101/9111/9121 standards to reflect a process based auditing approach of 9100, 9110 and 9120 standards. Auditor training and competency requirements development. Benefit of project Alignment of assessment tools and methods to process based management QMS. Shift audit focus to process, product, system performance and customer satisfaction. Reduced variation of training and training expectations. None – Sector rejection by the AAQG and member rejection by EAQG – Germany, triggered a revision to the document and 2 nd ballot process. Schedule for 2 nd ballot complete.

7. Development Model for a Process-Based 9101 Audit Records Tools Process Approach Process-Based Auditing InputsOutput Core, Support & Management Processes Effectiveness Focused Evidence of Conformance Evidence of Effectiveness

8. 9101: Major Changes Review  Introduction in the audit report of: an “Process Effectiveness Assessment Record” form (PEAR) which : an “Process Effectiveness Assessment Record” form (PEAR) which : State if for the audited process has been determined that State if for the audited process has been determined that – planned activities and results are realized – planned activities are realized but planned results are not realized – planned activities and results are not realized Give objective evidence, observed conditions, data, and comments to support effectiveness determination Give objective evidence, observed conditions, data, and comments to support effectiveness determination a process based “Audit Completeness Record” a process based “Audit Completeness Record” an “Objective Evidence Record” capturing quality manual and procedures with objective evidence of conformity an “Objective Evidence Record” capturing quality manual and procedures with objective evidence of conformity

9. 9101: What has not changed Determining conformance to the standard.Determining conformance to the standard. Nonconformance discovered during the audit are to be documented and must be closed before certification.Nonconformance discovered during the audit are to be documented and must be closed before certification. Based on information collected during the assessment, the auditor will have to draw conclusions on conformance of the organization quality management system. Note: Conclusions on effectiveness are newBased on information collected during the assessment, the auditor will have to draw conclusions on conformance of the organization quality management system. Note: Conclusions on effectiveness are new

10. Issue of a coordination draft : 30 June 2008 (Limit date for comments : 7 July 2008)Issue of a coordination draft : 30 June 2008 (Limit date for comments : 7 July 2008) Review of the comments (8-11 July in Amsterdam)Review of the comments (8-11 July in Amsterdam) –450 comments received, 233 accepted, 15 partially accepted 126 comments reviewed and disposition from ballot rejection (January 9-12, 2009 in Huntington Beach126 comments reviewed and disposition from ballot rejection (January 9-12, 2009 in Huntington Beach Publication of the 9101 Formal ballot : Sept./Oct 2008Publication of the 9101 Formal ballot : Sept./Oct 2008 Disposition on comments : end Nov. 2008 – RejectedDisposition on comments : end Nov. 2008 – Rejected Comments addressed – January 2009Comments addressed – January 2009 Preparing for ballot release – March 2009Preparing for ballot release – March 2009 9101 Formal Ballot 9101: Progress Status 9101 Release Target Release – July, 2009

11. 9101 Status Update Ballot version published 26 September 2008Ballot version published 26 September 2008 Results of ballot:Results of ballot: – APQAG : passed, with comments – AAQG: rejected, with comments – EAQG: France: passed, as isFrance: passed, as is Germany: rejected, with commentsGermany: rejected, with comments Italy: passed, as isItaly: passed, as is UK: passed, as isUK: passed, as is Switzerland: abstention, no expertsSwitzerland: abstention, no experts Sweden (VAC): passed, with commentsSweden (VAC): passed, with comments 9101 team conclusion: address the comments and go for 2nd ballot9101 team conclusion: address the comments and go for 2nd ballot

12. Feedback summary  126 detailed comments received  Major comments: – Checklist / record for objective evidence is missing (US, Germany, Sweden) – Structure of the document needs improvement (AAQG) – During OP assessment NO attention should be given to customer specific QMS requirements (Germany) – PEAR form and approach is too complex, e.g.‘all processes’  9101 team proposals: – To add record for objective evidence – To restructure part of the standard: All methodology / approach issues will be combined in one chapter All methodology / approach issues will be combined in one chapter Recommended practices will be in a separate document (ARP, TR?) Recommended practices will be in a separate document (ARP, TR?) – Use of PEAR form limited to Product Realization Processes only

13. Dispositions 126 comments received126 comments received Team meeting in Huntington Beach 10-12 February, 2009Team meeting in Huntington Beach 10-12 February, 2009 Disposition statistics:Disposition statistics: – Accepted: 59 – Partially accepted: 17 – Rejected: 39 – Deferred: 8 – Noted: 3

15. 9101 Release Schedule 01/09 04/09 07/09 Maximum 30 month Implementation from Publication IAQG Phoenix 9100:2009 Publication New 9101 Publication 9101 Ballot start 03/09 02/09 Notes: All changes in 2nd version are related to comments No new requirements, except use of Objective Evidence Record as checklist Request: to reduce 2 nd ballot in Europe to 2 months

16. Questions Comments/Feedback to: anthony.p.marino@boeing.com

17. Back Up

18. AS9101 Process Audit Flow From 3.0 Complete appendix # 6 Customer Quality Data (not applicable for this audit) 5.0 Complete appendix # 1 Identify clauses and processes 6.0 Complete appendix # 5 Stage 1 Audit 8.0 Conduct Interviews & follow Audit Plan 9.0 Complete appendix # 4 Summary 10.0 Update appendix # 1 12.0 Complete appendix # 7 EAR Form 14.0 Complete appendix # 3 Audit Report Audit Lead Auditors 11.0 Complete appendix # 2 Write NCRs Stage 1 Stage 2 Conclusions & Reporting 2.0 Conduct Planning Interview Pre-audit Planning 7.0 Develop micro audit plan 1.0 Conduct Process and Business Research 13.0 Convene team for Consolidation / roll up (at lead discretion) 4.0 Data Collection 8/20/08 – 9/3/08 Due 9/3/08

19. Beta Test Results – 1 st Party View Review and analysis of AS9101 Coordination DraftReview and analysis of AS9101 Coordination Draft Conclusion made by the auditors was that the approach is good. Our auditors are trained in the process approach and use the ISO 9001 Auditing Practices Group guidance documents.Conclusion made by the auditors was that the approach is good. Our auditors are trained in the process approach and use the ISO 9001 Auditing Practices Group guidance documents. All elements of a process audit were captured and defined but lacked sequencing of the activities.All elements of a process audit were captured and defined but lacked sequencing of the activities. Process flow was created.Process flow was created. Audit team formed and training on the process was conducted.Audit team formed and training on the process was conducted.

20. AAQG Project #61 (IAQG REQ-009) Scope Amend 9101 Assessment standard in order to enhance its clarity and compatibility with process based management QMS (ISO 9001) and IAQG 9100 Standard update to be published mid 2009

21. AAQG Project #61 (IAQG REQ-009) Purpose of Revision Focus a methodological approach of quality systems audits Enhance clarity of standard to resolve issues on how to use the standard Keep a standard report format Consider all stakeholder needs Ensure that the audit approach is more process oriented Extend the scope of the audit process not only to determine conformity but to include the assessment of the organization’s ability to meet customer, regulatory and the organization’s own requirements

22. Meeting 13-15 November 2007 Toulouse Summary of discussions/decisions Scoring will no longer be required under the new 9101 Audit Process Matrix list with future 9101 topics agreed Inputs for new 9101 determined: Audit Process Matrix List ISO TC 176 APG material Stakeholder inputs Automotive IATF material for for auditing ISO/TS 16949 Requirements flow down chapter from SCMH Referenced documents determined: ISO/IEC 17021 Part 1 and 2 (if available) ISO 19011 IAQG 9104 IAQG 9100/9110/9120 Relation with new 9101 structure and referenced documents ‘mapped’ Detailed writing rules for drafting new 9101 were agreed Writing tasks assigned to sub-teamsWriting tasks assigned to sub-teams Schedule revised, now aligned with 9100 scheduleSchedule revised, now aligned with 9100 schedule

23. Meeting 12-13 February 2008 Orlando Summary of discussions/decisions Sub-team report out Assembled working draft Working Draft Review Some requirements had direct relationship to AS9104 rewrite Developed list of input for Berlin OPMT meeting Team size is becoming a factor. Agreed after Naples working session, team would be scoped down increase velocity of development. Output of smaller team would come back to original team.