1. Charlie Robinson / NCAS Quality Assurance and Risk Management Services, Inc. 28-30 October 2008 Goddard Space Flight Center - Greenbelt, MD

2. PROCESS  Preparing for an Assessment RESULTS  Using Historical Data LESSONS-LEARNED  Document Control  Contracts  Software  Internal Audits SUMMARY AGENDA

3. General Information AS9100 Assignments Assessment Plan Blank Assessment Card Assessors Input Template E-mails to Team & NASA Rep. Evaluation Form In-BriefingAttendance Sheet Preparing for an Assessment The Assessment Team Takes Pre-Assessment Planning Seriously Team Conference Call

4. NCAS – 55% (18 out of 33) have Software problems (NOV 06 – SEP 08) SAC – 89% (32 out of 36) had Software problems (FEB 02 - JUN 05) 10 Suppliers have been assessed 2 to 4 times since FEB 2002 Using Historical Data

5. Using Historical Data (continued) HIGH LOW RISKRISK Average Small Medium Large University  Items produced & delivered before identifying the Environment & Contamination Requirements  Contractually required Stress Relieve Procedure not developed.  Unaware of DPAS rating  Wiring ‘Crimped” – Contract requires “Welding” Identify Requirements

6. HIGH LOW RISKRISK Average Small Medium Large University  No Configuration Management Process  Amendment to Instrument Coding Standard made without Peer-Review  Life-cycle metrics not distributed as required by contract  No Software Safety Program  Coding Standards in SDP not used.  A “Draft” Satellite Simulator Software is being used. Software Using Historical Data (continued)

7. Lessons-Learned Document Control 47 Non-compliances & 37 Observations Internal Conflicts between procedures  Matrix of procedure inter-relationships is a way to avoid these conflicts Industry Specifications not current  A consolidated listing of all industry specifications that are maintained Procedures not maintained to reflect current practices  Supervisor responsibility & Internal Audits should provide direction and assurance that procedures are current

8. Lessons-Learned Contracts 46 Non-compliances & 45 Observations There is not a reliable internal flow-down of requirements  Clean-room, DPAS, Calibration, QMS, Record Control, etc. The review is flawed as suppliers agree to requirements they do not meet.  Suppliers often do not know their basic system and therefore are not able to identify those contractual requirements that they do not satisfy. CDRL Items are either late or revised without updating the customer.  Assure that a single person is responsible for tracking the status of all CDRLs

9. Lessons-Learned Software 38 Non-compliances & 24 Observations The software controls are not all encompassing  Control of software includes deliverable, design, test, and operational software. There is not a compliant Software Safety Program (SSP)  Evaluate the SSP against NASA-Std-2100 The individual who developed the software integration are developing the integration test.  The need for objective independence is compromised by having the individual who developed the software integration also develop the integration test.

10. Lessons-Learned Internal Audits 18 Non-compliances & 14 Observations Contractual compliance and procedural support are not always part of the internal audit process.  Without auditing the contractual requirements the management will believe the Quality Management System is appropriate when in fact it does not support contractual requirements. Internal Audits are behind schedule.  This is a discipline that is necessary to avoid requirement creep or slippage. As Dwight D Eisenhower said “The uninspected inevitably deteriorate.” The internal auditors do not audit or are not qualified to audit special processes.  Assure that subject mater experts are used when auditing software, soldering, heat treat, plating, etc.

11. Summary Individual commitment to a group effort - that is what makes a  team work,  company work,  society work,  civilization work. Vince Lombardi