1. Protecting the Public, Astronauts and Pilots, the NASA Workforce, and High-Value Equipment and Property Mission Success Starts With Safety The Future Role of Quality in Aerospace: Improving the Chances for Mission Success International Aerospace Quality Group 3 rd IAQG General Assembly Meeting Kyoto, Japan March 14, 2002 Dr. Michael A. Greenfield Acting Associate Administrator Office of Safety and Mission Assurance (1)

2. Mission Success Starts With Safety 03/14/2002 (2) The Big Picture – What NASA Must Do NASA’s programs must contribute to national priorities: – Economic growth and security. – Increased understanding of science and technology. – Protection of Earth’s environment. – Educational excellence. – Peaceful exploration and discovery. NASA’s programs must continue to build on the Agency’s core foundation for aeronautics and aerospace research and development. NASA must demonstrate improvement in business practices.

3. Mission Success Starts With Safety 03/14/2002 (3) Our Current Environment Stakeholder Expectations Programs that are on time, within budget, and are safe/successful. High priority planetary exploration, climate research, space station experimentation, and biological science that is effective. New technology for improved access to space, and more capable planetary missions. Outcomes that are meaningful to the public. Business Realities Flat budgets.  Workforce retirement, loss of critical skills.  Aging infrastructure.  Cost growth.  Increased reliance on the performance of the supply base.

4. Mission Success Starts With Safety 03/14/2002 (4) To succeed, processes must be stable, capable, and in control.

5. Mission Success Starts With Safety 03/14/2002 (5) Defenses Are Never Perfect Source: James Reason, Managing the Risks of Organizational Accidents, 1997, p. 9 But the reality can look like this. Hazard We perceive our ideal system of defenses like this.

6. Mission Success Starts With Safety 03/14/2002 (6) Requirements Design Process Development When Events Lines Up Just Right, the Consequences Can Be Devastating Adapted from : James Reason, Managing the Risks of Organizational Accidents, 1997, p. 12 Mishap Manufacturing Test Operations Hazar d

7. Mission Success Starts With Safety 03/14/2002 (7) Requirements Design Success Depends on Proactive Quality Efforts Adapted from : James Reason, Managing the Risks of Organizational Accidents, 1997, p. 12 We need to understand that our current defenses are not invincible. We need to proactively build in ways to “close the holes.” Quality-based tools are the best bet. – Requirements Definition – Process Control – Surveillance – Independent Analysis – Peer Review – Other Quality-based tools have high leverage. Product inspections have limited impact. They address the “symptoms.”

8. Mission Success Starts With Safety 03/14/2002 (8) NASA’s Quality Approach Performance Based Risk Based Industry Based 2 nd, 3 rd, and other EEE Parts Risk Mgmt. Vendor Status Problem Advisory Maintain Awareness Application Information Testing Risk based supplier assessment by technology area + Industry Partnership (Quality Leadership Forum, Industry Associations) Outreach to Supply Base (Awareness, Tools, Best Practices, Forums, Conferences) Standards/Clauses (AS 9100) Tailored Quality Requirements Improve Acquisition Capability Maturity Models Process Control Continual Improvement Improve Performance Improve Oversight Address High Risk Suppliers Address High Risk Parts Address High Risk Processes + ++

9. Mission Success Starts With Safety 03/14/2002 (9) Integrated Risk Approach Quality/Procurement/Program Management Acquisition PlanningSolicitation and AwardPerformance Work Analysis Develop WBS Analyze risk of work Performance Analysis Select indicators, standards Determine acceptable quality level Analyze cost/quality tradeoffs Risk Assessment Use historical data, experts, brainstorming, checklists Identify, analyze, quantify: risks, triggers, probability, impact, task linkage, surveillance method Identify stakeholders Develop Draft SOW Solicitation Release Draft SOW Analyze industry comments Develop acquisition strategy Issue final RFP Proposal Evaluation Assess risk for each proposal Contract Negotiation Surveillance method and frequency Performance metrics Program management plan Risk allocation Risk Management Includes Surveillance/ Communication Contract Award Surveillance Monitor contractor performance Conduct risk based assessments Adjust surveillance and performance metrics based on changes in performance and risk Communication Manage communications per Surveillance, Communication and Risk Management Project Risk Management Monitor Track Adjust

10. Mission Success Starts With Safety 03/14/2002 (10) Consistent Requirements Tailored to Risk For example “Limited Life” A proliferation of different program/contract specific words used to express the same requirement Single set of recommended words Guidance to assure the requirement is applied in risk areas Will Become Today

11. Mission Success Starts With Safety 03/14/2002 (11) Trades Aren’t Simple: Risk Versus Total Cost of COTS

12. Mission Success Starts With Safety 03/14/2002 (12) To Improve Quality, Management Must Also Improve Establish Accountability – Industry based requirements – Performance based contracts – Industry self oversight Emphasize Basics – Addressing systemic issues Demand Evidence – Oversight based on risk Impose Consequences for Poor Performance – Award fee, incentive fee, future business Focus on Results – Mission success criteria Mission Checklist Clear Expectations Clear Accountability Clear Results

13. Mission Success Starts With Safety 03/14/2002 (13) What’s the Bottom Line? There’s a lot at stake: – Human life. – One of a kind hardware. – Scientific knowledge. – International cooperation in space. – Economic viability. NASA must focus on its core competencies – R&D. Quality is critical to our ability to meet our ambitious goals.

14. Mission Success Starts With Safety 03/14/2002 (14) The Future State of Quality The future state of quality will have these elements: Quality requirements and expectations that are proportional to the risk of the activity and its expected outcome. A supplier oversight strategy that is integral to comprehensive program management (cost, schedule, and technical). Emphasis on high risk areas such as EEE parts, printed wiring board production, and other similar space technology/ environment concerns. With clear requirements, focused oversight, and proper emphasis on risk, we increase the chances for mission success, and everybody wins.