1. ENGINEERING LESSONS LEARNED AND SYSTEMS ENGINEERING APPLICATIONS Paul S. Gill and Danny Garcia, NASA Technical Standards Program Office William W. Vaughan, University of Alabama in Huntsville 43 rd AIAA AEROSPACE SCIENCES MEETING JANUARY 8-13, 2005

2. 2 ENGINEERING LESSONS LEARNED AND SYSTEMS ENGINEERING APPLICATIONS OUTLINE PERSPECTIVE INTRODUCTION SOME LESSONS LEARNED SOURCES SYSTEMS ENGINEERING LESSONS LEARNED EXAMPLES CONCLUDING REMARKS

3. 3 Infusion of Lessons Learned into Programs and Projects Lessons Learned Implementation Process ATP SRRPDR CDRDCR FRR Post Flight Generate and Communicate Lessons Learned Lessons Learned Databases Technical Discipline Working Group Review PROGRAMS AND PROJECTS (NPG 7120.5) IMPLEMENTATIONFORMULATION Screened And Approved Lessons SYSTEMS ENGINEERING (NPG 71xx.x and SP-6105) CONCEPT DEVELOPMENT DESIGN PHASE DEVELOPMENT & OPERATIONS Proposed Lesson

4. 4 INTRODUCTION BACKGROUND –NASA TECHNICAL STANDARDS PROGRAM INITIATIVE –INTEGRATION OF LESSONS LEARNED WITH TECHNICAL STANDARDS LESSONS LEARNED INPUTS –FACILITIES OPERATIONS –AIRCRAFT DEVELOPMENT AND OPERATIONS –ROCKET SYSTEM DEVELOPMENT AND OPERATIONS –FAILURE ANALYSES –CUSTOMER FEEDBACKS –NEW TECHNOLOGY DEVELOPMENTS BENEFITS –A VIABLE INTEGRATED AND LIFE-CYCLE BALANCED SYSTEM –IMPROVED SYSTEM MODELS AND SYSTEM VERIFICATIONS –DEFINITIONS AND REQUIREMENTS FOR DESIGN, DEVELOPMENT, AND OPERATIONS (STANDARDS, GUIDELINES, HANDBOOKS, AND SPECIFIC DESIGN REQUIREMENTS AND OPERATIONS PROCEEDURES DOCUMENTS, ETC.)

5. 5 SOME SYSTEMS ENGINEERING LESSONS LEARNED SOURCES AIAA Satellite Mission Operations Best Practices Lessons Learned From The Tropical Rainfall Measurement Mission (TRMM) Mechanical And Deployable Subsystems Lessons from the Shuttle Independent Assessment Space Engineering Lessons Learned Chandra Lessons Learned Systems Engineering Office Lessons Learned Flight Projects Lessons Learned Database Johnson Space Center Lessons Learned Database NASA Lessons Learned Information System (LLIS) Electronic Systems Branch Design Handbook Items Lessons Learned A History of Aerospace Problems, Their Solutions, Their Lessons Working on the Boundaries: Philosophies and Practices of the Design Process

6. 6 SYSTEMS ENGINEERING LESSONS LEARNED EXAMPLES Establish A Systems Engineering And Integration Role With The Authority To Integrate All Program Technical Elements, Organizations, And Other Affected Parties. (OSP Topic 4) Rigorous Requirements Development Processes Consistent With Accepted Systems Engineering Practices Are Critical To Establishing A Good Requirements Foundation For A Program/Project. (OSP Topic 6) Ensure A Sufficient And Experienced Systems Engineering Staff Is Available And Equipped With Effective Tools And Well-documented Processes To Lead Requirements Development. For Each Requirement, A Corresponding Rational And Verification Approach Should Be Maintained. (X-37 LL) The Systems Engineering Process Must Adequately Address Transition From Development To Operations. Develop And Execute Systems Verification Matrix For All Requirements. (LLIS 0641) Inadequate Systems Engineering Process And Oversight Is Especially Dangerous In Combination With Nonstandard Engineering Practices. (LLIS 1385)

7. 7 The Systems Engineering Plan Should Be Baselined In The Mission Definition Phase Of A Program To Avoid Unclear Direction Of How To Handle Milestone Event Success Criteria. (OSP 12.0001) All Systems Engineering Functions Need To Be Located In A Centralized Systems Engineering Organization With Well Defined Roles And Interface Responsibilities For The Various Systems Engineering Functions. (OSP 12.0002) It Is Important To Have Systems Engineering Follow An Established Systems Engineering Guideline And Maintain A Strong Capability And Presence In A Program. For Example, It Is Imperative That A Program-level Systems Engineering Management Plan Be Established And Distributed To All Lower Level Projects. (Exploration Systems Workshop) Human Factors Engineering Needs To Be Treated As A Sub-Discipline Within Systems Engineering And Integration And Be Included In The Development Of The Work Breakdown Structure For The Program. (Exploration Systems Workshop) Define Procedures To Ensure Effective Systems Engineering On Safety Issues And Institute Cost Control System For Safety And Reliability. (LLIS 0495) SYSTEMS ENGINEERING LESSONS LEARNED EXAMPLES (Con’t)

8. 8 THE LESSONS LEARNED PROCESS BEST PRACTICES Obtain Enterprise-Wide Commitment To A Formal Lessons Learned Process. Review And Prioritize Candidate Lessons Based On Their Applicability To Current And Future Projects, And Periodically Review Published Lessons For Their Continued Relevance. Do Not Limit Lesson Topics To “Screw-Ups;” Also Document Successes That Should Be Replicated. Charter A Lessons Learned Committee That Represents The Major Technical Organizations And is Charged With Real-Time Development Of Draft Lessons. Designate A Single Author To Interview Sources And Prepare Lesson Drafts. The Recommendations Drawn From The Driving Event Should Be Actionable. Innovate To Assure That The Lessons Learned System Is A Dynamic Resource--Not a “Data Morgue.” *Source: The Lessons Learned Process: An Effective Countermeasure Against Avoidable Risk. J. Clawson and D. Oberhettinger. 2001 Proceedings of the Reliability and Maintainability Symposium, pages 94-97.

9. 9 ENGINEERING LESSONS LEARNED AND SYSTEMS ENGINEERING APPLICATIONS CONCLUDING REMARKS THE MODERN ERA –HIGH MOBILITY IN TECHNICAL WORK FORCE –”ONE DEEP” TECHNICAL CAPABILITIES –MINIMAL MENTORING LESSONS LEARNED RECORDING AND TRACKING IMPORTANCE –MAINTENANCE AND IMPROVEMENT OF KNOWLEDGE –COUNTERMEASURE AGAINST AVOIDABLE RISKS –PUBLICATION AND INFUSION INTO THE ENGINEERING PROCESS ASSEMBLY OF LESSONS LEARNED DATA SETS –NASA TECHNICAL STANDARDS PROGRAM WEBSITE –RELATING TO CURRENT AND NEW AEROSPACE VEHICLE DEVELOPMENTS

10. 10

11. 11 BACK-UP INFORMATION 1.ECSS-E-10A “SPACE ENGINEERING—SYSTEMS ENGINEERING” (APRIL 17, 1966) 2.ECSS-E-10 PART 6A “SPACE ENGINEERNG—SYSTEMS ENGINEERNG PART 6: FUNCTIONAL AND TECHNICAL SPECIFICATIONS “ (JANUARY 9, 2004) 3.NASA SP-6105 “NASA SYSTEMS ENGINEERING HANDBOOK” (JUNE 1995) 4.NASA LESSONS LEARNED INFORMATION SYSTEM (LLIS) http://llis.nasa.govhttp://llis.nasa.gov 5.LESSONS LEARNED/BEST PRACTICES DATABASES http://standards.nasa.gov http://standards.nasa.gov ECSS: European Cooperation for Space Standardization

12. 12 Programs and Projects Link To Requirements Infuse Into Engineering Process Link the Lessons Learned to System Development Lifecycle Requirements Infuse the Lessons Learned Into the Engineering Process Establish Feedback System Feedbacks Needed to Provide User Based Enhancements COMMUNICATE KNOWLEDGE Generate & Identify Document Categorize Characterize Evaluate Generate & Identify Lessons Learned Document Lessons Learned Categorize Into Discipline Categories Characterize into Programmatic Versus Technical Lessons Learned Validity is Determined by the Working Group GENERATE KNOWLEDGE Lessons Learned Closed-Loop Process