1. Learn. Perform. Succeed. LOG 203 (R&M) - LOG 103 (RAM) Migration Prepared for the Life Cycle Logistics (LCL) Functional Integrated Product Team (FIPT) Presented by Patrick M. Dallosta, CPL Performance Learning Director LMI Government Services McLean VA 15 January, 2010 1

2. Learn. Perform. Succeed. Agenda Change Drivers LOG 203 Status LOG 103 Content Migration Plan 2

3. Learn. Perform. Succeed. Change Drivers Recent Legislation/updated DoD Policy has significantly strengthened DOD Reliability, Availability and Maintainability (RAM) Policy, Systems Engineering/Reliability/Test & Evaluation processes and organizations. Life Cycle Logistics DAWIA certification path reflects new course requirements in the area of RAM and Supportability –LOG 103, “Reliability, Availability and Maintainability” Upgrading of existing LOG 203 –LOG 211, “Supportability Analysis” New curriculum development Expect great cross-functional interest as RAM / Supportability is reconstituted in program offices – a “pendulum swing” back to government resulting from the aftermath of Acquisition Reform 3

4. Learn. Perform. Succeed. Application/case based GS 9-12 & E7-O4 Level II Certification LOG 201 Intermediate Acquisition Logistics LOG 201 Intermediate Acquisition Logistics 35 hours, online LOG 200 Intermediate Acquisition Logistics LOG 200 Intermediate Acquisition Logistics LOG 235 Performance Based Life Cycle Product Support LOG 235 Performance Based Life Cycle Product Support 35 hours on-line 4.5 days classroom P LOG 211 Supportability Analysis LOG 211 Supportability Analysis 4.5 days classroom ACQ 202 IntermediateSystemsAcquisition IntermediateSystemsAcquisition P ACQ 203 IntermediateSystemsAcquisition IntermediateSystemsAcquisition 4.5 days classroom LOG 206 Intermediate Systems Sustainment LOG 206 Intermediate Systems Sustainment 40 hours, online NOTE: There would still be NO prerequisites for LOG235 xx hours on-line P Case/scenario based GS 13-14 & E7-O5 Level III Certification P P LOG 350 Enterprise Life Cycle Logistics Management LOG 350 Enterprise Life Cycle Logistics Management LOG 340 Performance Based Life Cycle Sustainment LOG 340 Performance Based Life Cycle Sustainment 8.5 days classroom Level II “Core Plus” Courses & CL Modules (Includes LOG 204 CM Course & new LOG 2xx Tech Data Mgt Course ) Level II “Core Plus” Courses & CL Modules (Includes LOG 204 CM Course & new LOG 2xx Tech Data Mgt Course ) P 25 hrs, on-line Level I Certification ACQ 101 Fundamentals of Systems Acquisition Management ACQ 101 Fundamentals of Systems Acquisition Management Knowledge based GS 5-9 & E7-O3 26 hrs, on-line 30 hrs, on-line P 2 Continuous Learning Modules: PBL & Designing for Supportability 2 Continuous Learning Modules: PBL & Designing for Supportability LOG 103 Reliability, Availability & Maintainability LOG 103 Reliability, Availability & Maintainability LOG 102 Systems Sustainment Management LOG 102 Systems Sustainment Management 1-3 hrs ea, on-line LOG 101 Acquisition Logistics Fundamentals LOG 101 Acquisition Logistics Fundamentals 28 hrs, on-line P = Prerequisite Level I “Core Plus” Courses & CL Modules (See DAU catalog for details) Level I “Core Plus” Courses & CL Modules (See DAU catalog for details) Level III “Core Plus” Courses & CL Modules (See DAU catalog for details) Level III “Core Plus” Courses & CL Modules (See DAU catalog for details) FY13 LCL Certification Construct P New Cert Courses 4

5. Learn. Perform. Succeed. LOG 203 Course Objective Understand the relationship between Acquisition Logistics and Reliability & Maintainability (R&M) Evaluate the impact of R&M decisions Knowledge of basic R&M concepts, terminology engineering practices and math Discussion of Practical applications 5

6. Learn. Perform. Succeed. Crosswalk to LCL Competencies Logistics Design Influence LOG 101, LOG 103 (203), LOG 200, LOG 201, LOG 350 Integrated Logistics Support (ILS) Planning LOG 101, LOG 200, LOG 201, LOG 210, LOG 350 Product Support & Sustainment LOG 102, LOG 206, LOG 235, LOG 236, LOG 350 Configuration Management LOG 204, LOG 206 Reliability & Maintainability Analysis LOG 103 (203), LOG 200, LOG 201 Technical/Product Data Management LOG 204, LOG 206 Supportability Analysis LOG 103(203), LOG 200, LOG 201, LOG 210 6

7. Learn. Perform. Succeed. LOG 203 Overview Assignment specific course for Reliability & Maintainability No prerequisites Distance learning format / Only 1 Section of LOG 203 LOG 203 metrics consistently meets performance expectations: 3-fold increase in student throughput since 2006 Key Personnel: –Performance Learning Director: Tom Simcik, LCIC-LOG –Course Manager: Richard DiLorenzo, Mid-West –Course Manager Regional Reps :Hal Ernest, South Phil Littrell, Mid-Atlantic Great Job, Tom and Team!! 7

8. Learn. Perform. Succeed. Question/ Categories FY06FY07FY08FY09 Courseware n =310 n =577 n =730 n =382 Online Delivery n =310 n =578 n =731 n =382 Learning Effectiveness n =304 n =571 n =729 n =380 Job Impact n =309 n =577 n =731 n =384 5.95 6.02 5.85 5.74 6.00 6.08 5.89 5.64 * Data (from Data Mart) as of 31 Dec 2010 6.12 6.18 5.99 5.91 5.89 5.98 5.82 5.74 LOG 203 Metrics & Demographics No FY 10 Data 8

9. Learn. Perform. Succeed. LOG 203 Metrics & Demographics (cont.) LOG 203FY 06FY 07FY 08FY 09 FY 10* (Actual/Projected) Graduates4397249261140306 / 1244 Army11423541044097 / 338 Air Force7913115421358 / 232 DoN12313517521261 / 244 Industry9118513718462 / 248 Other DoD1930326019 / 76 All Others13818319 / 36 * as of 14 Jan 2010 9

10. Learn. Perform. Succeed. 1.Introduction to LOG 103, Reliability, Availability & Maintainability 2.Historical Roots of Reliability, Availability & Maintainability 3.Impact of Acquisition Reform on Reliability, Availability & Maintainability 4.Mandatory Sustainment Metrics KPP/KSAs 5.Legislative/Policy Reliability, Availability & Maintainability Initiatives 6.The Systems Design Operational Model (SDOE) 7.The Systems Engineering – Design Interface 8.Software Reliability, Availability & Maintainability 9.Designing and Enabling Reliability, Availability & Maintainability 10.Modeling & Simulation of Reliability, Availability & Maintainability 11.Supportability Elements, Practices and Outputs 12.Impact of RAM on Mission Success and Logistics 13.Translating Operational Requirements to RAM Requirements 14.RAM Growth Programs, Test and Evaluation and Risk Reduction 15.Impact of R&M on Manufacturing Current and Proposed LOG 103 Lessons 10

11. Learn. Perform. Succeed. Migration Path Ahead On-going LOG 201-Log 103 Conversion Actions –Convert current LOG 203 to DAU standard application software –Update basic course information, e.g., re-title LOG 103 –Deployment of LG 103 Certification Level Course15 Mar 10 Additional LOG 103 Development –New LOG 103 competencies/proficiencies identified 30 Apr 10 –Draft curriculum developed 31 Jul 10 –Draft curriculum review 31 Aug 10 –Curriculum finalized 30 Sep 10 –Programming /Test31 Dec 10 –Instructor Pilot/Student Validation (Jan 11)31 Jan 11 –Deployment 15 Mar 11 11

12. Learn. Perform. Succeed. Questions & Answers LOG 203 – LOG 103 Migration Thank you for the Opportunity to Serve the Life Cycle Logistics (LCL) Functional Integrated Product Team (FIPT) 12

13. Learn. Perform. Succeed. Back-up!!!

14. Learn. Perform. Succeed. Proposed LOG 103 Learning Objectives/Outcomes Lesson 1: Impacts of R&M on Logistics –Recognize how R&M affects support costs –Recognize how R&M affects Total Ownership Costs –List 4 key elements in the definition of reliability –List 2 MTB_s that are logistics reliability parameters Lesson 2: Impacts of R&M on Missions –Distinguish between latent failures, random failures and wearout –Identify 3 measures of combat capability that are enhanced by R&M –Distinguish between reliability and maintainability Lesson 3: Operational & Contractual R&M Requirements –List 3 methods for converting operational R&M requirements into contractual R&M requirements –List 3 approaches for assessing R&M of Commercial and Non- Developmental Items

15. Learn. Perform. Succeed. Current Learning Objectives (Continued) Lesson 4: Providing Insight into Designing for Reliability –Identify the reason that failure rates are easier to deal with than MTBFs –Distinguish between mission reliability and logistics reliability –Identify the aim of Reliability Centered Maintenance Analysis Lesson 5: R&M and Risk Reduction –Recognize the Test-Analyze-and Fix process of achieving reliability growth –Recognize the concept of decision risk in reliability qualification test (RQT) Lesson 6: Impacts of R&M on Manufacturing –Identify the effect that latent defects can have on missions and logistics –How to recognize a manufacturing defect from field data –3 lines of defense to keep manufacturing from producing latent defects

16. Learn. Perform. Succeed. Proposed New Learning Objectives Lesson 4: Providing Insight into Designing for Reliability –Identify the reason that failure rates are easier to deal with than MTBFs –Distinguish between mission reliability and logistics reliability –Identify the aim of Reliability Centered Maintenance Analysis Lesson 5: R&M and Risk Reduction –Recognize the Test-Analyze-and Fix process of achieving reliability growth –Recognize the concept of decision risk in reliability qualification test (RQT) Lesson 6: Impacts of R&M on Manufacturing –Identify the effect that latent defects can have on missions and logistics –How to recognize a manufacturing defect from field data –3 lines of defense to keep manufacturing from producing latent defects

17. Learn. Perform. Succeed. Proposed LOG 103 Learning Objectives/Outcomes Lesson 5: Impact of Legislation and DoD Policy on Reliability, Availability and Maintainability  Discuss USD(AT&L) RAM Policy Memorandum of July 08 and its impact on systems engineering, reliability/logistics and test & evaluation, with emphasis on reliability growth programs  Discuss impact of the Weapons System Acquisition Reform Act of 2009 and updates to DoDI 5000.02 on the Defense Acquisition Management System, systems engineering processes, sustainment and governance Lesson 6/1: Impacts of RAM on Logistics –Recognize how RAM affects support costs –Recognize how RAM affects Total Ownership Costs –List 4 key elements in the definition of reliability –List 2 MTB_s that are logistics reliability parameters –Discuss the definitions of availability Lesson 7/2: Impacts of RAM on Missions –Distinguish between latent failures, random failures and wear out –Identify 3 measures of combat capability that are enhanced by RAM –Distinguish between reliability and maintainability Lesson 1: Introduction to LOG 203 Reliability, Availability and Maintainability (RAM)  Identify the role of LOG 103 within the LCL and other DAWIA Certification Paths  Identify the relationships between LOG 103 and other certification courses  Identify the role of Reliability, Availability and Maintainability(RAM)as a key part of the systems engineering process  Identify the role of Reliability, Availability and Maintainability(RAM) in minimizing the need for logistics Lesson 2: Historical Roots of Reliability, Availability and Maintainability (RAM)  Recognize Early Commercial/DoD Reliability Analyses, Tools and Outcomes  Recognize development and use of DoD Reliability, Availability and Maintainability processes, specifications and documentation  Recognize the impact of RAM on DoD Systems performance, sustainment and costs Lesson 3: The Impact of Acquisition Reform on Reliability, Availability and Maintainability  Discuss the scope and impact of the Perry Memorandum  Discuss the Loss of workforce and skilled RAM personnel  Discuss the Discontinuance of Military Standards and Specifications and its impact  Discuss the Impact of the use of Commercial-Off-The-Shelf on system reliability and performance  Discuss the decrease in effectiveness and suitability as determined by operational Test and Evaluation Lesson 4: Mandatory Sustainment Materiel Availability Key Performance Parameter and Supporting Key System Attributes of Materiel Reliability and Ownership costs  Discuss the origins of the Material Availability KPP, the Material Reliability KSA and the Ownership Cost KSA  Identify the impact of the KPP/KSA on the requirements management process  Discuss the development of the KPP/KSAs within the JCIDS process  Discuss the enablers to materiel readiness that impact RAM, e.g., RCM, CBM, diagnostics/prognostics

18. Learn. Perform. Succeed. Current Learning Objectives (Continued) Lesson 8/3: Operational & Contractual RAM Requirements –List 3 methods for converting operational RAM requirements into contractual RAM requirements –List 3 approaches for assessing RAM of Commercial and Non-Developmental Items Lesson 7: The Systems Design and Operational Effectiveness (SDOE) Model –Discuss the SDOE model and domains –Discuss specific SDOE engineering outputs and their impact on performance, RAM, and sustainment Lesson 8/4: Providing Insight into Designing for RAM –Discuss the concept of design interface –Discuss the elements of supportability –Discuss specific Reliability Availability & Maintainability trade space considerations –Discuss common methodologies and analytic tools –Identify the specific analyses to be conducts, e.g., block diagram development, allocation and prediction, failure modes and effects analysis, maintenance task analysis –Identify the reason that failure rates are easier to deal with than MTBFs –Distinguish between mission reliability and logistics reliability –Identify the aim of Reliability Centered Maintenance –Identify the aim of Condition Based Maintenance –Identify the aim of diagnostics and prognostics Lesson 9/5: RAM Growth Programs and Risk Reduction –Discuss requirements for reliability management programs –Discuss requirements for reliability growth test programs –Discuss best practices –Recognize the Test-Analyze-and Fix process of achieving reliability growth –Recognize the concept of decision risk Lesson 10/6: Impacts of RAM on Manufacturing –Identify the effect that latent defects can have on missions and logistics –How to recognize a manufacturing defect from field data –3 lines of defense to keep manufacturing from producing latent defects Lesson 6/3: Operational & Contractual RAM Requirements –List 3 methods for converting operational RAM requirements into contractual RAM requirements –List 3 approaches for assessing RAM of Commercial and Non-Developmental Items Lesson 7/4: Providing Insight into Designing for Reliability –Identify the reason that failure rates are easier to deal with than MTBFs –Distinguish between mission reliability and logistics reliability –Identify the aim of Reliability Centered Maintenance Analysis –Identify the aim of Condition Based Maintenance –Identify the aim of diagnostics and prognostics in reliability qualification test (RQT) Lesson 9/6: Impacts of RAM on Manufacturing –Identify the effect that latent defects can have on missions and logistics –How to recognize a manufacturing defect from field data –3 lines of defense to keep manufacturing from producing latent defects

19. Learn. Perform. Succeed. Defense Science Board Principal Findings and Recommendations

20. Learn. Perform. Succeed. Systems Failing Reliability During Operational Testing (OT)

21. Learn. Perform. Succeed. Impact of Low Reliability on Life Cycle Cost (LCC) Recent Logistics Management Institute (LMI) and Army studies investigated relationship between reliability and Life Cycle Cost (LCC). Both studies concluded that reliability investment during design and development reduces support cost –LMI study concludes that: 30-60% reduction in support cost is achievable Reacting to lower reliability after IOC (vice during development) results in significant support cost increases OSD CAIG study shows that support costs are two-thirds or more of total system cost –DoD should address acquisition and LCC strategies concurrently Support cost bill payers should participate in requirements definition and trade studies during development Upfront Investment in Robust RAM Effort Emphasizing Reliability Growth Reduces Life Cycle Costs Upfront Investment in Robust RAM Effort Emphasizing Reliability Growth Reduces Life Cycle Costs

22. Learn. Perform. Succeed. Reliability Availability & Maintainability (RAM) Findings Acquisition Reform implementation had a detrimental impact on RAM practices –With some exceptions, the practice of reliability growth methodologies was discontinued during System Design and Development (SDD) and deferred until production –Relevant military specifications and standards were cancelled and not, in all cases, replaced with industry standards –The technical/managerial workforce was reduced in most government program offices and test organizations RAM shortfalls are frequently identified during DT, but program constraints (schedule and funding) often preclude incorporating fixes and delaying IOT&E In some instances, programs had such serious RAM concerns that they were precluded from proceeding to production until the problems could be corrected. Improved RAM decreases life cycle costs and reduces demand on the logistics system

23. Learn. Perform. Succeed. USD(AT&L) Memo “RAM Policy” July 08 “…the single most important step necessary to correct high suitability failure rates is to ensure programs are formulated to execute a viable systems engineering strategy from the beginning, including a robust RAM program, as an integral part of design and development. No amount of testing will compensate for deficiencies in RAM program formulation. To this end, the following RAM-related actions are required as a minimum:” Identify and define RAM requirements during the Joint Capabilities Integration Development System (JCIDS) process and incorporate them into the Request for Proposal (RFP) and as a mandatory contractual requirement During source selection, evaluate the bidders’ approaches to satisfying RAM requirements Ensure flow-down of RAM requirements to subcontractors Make RAM, to include a robust reliability growth program, a mandatory contractual requirement and document progress as a part of every major program review Develop a military standard for consistent RAM development and testing that can be readily referenced in future DoD contracts Ensure an adequate cadre of experienced RAM personnel are part of the Service acquisition and engineering office staffs

24. Learn. Perform. Succeed. Roles and Responsibilities of Government Test & Evaluation (T&E) Organizations “…the aggregate lack of process guidance due to the elimination of specifications and standards, massive workforce reductions in acquisition and test personnel, and the high retirement rate of the most experienced technical and managerial personnel in government and industry has a major negative impact on DoD‘s ability to successfully execute increasingly complex acquisition programs. Major personnel reductions have strained the pool of experienced government test personnel A significant amount of developmental testing is currently performed without the needed degree of government involvement or oversight and in some cases, with limited government access to contractor data The current trend away from governmental involvement in DT makes operationally oriented testing in some areas during development even more difficult to accomplish

25. Learn. Perform. Succeed. LCL Course Development & Revision Roadmap LCL Course Development & Revision Roadmap Course TitleDeliveryPLD & CM DAWIA Cert Course? Funding Development/ Revision Year Planned to Deploy LOG 103 Reliability, Availability & Maintainability (RAM) Distance Learning PLD: Pat Dallosta CM: Rich DiLorenzo (MW) Yes, Level I Sec 852 $695,817 FY10FY11 LOG 206 Intermediate Systems Sustainment Management Distance Learning PLD: Pat Dallosta (Doug Killey) CM: Al Barnes (S) Yes, Level II Sec 852 $719,037 FY10FY11 LOG 340 Performance Based Life Cycle Sustainment Classroom PLD: Dave Floyd CM: Jack Cain (S) Yes, Level III Sec 852 $531,732 FY10-11FY12 LOG 204 Configuration Management (Revision) Distance Learning PLD: Pat Dallosta (Doug Killey) CM: Tom Edison (W) No, Core Plus DAU TBD FY11FY12 LOG 211Supportability AnalysisClassroom PLD: Tom Simcik (Pat Dallosta) CM: Jim Curry (CNE) Yes, Level II Sec 852 TBDFY11FY12 LOG 215 Technical Data/Product Data Mgt Distance Learning PLD: Pat Dallosta CM: TBD No, Core Plus Sec 852 TBDFY12FY13 LOG 235 Performance Based Life Cycle Product Support (Revision) Distance Learning PLD: Dave Floyd CM: Kevin Wedmark (CNE) Yes, Level II DAU TBD FY12FY13 Note: FY10 Deployment of LOG 103 & LOG 206 are a DAU Performance Plan Target under Task 1.1.1 (Develop and deliver high-quality learning assets and services to our customers).