Curriculum Review LOG 103 Reliability, Availability & Maintainability (RAM) Presented to the Life Cycle Logistics (LCL) Functional Integrated Product Team (FIPT) Presented by Patrick M. Dallosta, CPL Performance Learning Director Defense Acquisition University

Agenda Key Personnel FY 13 LCL Certification Curriculum “Path Ahead” LCL Competency – DAU Curriculum Crosswalk Background Current Status Course Description Course Content / TLOs & ELOs / Course Outline MTM / Projected Throughput PLD Assessment / Impact on Training Questions & Answers

LOG 103 Key Personnel Course Manager (CM) Dr. Andre Murphy Course Manager Regional Representatives (CMRR) Capital and Northeast: John Sullivan West: Andre Murphy Midwest: Steve Putman South: Bryan Hammond Mid-Atlantic: Dr. Bill Conroy GLTC Instructional Systems Specialist (ISS) James McDaniel LCIC Performance Learning Director (PLD) Patrick Dallosta LCIC Director for Logistics and Sustainment Bill Kobren

FY12 Life Cycle Logistics Certification Training Level I Certification Level II Certification Level III Certification LOG 101 Acquisition Logistics Fundamentals P Choice of one of: RQM 110, ACQ 265, BCF 215 , LOG 204 LOG-206 Intermediate Systems Sustainment P ACQ 201B Intermediate Systems Acquisition 2 CL Modules: PBL & Designing for Support LOG 200 Intermediate Acquisition Logistics P 18-32 hrs, on-line/clsrm 27 hrs, on-line 21 hours, on-line 3 CL Modules: LCSP, BCA & Log Assessment LOG 102 Systems Sustainment Management 4.5 days classroom 32 hours on-line 3 hrs ea, on-line P P P NOTE: There are NO prerequisites for LOG 235 LOG 235 Performance Based Logistics 54 hours, online 3 hrs ea, on-line P ACQ 101 Fundamentals of Systems Acquisition Management ACQ 201A Intermediate Systems Acquisition LOG 201 Intermediate Acquisition Logistics P 25 hrs, on-line LOG 340 Life Cycle Product Support LOG 103 Reliability, Availability & Maintainability P 37 hours, on-line 4.5 days classroom 25 hrs, on-line 4.5 days classroom P LOG 350 Enterprise Life Cycle Logistics Management 26 hrs, on-line 2 CL Modules: Life Cycle Mgt/ Sustainment Metrics & Supportability Analysis LOG 211 development scheduled for FY12, deploy initially as Core Plus in FY13, and DAWIA Certification course planned for FY14 LOG 340 deploys at start of FY12 as 4.5 day long classroom course (replacing LOG 236) LOG 350 is a 9.5 day classroom course Student choice of CL modules at Levels II and III are now specified SYS 101 added a mandatory Level I certification course Choice of RQM 110, ACQ 265, BCF 215 OR LOG 204 added at Level III LOG 365 PSM Course Development FY12, with deployment as Core Plus course targeted for FY13 SYS 101 SPRDE Fundamentals P 9.5 days classroom Knowledge based GS 5-9 & E7-O3 Experience: 1 yr 144 total hours Application/case based GS 9-12 & E7-O4 Experience: 2 yrs 228 total hours Case/scenario based GS 13-14 & E7-O5 Experience: 4 yrs 139-153 total hours 4+8 hrs ea, on-line 35 hrs, on-line Level I “Core Plus” Courses & CL Modules (See DAU iCatalog for details) Level II “Core Plus” Courses & CL Modules (See DAU iCatalog for details) Level III “Core Plus” Courses & CL Modules (See DAU iCatalog for details) P = Prerequisite

LCL Competency – DAU Curriculum Crosswalk Course LOG 101 102 103 200 201 204 206 211 LOG 215 235 LOG 340 350 365 Logistics Design Influence Integrated Product Support (IPS) Planning Product Support and Sustainment Configuration Management Reliability & Maintainability Analysis Technical/Product Data Management Supportability Analysis Legend: Principal course that addresses the competency Deployed course/significant upgrade in progress Courses where the competency is a major learning point New course in development/planning Addressed, but not major learning point

LOG 103 Competency Analysis COMPETENCY/PROFICIENCY 1. Historic Roots 2. Acquisition Reform 3 .KPP/KSA 4. Legislation/Policy 5. Life Cycle Product Spt 6. Warfighter Capability 7. Op/Contract Rqmts 8. System Design Model 9. Design for Reliability, 10. Test & Evaluation 11. Manufacturing/ LOGISTICS DESIGN INFLUENCE (Note 1) Use analysis tools and techniques to influence logistics design. * Apply Logistics Design Influences to include RAM, CBM/RCM, Supportability. Apply and relate diagnostics/prognostics and CBM to Design Influence. Evaluate impacts on supportability based on changes in policy, procedures and tools. Evaluate Performance Metrics and relate them to Design Influence. Incorporate Reliability Centered Maintenance (RCM) into systems Design Recognize shortfalls in program information and data Perform data collection/analysis of failure data and supply rates Recommend design changes to optimize sustainability/min Life cycle Cost/Footprint Ensure design incorporates sustainment-related design criteria Integrate supportability requirements into the systems engineering process Develop and implement product support strategies Assess the impact of design changes on supportability Perform Maintenance Task Analysis, Level of Repair Analysis, Core Workload Assessment, Depot Source of Repair

LOG 103 Background WSARA/DOD 500.02 significantly strengthened DOD Reliability, Availability and Maintainability (RAM) Policy, Systems Engineering/Reliability/Test & Evaluation processes and organizations. Life Cycle Logistics DAWIA certification path reflects new course requirements in the area of RAM and Supportability LOG 103, “Reliability, Availability and Maintainability” LOG 211, “Supportability Analysis” New curriculum development – deployed late FY 12 Expect great cross-functional interest as RAM / Supportability is reconstituted in program offices – a “pendulum swing” back to government resulting from the aftermath of Acquisition Reform

LOG 103 Current Status Legacy LOG 203 “Reliability & Maintainability” Transitioned to LOG 103 “Reliability, Availability & Maintainability” LCL Certification Course - 1 October 2010 (FY 11) Program Management Certification Course - 1 Oct 2011 Prerequisite – ACQ 101 Fundamentals of Sys Acquisition Mgt Related Learning Assets: CLL 008 Designing for Supportability in DoD Weapon Systems CLL 012 Supportability Analysis DAG Chapter 4, Systems Engineering / Chapter 5, Life Cycle Logistics LOG 211 Supportability Analysis DL / 5 Sections / 300 students / 300 enrolled avg. /150 waitlists Currently updating Test Questions/Raising Sections to 400

LOG 103 Course Description Understand the relationship between Reliability, Availability and Maintainability (RAM) as a critical factor in design, performance, cost, and sustainment. Identify and implement cross-disciplinary actions of Program Management, Systems Engineering, Test & Evaluation to evaluate the impact of Reliability and Maintainability on performance and product support. Introduce basic Reliability, Availability and Maintainability (RAM) terminology, mathematical concepts, and engineering practices. Discuss current legislation/policy that invigorated Systems Engineering processes, improved the requirements process, minimized risk through Reliability Growth programs and ensure Effectiveness and Suitability through Developmental And Operational Test And Evaluation.

LOG 103 Course “Take-Aways” Relationship between Logistics and RAM Impact of RAM on system design and product support RAM activities within JCIDS / Systems Engineering / Test & Evaluation and Sustainment processes RAM concepts, terminology, engineering practices and mathematical calculations Legislation/OSD policy/Sustainment governance Historical roots of RAM Apply DAU Learning Assets to establish a collaborative environment and facilitate insight into current events. 10

LOG 103 Lessons New Lesson Legacy Lesson Historical Roots of Reliability, Availability & Maintainability Impact of Acquisition Reform on RAM Origins of the Mandatory Sustainment Metrics KPP/KSAs Impact of Legislation/DoD Policy on RAM RAM as a Key Element of Product Support Impacts of RAM on Warfighter Capability Operational and Contractual RAM Requirements Providing Insight Into Designing for RAM Risk Reduction Through Test and Evaluation Impacts of Manufacturing Process Anomalies on RAM 11

LOG 103 Metrics That Matter (MTM) Question/ Categories FY10 FY11 Courseware n=499 n=2,313 Online Delivery n=500 n=2,303 Learning Effectiveness n=495 n=2,995 Job Impact n=2,312 5.98 6.07 5.89 5.74 5.90 5.67 5.81 * Data (from Data Mart) period 1 Oct 2011 thru 31 Dec 2011 12

LOG 103 Projected Throughput FY 10* FY 11 Graduates 1,688 2,343 LOG 203 FY 09 COMPARISION 1,140 Army 512 696 Air Force 421 628 Navy 327 616 Industry 367 46 Other DoD 286 All Others 15 71 * Period of Mar 15 – Sep 30 2010 13

LOG 103 PLD Assessment Rigorous discussion of RAM concepts Expands students’ knowledge beyond the ‘traditional logistics’ perspective. Consistent statements “…glad I took this course…I’m glad I passed.” “…never really understood what RAM was before.” “…it was difficult, but I really learned a lot.” “…really had to think about the answers.” “…I had to take something, so I chose this because it was different.”

LOG 103 Impact on Training Directly addresses critical LCL RAM Competency and Curriculum needs Weapons Systems Acquisition Reform Act (WSARA) DoD 5000.02 / OSD Memorandum / DOT&E Memorandum DTM 11-003, Reliability Planning, Analysis, Tracking and Reporting Provides basic instruction in new area for students Availability KPP/KSAs RAM metrics / Sustainment governance Systems Engineering – Reliability Growth – T&E processes Sustainment governance/metrics Cross-DAWIA Career Field training promotes communications between PM, SPRDE, T&E and LCL Career Fields

Questions & Answers LOG 103 Curriculum Review Thank you for the Opportunity to Serve the Life Cycle Logistics (LCL) Functional Integrated Product Team (FIPT) 27

Back-up Go Ravens 17

LOG 103 TLO/ELO - 1 Lesson 1: Historical Roots of Reliability, Availability, and Maintainability (RAM)   TLO: Recognize historical roots of reliability, availability, and maintainability (RAM) ELOs: Identify significant events that transformed and improved reliability. Indicate documentation used to write current RAM policy. Identify how RAM impacts sustainment and cost. Identify how developmental test and evaluation aids RAM achievement. Indicate the role of RAM in the systems engineering process. Identify how RAM aids supportability and impacts logistics. Recognize how operational test and evaluation aids weapon system effectiveness and suitability.

LOG 103 TLO/ELO - 2 Lesson 2: The Impact of Acquisition Reform on Reliability, Availability, and Maintainability (RAM)   TLO: Recognize the impact of acquisition reform on Reliability, Availability, and Maintainability (RAM).  ELOs: Recall the scope and impact of the “Perry Memorandum”. Indicate how the discontinuance of military standards and specifications impacted RAM. Identify how the loss of workforce and skilled personnel impacted RAM. Recognize how the use of commercial-off-the-shelf items impacted RAM. Indicate how poor RAM design and performance decreased effectiveness and suitability, as identified during OT&E. Identify how RAM and supportability affect early systems engineering activities and sustainment planning.

LOG 103 TLO/ELO - 3 Lesson 3: Mandatory Sustainment Availability Key Performance Parameter (KPP) and Supporting Key System Attributes (KSA) of Materiel Reliability and Ownership Costs   TLO: Recognize how the availability KPP and the material reliability and ownership costs KSAs strengthen Reliability, Availability, and Maintainability (RAM).  ELOs: Identify the origin of the availability KPP, the material reliability KSA, and the ownership cost KSA. Identify the impact of the availability KPP/KSA on the Joint Capabilities Integration and Development System (JCIDS) process. Indicate the role of the combat developer in the refinement of the KPP/KSAs within the JCIDS process. Recall how the Reliability, Availability, Maintainability – Cost Rationale Report Handbook is used to calculate the availability KPP and its KSAs. Recognize the use of enabling technologies in meeting availability KPP/KSA requirements.

LOG 103 TLO/ELO - 4 Lesson 4: The impact of legislation and DoD policy on RAM. TLO: Recognize the impact of legislation and DoD policy on RAM. ELOs: Identify the impacts of defense policy on RAM, per the GAO report. Identify the impact of the DOT&E 2007 Report on RAM. Identify the impact of the Defense Science Board Task Force on DOT&E on RAM. Identify the impact of the Reliability Improvement Working Group (RIWG) on RAM. Indicate the impact of the 2008 AT&L Memorandum on RAM. Indicate the impact of DoDI 5000.02 on RAM. Indicate the impact of the Weapons System Acquisition Reform Act on RAM. Indicate the impact of the Weapon System Acquisition Reform Product Support Assessment Report of November 2009 on RAM. Recognize the changes to the Defense Acquisition Guidebook (DAG) as a result of legislation and acquisition policy changes. Recall how changes in the JCIDS process improved RAM requirements. Identify how reliability growth programs have been implemented. Recall how integrating Developmental and Operational Testing strengthen RAM.

LOG 103 TLO/ELO - 5 Lesson 5: RAM as a Key Element of Life Cycle Performance-Based Product Support   TLO: Recognize how RAM is a key element of Life Cycle Performance-Based Product Support.  ELOs: Identify the key elements of availability. Identify the four key elements in the definition of reliability. Indicate two key MTBxs that are logistics reliability parameters. Identify the key elements of maintainability. Recognize how RAM affects support costs. Recognize how Availability KPP/KSA trade spaces are balanced. Recall how RAM is a key driver in the development of a Performance-Based Product Support Strategy. Recall how RAM is a key driver in demand forecasting, spares planning, provisioning, and supply support. Recall how RAM is a key driver of Total Ownership Costs. Recall how RAM is a key driver of Sustainment Maturity Levels.

LOG 103 TLO/ELO - 6 Lesson 6: Impacts of RAM on Warfighter Capability TLO: Given a scenario, recognize how RAM impacts Warfighter Capability   ELOs: Identify User RAM requirements Recall how the Joint Capabilities Integration and Development System (JCIDS) Process works List the measures of combat capability that are enhanced by RAM  Note changes

LOG 103 TLO/ELO - 7 Lesson 7: Operational and Contractual RAM Requirements TLO: Describe how contractual requirements are developed from Operational Requirements for a new development systems and one using COTS/NDI items. ELOs: State the methods for converting operational RAM requirements into contractual RAM requirements. List the differences between field reliability data and contractual reliability data. Distinguish between Mean Time Between Failure (MTBF) and Mean Time Between Maintenance (MTBM). Define Commercial and Nondevelopmental items. List the methods for assessing the RAM of Commercial and Nondevelopmental Items. List the methods for ensuring the RAM of Commercial and Nondevelopmental Items.

LOG 103 TLO/ELO - 8 Lesson 8: The Systems Design and Operational Effectiveness (SDOE) Model TLO: Explain how the Systems Design and Operational Effectiveness (SDOE) Model is used to enhance system design for supportability. ELOs: Explain how the SDOE model is used Identify the SDOE domains Explain the Supportability Analysis process Summarize how the SDOE outputs impact on performance, RAM, and sustainment characteristics in the final system design

LOG 103 TLO/ELO - 9 Lesson 9: Providing Insight into Designing for RAM TLO: Given a scenario, describe how the process inputs and expected outcomes of the System Engineering Process supports RAM requirements. ELOs: State Design Interface concepts Identify the Supportability elements State the Reliability Availability & Maintainability trade space considerations State the analytic methodologies and tools used in the design process Identify the design-supportability analyses processes State the relationships between Mission reliability and logistics reliability Identify the process for Reliability Centered Maintenance (RCM) Identify the process for Condition Based Maintenance Plus (CBM+) State how the Diagnostics, Prognostics and Health Management of the system are related

LOG 103 TLO/ELO – 9 (Continued) Lesson 9: Providing Insight into Designing for RAM TLO: Given a scenario, describe how the process inputs and expected outcomes of the System Engineering Process supports RAM requirements. ELOs: Identify the reason that failure rates are easier to deal with than MTBFs Identify the aim of FMECA List three purposes for reliability predictions Distinguish between maintenance and maintainability List the types of diagnostic errors and their associated effects Recognize the benefit of vertical testability Distinguish among mean down time, mean active maintenance time, and mean time to repair Describe why the maintainability of a system depends on the reliability and maintainability of its subsystems

LOG 103 TLO/ELO - 10 Lesson 10: Risk Reduction Through Test and Evaluation TLO: Recognize how Test and Evaluation activities contribute to achievement of the RAM requirements. ELOs: State the benefits of utilizing Developmental and Operational Test and Evaluation (D/OT&E). Define the purpose of the Test & Evaluation Master Plan (TEMP) List the requirements of the Reliability Management program Name the requirements of the Reliability Growth Test program Identify how the DT&E Test-Analyze-Fix process contribute to reliability growth State the benefits of the Operational Test and Evaluation program in evaluating military capability Identify the risks in making early decisions Identify industry and government best practices in the Test and Evaluation program Recognize the importance of a contractual definition of failure   Restored per discussion re Lesson 7.

LOG 103 TLO/ELO - 11 Lesson 11: The Impact of Manufacturing Process Anomalies on RAM TLO: Recognize the impact of manufacturing process anomalies on system RAM performance. ELOs: Recognize how stated RAM requirements drive the level of quality in the manufacturing processes.[DAU1]  Identify the Impact of latent defects on performance and logistics Identify how the process of the analysis of field data is used to in determine the root causes of defects State how latent defect issues are identified from field failure data Identify the techniques used to eliminate latent manufacturing process defects