Learn. Perform. Succeed. LOG 103 Reliability, Availability and Maintainability (RAM) Patrick M. Dallosta CPL, Performance Learning Director January 23,

Course Description Throughput / Demographics Metrics That Matter (MtM) Structure & Format PLD Assessment –Course Changes and plans Discussion Topics 2 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

LOG 103 Course Overview TopicDiscussion Course Description (iCatalog Concept Card) Professionals who take this course will be able to understand the relationship between reliability, availability and maintainability (RAM) as a critical factor in design, performance, cost, and sustainment. The course addresses the cross-disciplinary actions of Program Management, Systems Engineering, Test & Evaluation and both acquisition logistics and sustainment to evaluate the impact of reliability and maintainability decisions. Stressing a conceptual approach, the course presents basic reliability, availability and maintainability terminology and engineering practices. It discusses current legislative and DoD policy that have invigorated systems engineering and logistics engineering processes to improve the requirements process, minimize risk through reliability growth programs and ensure effectiveness and suitability through developmental and operational test and evaluation. Certification CourseLife Cycle Logistics Level 1 / Program Management Level 1 Delivery Method/LengthDistance learning ~ 22 hours (11 modules/60 days for completion) Target AttendeesLife Cycle Logisticians, Program Managers and Engineers. Encouraged for those in T&E. Prerequisites ACQ 101, Fundamentals of Systems Acquisition Management Other background information LOG 103 was developed in response to the AT&L/SSE Memorandum for President, DAU “Addition of RAM Courses to Selected DAWIA Certification Career Paths” July 2008 Current version fielded 2010, updated LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

LOG 103 Monthly Learning Effectiveness Trend Dec FY13 through Dec FY15 < 5.2 = Red = Yellow = Green Assessment While the trend appears to be cyclic, metrics indicate improvement since the FY14 revision. Values have been green for past 7 months. LE changes included review of Quiz Questions & Answers and verification of content. 4 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

LOG 103 Monthly Job Impact Trend Dec FY13 through Dec FY15 Assessment Job Impact trends are generally less than 5.2. A review of Student MTM comments reveals that the majority of students are in positions that do not require the knowledge or use of RAM. 5 < 5.2 = Red = Yellow = Green LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

LOG 103 Monthly Overall Trend Dec FY13 through Dec FY15 < 5.2 = Red = Yellow = Green Assessment While the trend appears to be cyclic, metrics indicate improvement since the FY14 revision, with some values in or approaching the green range. The MTMs reflect changes to the course structure, the continuing review of Quiz Questions & Answers and enhanced readability. 6 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

LOG 103 Graduates by Service LOG 103FY 10FY 11FY 12FY 13FY 14 FY 15 FYTD* Graduates1,6885,9187,6956,6089,1512,052 Army2, , Air Force1, , DoN/USMC1, , Industry , Other DoD All Others * DATA AS OF Dec LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015 Comments Significant growth in enrollment due to participation by LCL, ENG, PM and other career fields.

FY14 LOG 103 Student Demographics 8 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015 TopicDiscussion Throughput 80% civilian, 15% military, 5% Industry 71% ACDU Commissioned / 67% Civilian are GS-12 or higher Experience 49% < 5 years 28% between 6-15 years 23% > 15 years Career Field 67% - Satisfy DAWIA/primary career field requirements 13% - Alternative career field certification or general interest Hours to Complete43% < 30 hours Primary Course Access83% from their workplace Criticality to Job / Impact30% (Use subject less than 25% of the time) / 2.5% use the subject 100% of the time) Objectives Appropriate/MetData not captured Service Breakdown Air Force 434 9% Army % DON/USMC 2,164 46% DoD 1,034 22% Other 199 4% Total 4, % Would Recommend79% would recommend to a Colleague

Verbatim LOG 103 Student Comments The Good It is perfect. The course material was perfectly balanced in terms of relevance to my job. It would be helpful for me to see how a working knowledge of RAM helps specifically for a supply planner in sustainment to better understand its effects. It would be helpful to have two versions. One for logisticians, and one for people needing to be familiar with how their job interacts with logistics concerns/efforts. Add more to the sections on designing for reliability & maintainability. The Bad I typically do not like online courses and learn better in a classroom setting Improve the on-line access. The system 'dumped' me numerous time (e.g. more than 30) during course reviews & tests. Decrease the size of the modules. Design courses that tailor to our current jobs. The Ugly Videos. I don’t have time to watch videos…or wait for them on this network. Improve server capability, consistent slow delivery. Why require it? Good question….(How can we improve the training to make it more relevant) Minimize the history lessons. I don't care how we got to today, i just want to know what is true today. For someone in Program Management, this course is too long and extensive for the level of RAM knowledge needed for the job 9 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

LOG 103 PLD Assessment CategoryPLD Assessment Course Effectiveness / Achievement of Learning Objectives Learning Effectiveness addresses the % increase in skill or knowledge. MTMs have decreased from 5.8 to 5.3 as non- LCL students have taken the course. It is not clear how students who have minimal prior RAM knowledge state moderate-low increase in skill/knowledge Graduation rate has improved from 74% to 81% in three years Improvements FY2014 revision improved Courseware & Online Delivery MTMs. Student Feedback Majority of students do not use RAM / minimum Job Impact. Long Term Considerations DoD/Industry recognize LOG 103 as the primary RAM course. Service Considerations Course is not Service specific. ODASD MR Considerations LOG 103 supports LCL/ENG competencies and USD AT&L RAM policy/Sustainment Availability KPP/KSA metrics / ASOE Model / BBP Initiatives. Recommendations FY16-17 update to address content currency, course streamlining, and effectiveness of assessments. 10 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

Course Updates and Timeline 11 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015 TopicDiscussion What changes are required, and why?  Policy/Guidance –DoDI / BBP Initiatives / JCIDS  Content ―History of RAM (may be repurposed as a CL)  MTM Feedback ―Streamlining / Improved transition between Modules / Lessons / Lesson Summaries ―Quiz Questions & Answers What is the timeline?FY16-17 Summary of 2014 Update  Better Buying Power - 20 slides/R&M Tasks impact on Affordability  JCIDS 2012 KSA Terminology – Reliability / O&S Costs  Knowledge Reviews/Test Questions/Answers  “What’s In It For Me?” for each Lesson  Structure  “Right-Sizing” Lessons / “Standalone” Lessons  Improved Content Visibility/Emphasis  Improved Readability/Decreased “Long” Lessons ―Designing for Reliability ―Designing for Maintainability

Questions…? 12 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

Question/ Categories FY10FY11FY12FY13FY14 FY15 FYTD* Courseware n =499 n =2,313 n =2621 n =4194n=4,655n=978 Online Delivery n =500 n =2,303 n =2600 n =4169n=4,647n=974 Learning Effectiveness n =495 n =2,995 n =2588 n =4151n=4,633n=981 Job Impact n =499 n =2,312 n =2608 n =4189n=4,648n=982 * Data (from Data Mart) period Dec LOG 103 Metrics That Matter (MTM) LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

LOG 103 Monthly Online Delivery Trend Dec FY13 through Dec FY15 < 5.2 = Red = Yellow = Green 14 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015 Assessment While the trend appears to be cyclic, metrics indicate improvement since the FY14 revision. Values have been green for past 8 months. Online delivery changes included options for managing video/audio v. text options.

LOG 103 Monthly Courseware Trend Dec FY13 through Dec FY15 < 5.2 = Red = Yellow = Green 15 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015 Assessment While the trend appears to be cyclic, metrics indicate improvement since the FY14 revision. Values have been green for past 7 months. Courseware changes restructured Modules to improve readability / student time management.

Module/Title Terminal Learning Objective (TLO) LOG Competencies BBP Tenets Module 1. Historical Roots Of Reliability, Availability And Maintainability (RAM) Recognize the historical roots of Reliability, Availability and Maintainability (RAM). 1 – Product Support Management 2 – Design Interface 3 – Sustaining Engineering 4 – Maintenance & Maintenance Planning 5 – Supply Support 7 – Technical Data 8 – Support Equipment 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 5 – Promote Competition 6 – Improve Tradecraft Module 2. The Impact of Acquisition Reform on Reliability, Availability and Maintainability (RAM) Recognize the impact of Acquisition Reform on Reliability, Availability and Maintainability (RAM). 1 – Product Support Management 2 – Design Interface 3 – Sustaining Engineering 4 – Supply Support 5 – Maintenance Planning and Management 7 – Technical Data 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 5 – Promote Competition 6 – Improve Tradecraft Module 3. Mandatory Sustainment Availability Key Performance Parameter (KPP) and Supporting Key System Attributes (KSA) of Reliability and Operation and Support (O&S) Cost Recognize how the Availability KPP and the Reliability and O&S Cost KSAs strengthen Reliability, Availability and Maintainability (RAM). 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 5 – Promote Competition 6 – Improve Tradecraft TLOs – Modules 1 – 4 16 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

Module/Title Terminal Learning Objective (TLO) LOG Competencies BBP Tenets Module 4. The Impact of Legislation and DoD Policy on RAM Recognize the impact of legislation and DoD policy on RAM. 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 5 – Promote Competition 6 – Improve Tradecraft Module 5. RAM As A Key Element Of Life Cycle Performance-based Product Support Recognize how RAM is a key element Of Life Cycle Performance-based Product Support 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 5 – Promote Competition 6 – Improve Tradecraft TLOs – Modules 4 – 5 17 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

Module/Title Terminal Learning Objective (TLO) LOG Competencies BBP Tenets Module 6. Impacts of RAM on Warfighter Capability Given a scenario, recognize how RAM impacts Warfighter Capability. 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs Module 7. Operational and Contractual RAM Requirements Describe how contractual requirements are developed from Operational Requirements for a new development system and one using COTS/NDI items. 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 5 – Promote Competition 6 – Improve Tradecraft TLOs – Modules 6 – 7 18 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

Module/Title Terminal Learning Objective (TLO) LOG Competencies BBP Tenets Module 8. Designing for Reliability Identify how the design interface process promotes the concept of Designing for Reliability. 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 5 – Promote Competition 6 – Improve Tradecraft Module 9. Designing for Maintainability Identify how the design interface process promotes the concept of Designing for Maintainability. 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 5 – Promote Competition 6 – Improve Tradecraft TLOs – Modules 8 – 9 19 LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015

Module/Title Terminal Learning Objective (TLO) LOG Competencies BBP Tenets Module 10. Risk Reduction Through Test & Evaluation Recognize how Test and Evaluation activities contribute to achievement of RAM requirements. 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 7 - Technical Data 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 5 – Promote Competition 6 – Improve Tradecraft Module 11. Impacts of Manufacturing Processes on RAM Recognize the impact of manufacturing process anomalies on system RAM performance. 1 - Product Support Management 2 - Design Interface 3 - Sustaining Engineering 4 - Supply Support 5 - Maintenance Planning and Management 6 - Packaging Handling Storage & Technical Data 8 - Support Equipment 9 - Training and Training Support 10 - Manpower and Personnel 11 - Facilities and Infrastructure 12 - Computer Resources 1 – Achieving Affordable Programs 2 – Control Costs 3 – Incentivize Productivity 4 – Eliminate Unproductive Processes and Bureaucracy 6 – Improve Tradecraft TLOs – Modules 10 – LOG 103 Reliability, Availability and Maintainability (RAM) January 23, 2015