1. 1 Marine Corps Operational Test and Evaluation Activity GW-ITEA Luncheon 20 November 2008 Mark S Adams Science Advisor

2. 2 DIRECTOR Col David L Reeves S-1 S-2 S-3 S-4 S-5 FISCAL DEPUTY David M Havrin GROUND DIVISION CSS DIVISION MAGTF C4ISR DIVISION EXPEDITIONARY TEST DIVISION CHIEF OF TEST Science Advisor Mark Adams Infantry Branch Fires Branch Combat Vehicles Cbt Svc Spt Branch Cbt Eng Branch CBRN Branch Info Sys Branch C4ISR Branch MAGTF C2 Branch Naval Test Branch EFV Branch AAV Section MCOTEA Organization CONTRACTS

3. 3 S-2, Decision Sciences Responsibilities MCOTEA Entry point for new OT&E candidate programs Provide initial review on programs & develop the Evaluation Strategy Provide Live Fire Oversight Manages programs for Information Assurance, instrumentation, and M&S Principal adviser to the Divisions on methodology, threat integration, instrumentation, and M&S Supports development of new T&E methodologies Support Division preparation of project proposals for instrumentation, methodology, and M&S Conducts special studies for the Director, Deputy Director, and Science Advisor Recommends updates to the MCOTEA Reference Publication and associated documents

4. 4  231 Initiatives  MCOTEA Implementation  Opportunities Ahead Agenda

5. 5 Background National Defense Authorization Act of 2007 Public Law 109-364, Section 231 “The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the Director of Operational Test and Evaluation and the Director of the Defense Test Resource Management Center, shall conduct a review of test and evaluation policies and practices of the Department of Defense and issue such new or revised guidance as may be necessary to address emerging acquisition approaches. “

6. 6 THE ISSUES TO BE ADDRESSED DURING REVIEW  Ensuring the adequacy and expediency of test and evaluation Systems acquired under rapid acquisition authority Programs conducted under the spiral development authority Systems acquired under other emerging acquisition approaches Materiel not subject to OT&E requirements but require limited OT&E to ensure safety and survivability  The appropriate use of OT&E resources  Assess technology readiness levels  Section 2366a of USC 10 Background

7. 7 Section 231 Report  08 July 2007  Review of current policy and practices  Considered traditional and emerging acquisition approaches  Incorporated findings of recent studies: Defense Science Board (1999) Government Accountability Office (2000) Defense Acquisition Performance Assessment Panel (2005) National Research Council of the National Academies (2006)

8. 8 RDT&E Procurement O&S Ground Combat Systems Ground Combat Systems Rotary Wing Aircraft Surface Ships Life Cycle Costs Source: PA&E CAIG

9. 9 Section 231 Report “In short, the following additional policy guidelines will be considered with respect to both traditional and emerging acquisition approaches:”

10. 10 1. Focus on measuring improvements to capability and operational support 2. Experiment to learn strengths & weaknesses - impact on capabilities 3. Integrate Development & Operational Testing 4. Start early, be operationally realistic, continue throughout the life cycle 5. Evaluate in mission context at time of fielding 6. Compare to current mission capabilities 7. Use all available information 8. Exploit benefits of Modeling & Simulation The OSD Eight

11. 11 Policy Initiatives 1.Focus on measuring improvements to capability and operational support. What does it mean to us? Tracking performance improvement and determining if the program is demonstrating progress towards entering formal test events.

12. 12 2. Experiment to learn strengths & weaknesses – impact on capabilities. What does it mean to us? Early involvement allows us to provide more informed evaluations that facilitate more timely feedback Policy Initiatives

13. 13 3. Integrate Developmental Testing & Operational Testing. Policy Initiatives What does it mean to us? Whenever possible combined test events save resources, eliminate duplicate testing, and identifies problems earlier

14. 14 4.Start early, be operationally realistic, continue throughout the life cycle. Policy Initiatives What does it mean to us? System problems are identified earlier and can be corrected quickly with less cost if done early in the process

15. 15 5. Evaluate in mission context at time of fielding. Policy Initiatives What does it mean to us? Addresses the “so what” question with regard to the mission impacts of systems not meeting requirements

16. 16 6. Compare to current mission capabilities. Policy Initiatives What does it mean to us? Finding or collecting usable baseline performance data for comparative analysis

17. 17 7. Use all available information. Policy Initiatives What does it mean to us? Historical data, data from contractor testing, data from Modeling and Simulations, etc.

18. 18 8. Exploit benefits of Modeling & Simulation. Policy Initiatives What does it mean to us? New area of involvement – need to develop greater capabilities.

19. 19 Recent Developments –Capabilities Based Testing –S-2, Decision Sciences –Activity-wide PME on 231 Initiatives –Incorporate 231 Checklist In Test Concept IPTs and all Programs –Dr. McQueary Visit –MarCorSysCom Cdr Info Brief –Staff-level Coordination w/ Acquisition Triad MCOTEA Implementation

20. 20 Focus Purpose of T&E - Measure Progress Earlier Info on Strengths And Weaknesses Using All The Data Available In Evaluation Validating Technology Maturity Combined 231 Initiatives - MCOTEA Goals MCOTEA Implementation Support the Marine Corps Acquisition Process

21. 21 Opportunities Ahead Marine Corps T&E Handbook MCOTEA: S-2, COT, SA MCSC: ACProg, SIAT, PG-11 MCCDC: MAGTF Integration Combining MCSC DT Handbook and MCOTEA T&E Handbooks Define critical processes in developmental and operational testing Comprehensive tool that covers all test requirements Interactive tool of relevant, authoritative T&E acquisition information Requirements for coordinating for Fleet operational forces Goal: T&E Integration From The Start

22. 22 Opportunities Ahead Integrated Digital Environment MCOTEA: S-2, COT, SA MCSC: SIAT, PG-10 Collaborative Environment to Support T&E Decision Support System Capabilities Workflow Monitoring Integrating with Data Collection Procedures Delivery: 3Q09; Future Plans to Include MCCDC Goal: Integrated Requirements Traceability

23. 23 Discussion - MCOTEA Oversight Systems Program DTOTLF ACAT Notes Common Aviation Command and Control System (CAC2S) XXIAC Distributed Common Ground System - Marine Corps (DCGS - MC) XXIIIOversight Expeditionary Fighting Vehicle (EFV)XXXID Global Combat Support System - Marine Corps (GCSS - MC) XXIAM Joint Mission Planning System - Expeditionary (JMPS - E) XIVTOversight Logistics Vehicle System Replacement (LVSR)XII Medium Tactical Vehicle Replacement Program (MTVR)XXIC Oversight, but not MDAP, no BLRIP Report required Marine Expeditionary Armored Forces (M1A1 Upgrade, LAV Upgrade, AVLB Upgrade, AAV Upgrade) XXXSpecial Interest

24. 24 Discussion - Methodology Oversight Programs, MCOTEA Lead MCOTEA Self-Assessment Based on Current Familiarity With Programs Consensus Rating among Test Divisions And Staff Used 231 Initiatives Report For Definitions Anecdotal Data Provided for Rationale

25. 25 EFV CAC2S DCGS-MC GCSS-MC JMPS-E LVSR MTVR Armored Summary n/a Y Y↑Y↑ Y↑Y↑ Y↑Y↑ Y↑Y↑ G G G G G G G G G G G G G G G G Y Y G G G G G G G G G G G G G G G G G unk n/a G G G R Y Expd Div MAGTF C4ISR CSS Ground Observations "Green" Systems Either Long-term Running Programs, In Late Spirals, or Restarts with Lots of Lessons Learned "Less-than-Green" Ratings from Uncertainty of the Requirements or In Initial Stages of Program Coordination Mod/Sim an Area for Improvement Observations "Green" Systems Either Long-term Running Programs, In Late Spirals, or Restarts with Lots of Lessons Learned "Less-than-Green" Ratings from Uncertainty of the Requirements or In Initial Stages of Program Coordination Mod/Sim an Area for Improvement 1234567812345678

26. 26 On-going Challenges Ensure New Programs Establish Integration Early Aggressively Pursue All Sources of Data Formalize the Integration Process with the MC T&E Handbook and the IDE Educate and Integrate the MC Acquisition Triad

27. 27 QUESTIONS