1. US Army Operational Test Command 1 Test Technology Directorate (TTD) Challenges of System of Systems Testing In a Contemporary Operational Environment

2. US Army Operational Test Command Provide an overview of the challenges to properly operationally test systems of systems given the contemporary operational environment primarily from a technology viewpoint. Purpose 2

3. US Army Operational Test Command Definitions 3 Operational test (OT) Any testing conducted with the production or production like system in realistic operational environments, with users that are representative of those expected to operate, maintain, and support the system System of systems is a collection of task-oriented or dedicated systems that pool their resources and capabilities together to obtain a new, more complex, 'meta-system' which offers more functionality and performance than simply the sum of the constituent systems.

4. US Army Operational Test Command Definitions Operational environment is a composite of the conditions, circumstances, and influences that affect the employment of military forces and bear on the decisions of the unit commander. Contemporary operational environment (COE) is the overall operational environment that exists today and in the near future (out to the year 2020). The range of threats extends from smaller, lower- technology opponents using more adaptive, asymmetric methods to larger, modernized forces able to engage deployed U.S. forces in more conventional, symmetrical ways. In some possible conflicts (or in multiple, concurrent conflicts), a combination of these types of threats could be especially problematic.

5. US Army Operational Test Command Critical Variables Physical environment Nature and stability of the state Military capabilities Technology Information External organizations Sociological demographics Regional and global relationships National will Time Economics

6. US Army Operational Test Command ASOC/DASC JCAS aircraft JTAC Hostile targets Airborne C2 Friendly forces ISR assets JFO/Observers TACP/AO TOC/COC FSE/FSCC JFACC/JAOC/ TACC Senior FSE/FSCC III/X II TACP/AO TOC/COC FSE/FSCC JFMCC/ AREC/ SACC JFLCC (Corps/MEF HQ) JSOTF WOC CRC/TAOCXX XXX FAC(A) aircraft Immediate Request for CAS Process Through Execution The Challenge: Replicating a Complex Environment

7. US Army Operational Test Command Given: Rapidly changing technology System-of-systems more complex Size and scope of tests is increasing Resource constraints – Units, Space, Time, Funds OTC’s Technology Challenge Providing robust operational environments in which to test the network-centric system-of-systems and collecting the information needed for evaluation. OTC’s Response: Technology – Strategy to build on what works to prepare for the future Expertise – Depth of experience and expertise is nurtured and shared Processes – Accredit sim/stim; certify instrumentation; validate data management, accredit threat

8. US Army Operational Test Command Use of Simulations Stimulate the player unit to cause actions to be performed. Simulate a capability or elements that cannot be provided due to resource constraints, safety restrictions, or other limitations. Generate test designs and provide rehearsal and modification capabilities, provide scenario or event list generation, and support test control. 8 Source: TOPM 73-182

9. US Army Operational Test Command ASOC/DASC JCAS aircraft JTAC Hostile targets Airborne C2 Friendly forces ISR assets JFO/Observers TACP/AO TOC/COC FSE/FSCC JFACC/JAOC/ TACC Senior FSE/FSCC III/X II TACP/AO TOC/COC FSE/FSCC JFMCC/ AREC/ SACC JFLCC (Corps/MEF HQ) JSOTF WOC CRC/TAOCXX XXX FAC(A) aircraft Immediate Request for CAS Process Through Execution Live Virtual Constructive Operational Testing: Providing a Wrap Around

10. US Army Operational Test Command Instrumentation Definition 10 Instrumentation is the use of devices to measure properties, monitor activity, collect data, store data, organize data, and control actions Data requirements dictate instrumentation requirements Evaluator requirements dictate data requirements

11. US Army Operational Test Command Instrumentation Issues 11 Test platform configurations rarely consistent Multiple trips required to complete and test instrumentation Test platforms continue to evolve throughout the development period Certification and VV testing requires planning and support similar to record test Late system access and delayed system readiness delays certification testing and the certification request Instrumentation use increases test cost

12. US Army Operational Test Command Instrumentations Challenges Instrumentation Intrusiveness and Ruggedness

13. US Army Operational Test Command Questions? US Army Operational Test Command 13

14. US Army Operational Test Command Key Definitions Model – A physical, mathematical, or otherwise logical representation of a system, entity, phenomenon, or process. Simulation – A method for implementing a model over time. Stimulation – Providing the System Under Test with a representation (quantity, quality, and necessary content) of input forces, conditions, or data at the times required, to produce a system response. Federation of Models & Simulations – A system of interacting M&S with supporting infrastructure, based on a common understanding of the objects portrayed in the system. Accreditation – The official certification that a model, simulation, or federation of M&S and its associated data is acceptable (both credible and suitable) for use for a specific purpose. Verification – The process of determining that a model implementation and its associated data accurately represents the developer’s conceptual description and specifications. 14 Source: TOPM 73-182