1. M&S Based System Development and Testing in a Joint Net-Centric Environment Bernard P. Zeigler, Ph.D., Co-Director, Arizona Center for Integrative Modeling and Simulation www.acims.arizona.edu and Joint Interoperability Test Command Fort Huachuca, AZ 85613-7051

2. There is an acute need for a new Net-centric testing paradigm at the enterprise level where joint and coalition operations are conducted (Editorial, ITEA Jnl, Sept, 2005) Net-Centric T&E places an increased emphasis on standards conformance self-testing over the Global Information Grid Service Oriented Architecture (GIG-SOA ) and emphasizes services rather than systems

3. Simulation-based acquisition requires the use of M&S in all phases of the system development life-cycle Concept Refinement Technology Development System Development & Demonstration Production & Deployment Operations & Support Modeling and Simulation especially starting early in system development when defects can be caught and corrected with low cost Systems of Systems

4. Simulation-based development implies the need for simulation-based testing -- an infrastructure to test a system that is first formulated as an abstract model is offered by distributed simulation -- both the System under Test (SUT) and the test device are coupled by exchanging data packets on a network Network Test Device send/receive messages System Under Test (SUT) send/receive messages Raises the question: How is the Test Device developed in an authoritative manner? Specified as abstract model, e.g. UML Connecting middle ware, e.g. HLA

5. Problem: Testing Polar Opposite Requirements  To deal with the increasing complexity and advanced decision capabilities of C4ISR systems =>testing methodology has to become more rigorous, in-depth and thorough  To keep up with the rapid change and short development life cycles expected from the system builders => tests have to be ready to conduct in time scales compatible with the agile development strategies of new systems. Solution: employ formal M&S to increase capabilities for simulation-based testing and as a basis to increase the automation of testing processes.

6. Testing of interface standards is a focus area for automated simulation-based testing. Link-16 is required in all Joint and multi-national operations. The Joint Interoperability Test Command (JITC) has developed an automated test generation methodology as its core technology for testing conformance of systems to Link-16 This methodology is fundamentally enabled by the DEVS formalized modeling and simulation approach AWACS Theater Warning ABL DSP/SBIRS F-15 JLENS THAAD PATRIOT MEADS ATACMS AVENGER TEL AEGIS (CEP) SIS(MSCS) Link-16 specification

7. The JITC employed M&S-based testing for the initial major milestone evaluation of the Integrated Architecture Behavior Model (IABM) developed by the Joint Single Integrated Air Picture (SIAP) System Engineering Organization (JSSEO) in 2005. Recent Successful Application The test exercise produced significant results that uncovered flaws in the model design and added acknowledged value to the model development. The ACTGen Development Team, NGIT & ACIMS was selected as the winner in the Cross-Function category for the 2004/2005 M&S Awards presented by the National Training Systems Association (NTSA).

8. Types of Distributed Simulation Testing TestingDescriptionExample standards conformance test whether system conforms to standard supports interoperability Tactical Data Link Standards, Link-16, VMF, USMTF, interoperabilitytest whether systems can interoperate (at the syntactic, higher, levels) Joint Translator Forwarder (JXF) – air to/from land exchange of tactical data mission/capabilitiestest whether system of systems have capabilities required for mission Joint Close Air Support (JCAS)

9. Levels of Distributed Simulation Testing Level/VariationDescriptionExample One-on-One Stimulus/ Response - managed time - real time Test Driver sends message stimuli to which SUT responds IABM SIAP One-on-Many Federation ------------------------- - managed time - real time Test Driver sends message stimuli and controls supporting federates including sensors IABM Peer to Peer Architecture Multiplatform (Many-to-Many interaction) ------------------------- - managed time - real time Active mode (test driver controls scenario interaction) Passive Mode (test manager opportunistically observes interaction JIT 06-03 live C2 systems from USAF, USMC, Navy in remote locations Joint Combined Hardware- in-the-Loop (HWIL) Event (JCHE) Phase 3 CNI pilot event “Runs for Record” (~Aug 06) (CNI = Navy Common Network Interface) JXF tests

10. Test Driver controls the scenario Multiplatform Distributed Simulation – controlled testing Platform (System, Component) Platform (System, Component) Platform (System, Component) Test Driver

11. Distributed Observers look for opportunities to test Multiplatform Distributed Simulation - uncontrolled testing Platform (System, Component) Platform (System, Component) Platform (System, Component) Observer Test Coordinator

12. Joint Test Enterprise Requirements Must respond to requirements for increasing complexity, evolving from standards conformance, to interoperability, to joint operations capabilities testing Top down test development methodology – aim for maximum reuse of test assets by supporting configuration of assets to meet objectives of each new exercise Requires support of appropriate concepts and levels of abstraction – e.g., Objectives, capabilities, functionalities, rule sets, test model federates Requires development of software at levels of abstraction with appropriate flexibility to interface with configurable middleware and hardware to meet conditions of tests, e.G., High speed/accuracy implies real-time operating system replaces default windows OS Develop new assets when no configuration of existing assets can satisfy objectives; then develop with reusability and fit within methodology in mind

13. Joint Test Enterprise Methodology Determine objectives up front and drive test configuration process Determine capabilities to be tested Develop scenarios/mission threads to involve SUT use of identified capabilities Identify lower level functions supporting capabilities Decompose functions into relevant sub sets of rules Develop testable sequences of rules from relevant rule sets Apply automated development of test sequences as test model federates

14. Semi-automated test suite design based on Experimental Frames Standard Provides Behavior Requirements at one or more levels of System Specification Reference Master Model Formalization by mapping into DEVS representations Corresponding levels of System Specification Real-time implement- ation and execution Simulation based testing in net-centric environment Integrated Development and Testing Process

15. Summary Interoperability is mandated throughout the system development process to achieve mission effectiveness The emerging challenges of testing in a joint environment require a new paradigm A test methodology at the enterprise level that scales to net-centric services on the GIG is needed Automation of the generation of test environments is needed – illustrated for tactical data link standards conformance testing