Presentation is loading. Please wait.

Presentation is loading. Please wait.

Research Institute for Communication, Information Processing and Ergonomics FGAN Nico Bau, Michael Gerz & Michael Glauer FGAN FKIE 13th ICCRTS Bellevue,

Similar presentations


Presentation on theme: "Research Institute for Communication, Information Processing and Ergonomics FGAN Nico Bau, Michael Gerz & Michael Glauer FGAN FKIE 13th ICCRTS Bellevue,"— Presentation transcript:

1 Research Institute for Communication, Information Processing and Ergonomics FGAN Nico Bau, Michael Gerz & Michael Glauer FGAN FKIE 13th ICCRTS Bellevue, WA June 17 th -19 th, 2008 Testing C2 Interoperability Advancements in Testing of the MIP Baseline 3 Solution

2 Research Institute for Communication, Information Processing and Ergonomics FGAN Outline Introduction to MIP u Goals & Organization u MIP Solution of Baseline 3 MIP Tests u MIP Interoperability & Conformance Tests u MIP Test Reference System u SLT1 / SLT2 / SLT3 Status Quo & Usage Statistics u MIP System Level Tests u Use of the MTRS Summary

3 Research Institute for Communication, Information Processing and Ergonomics FGAN Goals The aim of the Multilateral Interoperability Programme (MIP) is to achieve international interoperability of Command and Control Information Systems (C2IS) at all levels from corps to battalion, or lowest appropriate level, in order to support multinational (including NATO), combined and joint operations and the advancement of digitization in the international arena. Voluntary Project of 26 Nations & NATO ACT u Technical specifications based on consensus u System-independent capabilities u No common funding

4 Research Institute for Communication, Information Processing and Ergonomics FGAN The MIP Solution Source: MIP

5 Research Institute for Communication, Information Processing and Ergonomics FGAN MIP Baseline 3 – JC3IEDM JC3IEDM = Joint Command, Control, and Consultation Information Exchange Data Model MIP Baseline 3 available as draft specification u Current version of the data model: JC3IEDM 3.1c Entity relationship model u 271 Entities, 2005 attributes, » fixed domain values +Semantic definitions +Business Rules (Integrity & transformation rules) u Valid attribute value combinations u Valid associations u Modeling of geometry of objects and actions u Mapping onto APP6a symbols

6 Research Institute for Communication, Information Processing and Ergonomics FGAN MIP Baseline 3 – Data Exchange Mechanism Automatic node identification u Broadcast mechanism Publish/Subscribe technology u Partial replication of JC3IEDM data base content u Based on subscriptions to operational information groups u Taking into account Referential integrity Semantic completeness u DEM exchange format coupled with relational DB schema

7 Research Institute for Communication, Information Processing and Ergonomics FGAN Outline Introduction to MIP u Goals & Organization u MIP Solution of Baseline 3 MIP Tests u MIP Interoperability & Conformance Tests u MIP Test Reference System u SLT1 / SLT2 / SLT3 Status Quo & Usage Statistics u MIP System Level Tests u Use of the MTRS Summary

8 Research Institute for Communication, Information Processing and Ergonomics FGAN Testing the MIP Solution Without MTRS u International test sessions in Greding (Germany) Bilateral/Multilateral u C2ISs alternatively take role of System Under Test Test System u Semi-formal test documents Problems u C2ISs do not support all test phases Specification, preparation, execution, evaluation u Testing (esp. test setup and coordination) is expensive u Legacy systems are not suitable as unbiased testers u Test specifications are subject to interpretation

9 Research Institute for Communication, Information Processing and Ergonomics FGAN Bilateral MIP Interoperability Tests LAN / Internet controls & observes National C2IS B controls & observes National Test Operator B National MIP Gateway National C2IS A National Test Operator A Test Coordi- nation (Chat) Assessment MIP Test Documents MIP Test Documents Interpretation Test Result National MIP Gateway

10 Research Institute for Communication, Information Processing and Ergonomics FGAN MTRS MIP Conformance Tests LAN / Internet National C2IS National Test Operator National MIP Gateway Test Cases Test Results User Data Interpretation MIP Test Documents Test Manager MIP Gateway Java RMI MTRS controls & observes

11 Research Institute for Communication, Information Processing and Ergonomics FGAN Source: MIP Testing the MIP Solution Common information model Staff officer Commander Common understanding Automated information exchange Shared tactical picture Effective C2 for international operations Collaboration Common information model Commander MCI MIP Solution Specifications SOPs Documents Mapping C2IS SLT1 SLT2 SLT3 OLT

12 Research Institute for Communication, Information Processing and Ergonomics FGAN Source: MIP Testing the MIP Solution Common information model Staff officer Commander Common understanding Automated information exchange Shared tactical picture Effective C2 for international operations Collaboration Common information model Commander MCI MIP Solution Specifications SOPs Documents Mapping C2IS SLT2 SLT3 OLT SLT1

13 Research Institute for Communication, Information Processing and Ergonomics FGAN SLT1 (Protocol Tests) T1| | T2|D1|{ |BLK3 SLT1 DEM A RESP ORG OIG_1|GLBSGN|BLK3 SLT1 DEM A RESP ORG|0} T1| | T2|D1|{ |BLK3 SLT1 DEM B RESP ORG OIG_3|GLBSGN|BLK3 SLT1 DEM B RESP ORG|0} T2|D2|{ ||H} T2|D4| |0|3085|x?½VÛr?ý¾ T2|D2|{ ||H} T2|D4| |0|826|x??Q?0Do$³õÓª¡¨X?z·wo¢ÖbJ0ÈËÎÎdó testcase MSLT1MTRS_M101 { 02 request("Please start your C2IS."); 03 //send our own DEM Connection Information 04 OwnConnectionInformation ownDCI = getOwnConnectionInformation(); 05 [!] send(ownDCI) to validator; //now we want to receive a reply 08 [?] receive(ConnectionInfo dci) from validator in { 09 //check whether all fields are set correctly: 10 assertEquals("NodeId", getSut().getNodeId(), dci.getNodeId()); 11 assertEquals("IpAddress", getSut().getIpAddress(), dci.getIpAddress()); 12 assertEquals("TcpPort", getSut().getTcpPort(), dci.getTcpPort()); 13 if (dci.getScope() == Scope.ANNOUNCE) repeat; 14 } 15 } Data Test Script

14 Research Institute for Communication, Information Processing and Ergonomics FGAN SLT1 (Protocol Tests)

15 Research Institute for Communication, Information Processing and Ergonomics FGAN Source: MIP Testing the MIP Solution Staff officer Commander Common understanding Automated information exchange Shared tactical picture Effective C2 for international operations Collaboration Commander MIP Solution Specifications SOPs Documents Mapping C2IS SLT3 OLT Common information model MCI SLT2

16 Research Institute for Communication, Information Processing and Ergonomics FGAN SLT2 (DB to DB) X {{ |OR|NO|MECHANISEDINFANTRYCOMPANY| |0}{ |FE|NO|Ctrl FeatBoundaryOrg/TC1201_01| |0}} {{ |AFLGEO| |0}} {{ |OR|BLK3SLT2DEMARESPORG| |0}{ |FE|Feature/TC1201_01| |0}} {{ |CF| |0}} {{ |CF| |0}} {{ | | |0}} {{ |GVTORG|NO||MECHINFCOY| |0}} {{ |UN| |0}} {{ |FRA| |0}} {{ |NOS| |0}} {{ |MILORG|| |0}} {{ |BLK3SLT2COYA|| |0}} {{ |BDYOR| |0}} {{ | |INCLDE| |0}{ | |INCLDE| |0}} {{ |1|REP|NO||A| ||RDABST||| |OITYPE| |0}{ |1|REP|NO||A| ||RDABST||| |OISTAT| |0}{ |1|REP|NO||A| ||RDABST||| |OIHSTS| |0}{ |1|REP|NO||A| ||RDABST||| |OITYPE| |0}{ |1|REP|NO||A| ||RDABST||| |OIASST| |0}{ |1|REP|NO||A| ||RDABST||| |OISTAT| |0}{ |1|REP|NO||A| ||RDABST||| |OIHSTS| |0}} {{ | || |0}{ | || |0 }{ | || |0}{ | ||… INSERT INTO CONTXT_ELMT_STAT (contxt_id, contxt_elmt_ix,contxt_elmt_stat_ix, cat_code, effctv_dttm, estblng_org_id, creator_id, update_seqnr) VALUES ( , , , 'ADDITN', ' ', , , 0); -- Link the elements to the OIG: HOSTILITY INSERT INTO CONTXT_ELMT (contxt_id, contxt_elmt_ix, rptd_id, creator_id, update_seqnr) VALUES ( , , , , 0); INSERT INTO CONTXT_ELMT_STAT(contxt_id, contxt_elmt_ix, contxt_elmt_stat_ix, cat_code, effctv_dttm, estblng_org_id, creator_id, update_seqnr) VALUES ( , , , 'ADDITN', ' ', , , 0);-- CONTEXT part INSERT INTO CONTXT(contxt_id, cat_code, name_txt, security_clsfc_id, creator_id, update_seqnr) VALUES ( , 'OIG', 'OIG_Subgroup_0200_2', NULL, , 0); Data „Test Script“ Identify records that shall (not) be replicated

17 Research Institute for Communication, Information Processing and Ergonomics FGAN SLT2 (DB to DB)

18 Research Institute for Communication, Information Processing and Ergonomics FGAN Source: MIP Testing the MIP Solution Staff officer Commander Common understanding Effective C2 for international operations Collaboration Commander MIP Solution Specifications SOPs Documents OLT Common information model Automated information exchange Shared tactical picture Common information model MCI Mapping C2IS SLT3

19 Research Institute for Communication, Information Processing and Ergonomics FGAN X {{ |OR|NO|MECHANISEDINFANTRYCOMPANY| |0}{ |FE|NO|Ctrl FeatBoundaryOrg/TC1201_01| |0}} {{ |AFLGEO| |0}} {{ |OR|BLK3SLT2DEMARESPORG| |0}{ |FE|Feature/TC1201_01| |0}} {{ |CF| |0}} {{ |CF| |0}} {{ | | |0}} {{ |GVTORG|NO||MECHINFCOY| |0}} {{ |UN| |0}} {{ |FRA| |0}} {{ |NOS| |0}} {{ |MILORG|| |0}} {{ |BLK3SLT2COYA|| |0}} {{ |BDYOR| |0}} {{ | |INCLDE| |0}{ | |INCLDE| |0}} {{ |1|REP|NO||A| ||RDABST||| |OITYPE| |0}{ |1|REP|NO||A| ||RDABST||| |OISTAT| |0}{ |1|REP|NO||A| ||RDABST||| |OIHSTS| |0}{ |1|REP|NO||A| ||RDABST||| |OITYPE| |0}{ |1|REP|NO||A| ||RDABST||| |OIASST| |0}{ |1|REP|NO||A| ||RDABST||| |OISTAT| |0}{ |1|REP|NO||A| ||RDABST||| |OIHSTS| |0}} {{ | || |0}{ | || |0 }{ | || |0}{ | ||… SLT3 (C2IS Level) 01 prompt 'Please create a friendly unit with some location.'; 02 oig FRDNEU { 03UNIT unit; 04OBJ_ITEM_LOC objItemLocOld; 05GEO_POINT geoPointOld; 06objItemLocOld -> unit; 07objItemLocOld -> geoPointOld; 08 } 09 prompt 'Please change the location of the unit.'; 10 oig FRDNEU { 11new OBJ_ITEM_LOC objItemLocNew; 12new GEO_POINT geoPointNew; 13objItemLocNew -> unit; 14objItemLocNew -> geoPointNew; 15assert:geoPointNew.lat_coord <> geoPointOld.lat_coord 16OR geoPointNew.long_coord <> geoPointOld.long_coord; 17 } MIP = ? MIP MTRS Data „Test Script“

20 Research Institute for Communication, Information Processing and Ergonomics FGAN SLT3 (C2IS Level) Generated automatically, displays received/expected data

21 Research Institute for Communication, Information Processing and Ergonomics FGAN Outline Introduction to MIP u Goals & Organization u MIP Solution of Baseline 3 MIP Tests u MIP Interoperability & Conformance Tests u MIP Test Reference System u SLT1 / SLT2 / SLT3 Status Quo & Usage Statistics u MIP System Level Tests u Use of the MTRS Summary

22 Research Institute for Communication, Information Processing and Ergonomics FGAN Status Quo – System Level Tests Available test cases u 107 formal test cases for SLT1 (protocol stack) u 166 test scripts for SLT2 (replication mechanism) u > 240 MTRS test scripts under development for SLT3 (C2IS-JC3IEDM mapping) u Compare to 17 SLT1 and 20 SLT2 test cases in MIP Baseline 2!

23 Research Institute for Communication, Information Processing and Ergonomics FGAN Usage Statistics (1) MTRS used for 20 systems of 17 nations + NATO Tremendous testing activity … even during weekends! u tests each day (with a peak of 500+ tests) u More than 26,400 test runs as of 05 May 2008 u SLT1: 99,9% completed u SLT2: about 75% completed u SLT3: starting in June 2008

24 Research Institute for Communication, Information Processing and Ergonomics FGAN Usage Statistics (2) Total testing time: 4191 hours ≈ 524 man days

25 Research Institute for Communication, Information Processing and Ergonomics FGAN Usage Statistics (3)

26 Research Institute for Communication, Information Processing and Ergonomics FGAN Usage Statistics (4) Test executions Test execution takes about 32% of total test time Successful tests are executed fastest VerdictCountDurationAvg. Duration Pass :14 h2:02 min Fail :56 h2:56 min Inconclusive :16 h5:50 min Total :27 h3:04 min

27 Research Institute for Communication, Information Processing and Ergonomics FGAN Outline Introduction to MIP u Goals & Organization u MIP Solution of Baseline 3 MIP Tests u MIP Interoperability & Conformance Tests u MIP Test Reference System u SLT1 / SLT2 / SLT3 Status Quo & Usage Statistics u MIP System Level Tests u Use of the MTRS Summary

28 Research Institute for Communication, Information Processing and Ergonomics FGAN Summary (1) MIP Test Reference System u Functional black box testing u Tests conformance to MIP specification u Automated test execution Flexible Test Execution u 24 x 7 Availability over the Internet u No need for test coordination u Support for multilateral tests Different test configurations thanks to modular test gateways u Parallel use by multiple C2IS without interference Simplified Test Evaluation u Error analysis by means of detailed logs MIP Project Management Group "strongly supports the goal and use of MIP Test Reference System"

29 Research Institute for Communication, Information Processing and Ergonomics FGAN Summary (2) Client-Server Architecture with alternative MTRS clients u Graphical user interface u Java API for test automation on the nations’ side u MTRS Website incl. test repository, ticketing system… MIP Baseline 3 Roadmap u System Level Test started in September 2007 u Expected to be completed by January 2009 Significant improvement to MIP test specification process u All MIP test documents put under version control u Significant increase in quantity and quality of tests u Tool support to keep test artifacts consistent https://trac.fkie.fgan.de/MTRS

30 Research Institute for Communication, Information Processing and Ergonomics FGAN Thank you very much for your attention


Download ppt "Research Institute for Communication, Information Processing and Ergonomics FGAN Nico Bau, Michael Gerz & Michael Glauer FGAN FKIE 13th ICCRTS Bellevue,"

Similar presentations


Ads by Google