1. Vladimir MarbukhWCTG - NIST Verification Plan for SEAMLSS (version 1.6) V. Marbukh and J.S. Pegon Wireless Communications Technology Group National Institute of Standards and Technology

2. Vladimir MarbukhWCTG - NIST Major Components of SEAMLSS HLA/RTI SDF Simulator SDF file Scenario Processing Scenario Generation

3. Vladimir MarbukhWCTG - NIST Scenario Generation Components Modsaf SDF file SDF Generator Excel Spread Sheet Excel SDF PDEF Generator PDEF file

4. Vladimir MarbukhWCTG - NIST Scenario Processing Components

5. Vladimir MarbukhWCTG - NIST Test (a) to verify PDEF Excel Generator Specify treads using Excel thread database (spreadsheets) Run these spreadsheets through the Excel PDEF Generator Manually check the resulting PDEF file for: –thread definitions –firing rules –triggers –IERs (Information Exchange Requests)

6. Vladimir MarbukhWCTG - NIST Test (b) to verify SDF Excel Generator Specify scenario using Excel thread database (spreadsheets) Run these spreadsheets through the Excel SDF Generator Manually check the resulting SDF file for –number of entities –entity names –propagation model (Free Space, TIREM) –simulator (OPNET, PARSEC)

7. Vladimir MarbukhWCTG - NIST Test (c) to verify ModSAF SDF Generator Create scenario using ModSAF Run this scenario through the SDF Generator Manually check the resulting SDF file for –number of entities –entity names –propagation model (Free Space, TIREM) –simulator (OPNET, PARSEC) For each of tests (a), (b) and © also create large- scale scenarios to stress SDF and PDEF Generators

8. Vladimir MarbukhWCTG - NIST Test (d) to verify SDF Parser, Thread Manager and CE for OPNET Create simple scenarios in SEAMLSS Run these scenarios through SEAMLSS Calculate the corresponding delays and losses Analytically calculate the delay and losses Compare the simulation and analytical results

9. Vladimir MarbukhWCTG - NIST Test (e) Run the same SDF and PDEF through SEAMLSS- Lite with OPNET vs. SEAMLSS Lite with PARSEC This test verifies –SDF Parser (since OPNET and PARSEC have different parsers) –Thread Manager –CE (Communication Elements) for SEAMLSS executing OPNET and PARSEC

10. Vladimir MarbukhWCTG - NIST Test (f) Run the same SDF and PDEF through complete SEAMLSS vs. SEAMLSS Lite This test verifies –part of SDF simulator –HLA/RTI without mobility

11. Vladimir MarbukhWCTG - NIST Test (g) Run the same SDF and PDEF through complete SEAMLSS with OPNET vs. SEAMLSS-Lite with PARSEC This test verifies –SDF simulator –HLA/RTI with mobility since SEAMLSS-Lite executing PARSEC has mobility

12. Vladimir MarbukhWCTG - NIST Test (e), (f), and (g) Will be run with all three scenarios (Glomo, Scenario 1, and Scenario 2) and with all three devices ( HMT, WaveLAN, and 802.11) Will be run first with Free Space model and ideal radios, and then with TIREM and real radios After running these tests we will –manually check output files for the number of communications –compare thread level metrics (delays and completions) using the Excel macros created by NIST

13. Vladimir MarbukhWCTG - NIST Simulation Time Estimate The proposed plan calls for 100 or more simulation runs (7 tests by 3 scenarios by 3 types of radios plus additional runs for stress testing, varying parameters, etc) Fair estimate for the run time for one medium size scenario with Free Space propagation is 5-10 hours. Running TIREM probably takes much longer due to considerable amount of processing. Conclusion: optimistic estimate of just running simulations is 1000 hours.