1. 1 Measuring and Explaining Differences in Wireless Simulation Models Dheeraj Reddy, George F. Riley, Yang Chen, Bryan Larish Georgia Institute of Technology

2. 2 Outline Introduction Simulation Experiments Parameters Topology Description Results and Discussion Conclusions

3. 3 Introduction Discrete-Event Simulations Why wireless simulations ? GTNetS, ns-2 and GloMoSim IEEE 802.11 (MAC and PHY) Motivated by Cavin et. al (POMC 2002) Same specification but different results Why is this important ?

4. 4 Introduction Quantifiable differences Goals Not to point correct/incorrectness Why wireless simulations will often provide differing results Precautions when drawing conclusions from simulations

5. 5 Simulation Experiments Two experiments Concentration at MAC layer only Ideal Behavior Verification under no contention Behavior during contention resolution

6. 6 Simulation Experiments ParameterValue Basic Rate2Mbps Data Rate11Mbps Preamble Rate1 Mbps RTS Size20 bytes CTS Size14 bytes ACK Size14 bytes DIFS50 us SIFS10 us Slot time20 us UDP Header8 bytes IP Header20 bytes ParameterValue LLC/SNAP Header8 bytes Preamble24 bytes Data Header34 bytes Payload512 bytes Forwarding delay500 us Initial CW31 Slot times Node Spacing100 meters Speed of Light300 meters/us Radio Range250 meters Hops per round100 Number of Rounds10 IEEE 802.11 parameters used in simulations

7. 7 Simulation Experiments Experimental Methodology Simple forwarding protocol Node n forwards packet to Node (n+1)%100 A round is finished when node 0 receives its packet from node 99 Experiment finishes at the end of 100 rounds

8. 8 Simulation Experiments Simulation Network Topology

9. 9 Simulation Experiments First Experiment Node 0 originates a packet Experiment finishes at the end of 100 rounds Deterministic results ? There is contention even when a single packet is making rounds 500 us forwarding delay

10. 10 Simulation Experiments Implementation artifact resulting in contention (all three simulators)

11. 11 Results and Discussions Pristine Simulators GloMoSim sends control frames as well as data frames at the same rate ns-2 uses ARP ns-2 and GloMoSim ignore LLC/SNAP layer Additional random delays in ns-2 Experiment 1 using default Simulation Parameters

12. 12 Results and Discussions Theoretical Analysis DR=BR=11Mbps Deterministic results Duration (usec) Cumulative Time (usec) RTS Rx Time257 CTS Rx Time213470 Data Rx Time6201090 Forward Delay5001590 Per Round (100 hops) 0.1590 seconds 100 Rounds15.90 seconds

13. 13 Results and Discussions Set control frame rate as well as data frame rate to 11 Mbps Ignore 1st round in ns-2 Adjust payload size in ns-2 and GloMoSim to account for missing headers Extra random delays removed in ns-2 Closely matches with theoretical analysis

14. 14 Simulation Experiments Second Experiment Exercise the contention resolution mechanisms All nodes create and send a packet to their neighbor at time picked from [0,10ms) 100 packets contending for medium Significant packet loss Widely varying results

15. 15 Results and Discussions Code Inspection and Testing Causes for differing behavior Sample the contention window before/after incrementing when initiating a backoff Contention window increment while sending back-to-back packets Detecting a busy medium (VCS vs. PCS) Interface between MAC and higher layers

16. 16 Results and Discussions 100 simulation runs GTNetS and GloMoSim have some overlap ns-2 takes longest to finish its rounds Later rounds finish significantly faster Data ignored when packet originated by node 0 is dropped

17. 17 Results and Discussions Backoff behavior (Contention Window Sampling) Average number of rounds completed per run/100 BinGTNetSGloMoSim 0-311028721016 31-6327193328 63-12718401023 127-2551325510 255-511867274 511-1023746253 SimulatorGTNetSGloMoSimNS-2 Avg. Completions 78.0062.1856.97

18. 18 Results and Discussions Complicated Scenarios still have considerable variations Analyses/Corrections needed to verify if identified variations help bridge the differences Undetermined variations ? Identify sources of differences and effects of them in a big picture

19. 19 Conclusions Simple Experiments eliminating effects of mobility, path-loss and modulation choices. Simple scenarios have been fixed to provide identical results. Complicated scenarios with channel contention still have considerable variation

20. 20 Questions ?