An Efficient Gigabit Ethernet Switch Model for Large-Scale Simulation Dong (Kevin) Jin

2 Overview and Motivation Gigabit Ethernet Widely used large-scale network with many applications High bandwidth, low latency Packet delay and packet loss is now mainly caused in switch

3 Overview and Motivation Use simulation to study applications running on large-scale Gigabit Ethernet Need an efficient switch model in RINSE Control station Data aggregator RTU/Relays RINSE Simulator Expand the network Explore different architectures

4 Existing Switch Models Detailed models (OPNET, OMNet++) Different models for different types of switches High computational cost Require constantly update and validation Simple Queuing Model (Ns-2, DETER) Simple FIFO queue One model for everything Queuing model based on data collected from real switch [Roman2008] [Nohn2004] Device-independent Model parameters based on experimental observations

5 Model Requirements Fast simulation speed No internal details, no queue Accurate packet delay and loss Device-independent Parameters derived from real switch without knowing device internals Same model derivation process Expected Model Ns-2 OMNet++ OPENNet Queue model based on experiments Ns-2 OMNet++ OPENNet Queue model based on experiments Expected Model Accurate packet delay and packet loss Less accurateMore Accurate Simulation Speed Slow Fast Black-box Switch Model

6 Model Design Approach Perform Experiments on real switch Build Analytical model Build RINSE model Evaluate Simulation Speed and Accuracy

7 Experiment Goal Obtain one-way delay per packet in switch Obtain packet loss sequence Challenge in Gigabit Environment High bit rate - 1Gb/s Low latency in switch -  s

8 Experiment Difficulties Clock synchronization Sender and receiver on the same computer Accurate timestamp for one way delay One Way Delay = transmission delay + wire propagation delay + delay in switch + delay in end host Software Timestamp at NIC driver,  s resolution Large delay at end hosts at high bit rate (>500Mb/s) Have to use hardware timestamp (NetFPGA) 4 on-board Gigabit Ethernet ports 10 ns resolution No end-host delay, processing on the card

9 NetFPGA Card 1234 Experiment Setup CBR UDP flows packet size sending rate, #background flows Time_2 - Time_4 = delay per packet Problem: capture 2000 packets without missing at 1Gb/s Input pcap Time 2Time 4

10 Experimental Results - Packet Delay (Low Load) Single flow Delay NOT depends on sending rate Sufficient processing power to handle single flow up to 1Gb/s Model packet delay as a constant Packet Delay Vs Sending Rate (packet size = 100 Bytes)

11 Experimental Results - Packet Delay (High Load) Mean Delay Vs Sending Rate (packet size = 100 Bytes) 3 extra non-cross interface UDP flows, 950Mb/s each NetGear Low delay with small variance Sufficient processing power to handle 4 flows 3COM Use processor-sharing scheduling and assign weight to a flow according to its bit rate

12 Experimental Results - Packet Loss 0 - received 1 - lost A Packet Loss Sample Pattern 3COM Loss rate NetGear 0.4% 3COM 0.6% Strong autocorrelation exists among neighboring packets

13 Model Design Approach Perform Experiments on real switch Build Analytical model Build RINSE model Evaluate Simulation Speed and Accuracy

received 1 - lost state 1 state 2 state 3 Packet Loss Model 132 Kth order Markov Chain Large K, large state space Our Model - State Space state 1 - long burst of 0s state 2 - short burst of 0s state 3 – burst of 1s Next state depends on current state Number of successive packets in the current state State transition probabilities counting number of pattern occurred in experimental data store in a table in simulator

15 Conclusion Experimental results justified our approach as necessary Building models based on data collected on real switches Created a packet loss model based on experimental data

16 Ongoing Work Experiment Collect long trace with Endace DAG card Cross-interface traffic Model Design a complete packet delay model Study correlation between packet loss and delay Develop black-box in RINSE Evaluation Compare simulation speed with existing queuing models Compare accuracy of the black-box model with real data traces and existing packet delay/loss models

17 Thank You

18 Experimental Results - Packet Delay (High Load) Packet Delay at Beginning of experiment under differenet sending rate (Mb/s) 3COM - Processor Sharing No idea about bit rate until sufficient packets passed Assign max weight at beginning Passed packets   bit rate dertermined  weight   delay 

19 Experimental Results - Packet Delay (Low Load) Single flow Delay NOT depends on sending rate Sufficient processing power to handle 1Gb/s single flow Model packet delay as a constant

20 Experiment Setup I Host NIC 1NIC 2traffic sendertraffic receiver timestamp Packet capture Switch send to self timestamp at NIC driver NIC to NIC overhead

21 RINSE - Architecture Scalable, parallel and distributed simulations Incorporates hosts, routers, links, interfaces, protocols, etc Domain Modeling Language (DML) A range of implemented network protocols Emulation support DML Configuration SSFNet configure SSF [Simulation Kernel] enhance SSF Standard/API implements Protocol Graph Interface 1 MAC PHY Interface N MAC PHY IPV4 ICMP Emulation Socket TCPUDPDNP3 MODBUS BGPOSPF …

22 RINSE - Switch Model Switch Layer black-box model Simple output queue model Flip-coin model - random delay and packet loss Simulation Time: complex queuing model > simple output queuing model > our black-box model ≥ coin model Switch Ethernet MAC Ethernet PHY Switch IP Ethernet MAC Ethernet PHY Host A UDP APP IP Ethernet MAC Ethernet PHY Host B UDP APP

23 Outline Overview and Motivation Our Approach Measurement Experimental Results and Model Conclusion and Ongoing Work

24 Our Approach Black-Box Switch Model Focus on packet delay and packet loss No detailed architecture, no queues Explore the statistical relation between data- in and data-out Paramters derived from data collected on real swtiches