doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 1 A Measurement Study of WiFi Backoff Protocols Date: Authors:

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 2 Abstract Despite much theoretical work, different modifications of backoff protocols in networks lack empirical evidence demonstrating their real-life performance. To fill the gap we have set out to experiment with performance of exponential backoff by varying its backoff factor. Despite the satisfactory results for throughput, we have witnessed poor fairness manifesting in severe capture effect. The design of standard backoff protocol allows already successful nodes to remain successful, giving little chance to those nodes that failed to capture the channel in the beginning. With this at hand, we ask a conceptual question: Can one improve the performance of wireless backoff by introducing a mechanism of self-penalty, when overly successful nodes are penalized with big contention windows? Our real-life measurements using commodity hardware demonstrate that in many settings such mechanism not only allows to achieve better throughput, but also assures nearly perfect fairness.

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 3 Problem Resources in IEEE networks are allocated randomly with BEB The allocation scheme is largely unfair  The disparity is more prominent when stations are exposed in uneven environment (e.g., stations have different spatial positions) Can we improve resource allocation by Changing operation of IEEE backoff protocol Experimental evidence is missing

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 4 Standard backoff with modified backoff factors Increase contention window exponentially after each failure  CW=CW 0 r i-1  CW 0 =16  r=2 Up to i=7 retries before a frame is discarded Vary r for different number of stations, N

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 5 Penalty backoff Change the CW depending on whether the station is successful after first transmission attempt or not  If the station failed, continue with standard backoff protocol  If the station succeeded, assign largest contention window (CW=CW 0 r 6 ) for transmission of the next frame Vary backoff factor, r, depending on N Rational: By penalizing too successful stations, we increase the chances of unsuccessful stations to transmit the frames

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 6 Rollback backoff Reverse the standard backoff protocol:  Exponentially decrease CW on every failed attempt Vary backoff factor, r, depending on N Rational: Increase the odds of unsuccessful stations to access the channel by decreasing their waiting time

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 7 Backoff with fixed CW Assign all stations with a fixed CW  CW is not changed after failed or successful frame transmission Vary CW depending on number of stations, N, only Rational: If CWs are computed properly, channel access can be optimized leading to a better performance

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 8 Implementation Open wireless firmware (OpenFWWF project) Broadcom B43 wireless cards Implemented 4 aforementioned backoff protocols in firmware Changes to Linux kernel drivers More on changes made: Source codes and installation instructions:

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 9 Experimental environment Two testbeds used:  Wireless nodes are close (~1m) to the access point (idealized environment)  Wireless nodes are scattered (~17m from the access point) around the office (normal environment) A master node and 3 slave nodes (multiple wireless cards per slave node)  Wired connection for sending control traffic (e.g., calibration packets – discussed later)  Wireless connection for experimental traffic In total 12 wireless stations were used

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 10 Experimental Setup

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 11 Data collection and calibration 3 main traffic patterns used: bulky download and upload (from master node to slave nodes, and vice versa), delay sensitive UDP streams Tools used to generate traffic:  Wget (for upload from slave to master) and scp (for download from master to slave) to generate bulky streams  Custom “C” application for generating periodic UDP packets

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 12 Data collection and callibration (cont'd) Logging on slave nodes:  For every wireless card using printk and debugging statements introduced in wireless card driver we logged (on per frame bases): Transmission time Number of retries Acknowledgment flag (success or failure) Frame size Last used contention window and backoff interval  NOTE! printk uses ring buffer If not freed on time the logs get erased  Solution: Increased ring buffer size and dumped buffer every 0.1 second

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 13 Problem: How to merge logs from different slaves (note, clocks are not in sync for different slave machines) ?  Merged logs are essential to study such performance metrics as: Total throughput Total collision rate Fairness Solution: Send calibrating beacons with unique id (over wired link) from master node to all slave machines every 10ms and then realign the logs according to beacons Data collection and calibration (cont'd)

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 14 Data collection and calibration (cont'd) To validate the precision of beaconing, we have computed the distribution of their interarrival times and differences between interarrival times for all slave machines  Interarrival times turned to be sharply clustered around 10ms with rare outliers  The distribution of differences in interarrival times was clustered sharply around 0

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 15 Using the beacons we split the merged log into bins of 100ms In each bin we calculated (per wireless station and total):  Number of packets successfully transmitted  Number of failed transmissions  Total number of bytes transmitted For analysis we used only bins in which all stations where transmitting the packets:  We discarded all bins from the beginning of log file up to a bin in which all stations sent at least one packet  We discarded all bins from the end of the log file starting from bin in which one of the stations sent its last packet  We ensured that the trimmed log file was big enough providing statistically valid data Data collection and calibration (cont'd)

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 16 Experimental results: Idealized environment, bulky upload For all protocols we report:  Median throughput for different values of r (except for a backoff with fixed CW for which we report entire CDF)  Jain's fairness index  Median collision probability Observations:  Penalty and rollback backoff protocols deliver 145% and 77% better throughput in comparison to standard backoff protocol for optimal values of r  Penalty and rollback backoff protocols significantly decrease the collision probability and show considerable improvement in fairness  Backoff protocol with fixed CW does not have highest throughput and fairness

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 17 Experimental results: Idealized environment, bulky upload PenaltyFixed CW Throughput Fairness Standard

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 18 Experimental results: Normal environment Nodes are scattered around the office  ~15-16 meters away from the access point Observation: The trends observed in idealized environment repeat for all protocols Conclusion: Penalty and rollback backoffs deliver better performance in environment common to many real life deployments

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 19 Experiment with hidden stations Involved two wireless stations and an access point operating in normal environment:  Wireless stations are hidden from each other One station has slightly stronger signal Penalty and rollback increase the odds of accessing the channel for disadvantageous stations as r gets large enough (discussion can be found in Section VII in the paper)

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 20 Experiment with download traffic Wireless stations performing download from master node in normal environment Comparable throughput for penalty, rollback and standard backoff protocols achieved Penalty and rollback backoff show better fairness

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 21 Experiment with delay sensitive traffic 9 wireless stations perform bulky upload in in normal environment 1 station sends UDP packets every 10ms Observation: Penalty and rollback backoff protocols do not increase significantly delays in comparison to standard protocol

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 22 Mathematical model: Optimal values of r as function of N Goal: Find optimal values of backoff factor r as a function of number of active stations N  Solidify and corroborate empirical results  Are useful for dynamic backoff factor adaptation In realistic networks number of stations N can change frequently Mathematical derivations and techniques used can be found in the paper Theoretical vs. Empirical: optimal backoff factor values for different number wireless stations

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 23 Backoff factor adaptation algorithm: Metrics 1.We use two metrics to estimate the number of active stations: i.Threshold-based (simple) ii.Ratio-based (accurate) 2.Threshold-based metric: Count each station as active if it transmits longer than some threshold 3.Ratio-based metric: Count each station that fully saturate the channel, and aggregate the stations that do not fully saturate the channel Please see paper for more details about the metric and its properties

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 24 Two previous metric work well when there is AP AP coordinates the selection of backoff factors How to estimate number of active stations when no AP available, e.g., in mesh networks? Or when multiple APs, working at the same channel, present in the environment? Count idle slots as per IDLE SENSE Will not work when hidden stations present Can we use consensus algorithms between stations to figure it out? Future research direction! Backoff factor adaptation algorithm: Metrics (cont'd)

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 25 Backoff factor adaptation algorithm: Evaluation Implementation:  Metrics are implemented in hostapd access point  Introduced new management frame to convey optimal backoff factor to stations  Modified B43 driver to adapt the backoff factors of wireless card Experiment:  12 wireless stations: 6 stations follow On/Off pattern and generate 40KB UDP stream every 30 seconds during 30 seconds interval 6 stations constantly upload large file using TCP protocol

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 26 Deployment How to deploy our protocol? At least two incremental deployment possibilities exist! Approach 1: Implement fall-back mechanism: Similarly how a and b variants of coexist today! If at least one station that does not support modified backoff protocol attaches to network all attached nodes start to use standard backoff protocol Simple and doable! Approach 2: The protocol can be readily used in mesh networks: Use modified backoff for backbone links Use standard backoff for interacting with legacy clients Of course, backbone links and clients have to use different channels!

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 27 Conclusions Penalty and rollback protocols improve throughput and fairness Choose penalty over rollback backoff for slightly better performances in terms of throughput Choose rollback over penalty backoff for slightly better fairness If hidden terminals exist, penalty and rollback protocols increase the chances of a disadvantageous wireless station to access the channel Penalty and rollback backoff protocols do not increase the delays comparing to standard backoff protocols In practice the optimal backoff factors for penalty and rollback protocols can be efficiently computed and distributed by an access point

doc.: IEEE /0494r0 Submission May 2013 Dmitry Kuptsov, HIIT Slide 28 References D. Kuptsov, B. Nechaev, A. Lukyanenko, A. Gurtov, A Novel Demand- Aware Fairness Metric for IEEE Wireless Networks, Proc. of ACM SAC, March A. Lukyanenko, A. Gurtov, Performance analysis of general backoff protocols, Journal of Communications Software and Systems, 4(1), March A. Lukyanenko, E. Morozov, A. Gurtov, An adaptive backoff protocol with Markovian contention window control, in Communications in Statistics - Simulation and Computation, Volume 41, issue 7, D Kuptsov, B Nechaev, A Lukyanenko, A Gurtov, How Penalty Leads to Improvement: a Measurement Study of Wireless Backoff, arXiv preprint arXiv: