1. doc.: IEEE 802.11-13/0545r0 Submission May 2013 Veli-Pekka Ketonen (7signal)Slide 1 WLAN QoE, End User Perspective Opportunities to Improve Date: 2013-05-12 Authors:

2. doc.: IEEE 802.11-13/0545r0 Submission May 2013 Slide 2Veli-Pekka Ketonen (7signal) Abstract Presentation for 802.11HEW SG in Hawaii May 12 th -17 th Interim meeting Includes data from various anonymous networks from 7signal Sapphire automated client device QoE measurements for performance optimization and management work Bottlenecks in current networks and suggestions for improvements are presented as background for further HEW SG work

3. doc.: IEEE 802.11-13/0545r0 Submission The Key Challenge May 2013 Slide 3Veli-Pekka Ketonen (7signal)

4. doc.: IEEE 802.11-13/0545r0 Submission In live networks, high max performance does not translate to sufficient user experience* Capacity, range and end user experienced quality do not any more meet needs Example from University. Nw. vendor has less impact, network config. higher impact * = these are hourly average values from an area of multiple APs/SSID over a week time Veli-Pekka Ketonen (7signal)Slide 4

5. doc.: IEEE 802.11-13/0545r0 Submission Key findings May 2013 Slide 5Veli-Pekka Ketonen (7signal)

6. doc.: IEEE 802.11-13/0545r0 Submission #1. Too aggressive rate control, average retrys often exceed 50% (1/2) May 2013 Slide 6Veli-Pekka Ketonen (7signal) With 802.11n products, regularly 30-50% of packets require at least one retry. Often even more. Too high rate selected A lot of retries, multiplied by MIMO-X factor High utilization Lower SNR More radio retries, up to 7 times/packet No air time & lost packets Capacity and TCP throughput collapse

7. doc.: IEEE 802.11-13/0545r0 Submission #1. Too aggressive rate control, less aggressive has been proven better (2/2) May 2013 Slide 7 Veli-Pekka Ketonen (7signal) Individual network behavior impacts also all other nearby networks 802.11ac has much more demanding requirements than 802.11n Suggestions: Use clearly less aggressive rate control Make rate control dynamic, adjusting based on observed radio conditions (like continuous Bluetooth) 802.11 to specify proper rate control schemes, instead of leaving this to vendors RX-level during this part of the route: -70 – -88 dBm Original link adaptation Optimized link adaptation -> Reason for improvement: Using more MCS-8 instead of MCS-9 RX-level Past findings from Nokia Networks EDGE link adaptation (Public, Ref [1]) Throughput

8. doc.: IEEE 802.11-13/0545r0 Submission #2 Automated channel control algorithms need clear improvements Automated features, like channel selection do not work properly Continuous channel hopping in the whole network and no stable state Impacts also surrounding networks Suggestion: 802.11 to specify more in more detail requirements May 2013 Slide 8Veli-Pekka Ketonen (7signal)

9. doc.: IEEE 802.11-13/0545r0 Submission #3. Already available radio settings are not utilized since their impacts are not understood Radio should more accurately and dynamically operate its settings, like –Data rates; supported, default, control –Management/control traffic data rates, –Fragmentation process, MTU –QoS –Ack/block ack schemes usage –Long/short pre-amble configuration –RTS/CTS process –Supported 802.11 standards –Minimum limit for probe response –Load balancing, etc Suggestions –Performance management practices –Automated operation –802.11 to specify in more detail May 2013 Slide 9Veli-Pekka Ketonen (7signal)

10. doc.: IEEE 802.11-13/0545r0 Submission #4. Interference due to lacking channel coordination and Bluetooth devices Channel plans are almost random in public areas –Resulting packet loss, jitter Proper radio operation in the mid- term still requires significantly better channel plans Suggestions –Better, proven automated algorithms for channel negotiation –Cloud based control –Regulation for allowed channels –Better industry defaults –Adhoc channel selection limited/guided May 2013 Slide 10Veli-Pekka Ketonen (7signal)

11. doc.: IEEE 802.11-13/0545r0 Submission #5. Too dense beacons load air unnecessarily In practice solely 100ms used globally with 1 Mbit/s as mandatory rate in consumer grade APs. This congests air significantly everywhere In 100ms, a person walking full speed moves ~ 10 inches ( ~ 25cm). Is this dense beaconing necessary? May 2013 Slide 11Veli-Pekka Ketonen (7signal) Suggestions Define default beacon intervals longer, ~ 300ms Dynamic/adaptive beaconing, beacon interval automatically dependent on the observed time between roaming. Consider add few % variance to beacon intervals to avoid continuously repeating collisions (compare to spread spectrum CPU clocking for EMI reduction) Consider impact to power save functionality

12. doc.: IEEE 802.11-13/0545r0 Submission #6. Mobile networks interfere 2.4 GHz band WLANs through 3 rd harmonic distortion When cellular network indoor antennas are near (30ft/10m) to WLAN APs and/or clients, they may saturate the receiver with off band signals and receiver generates distortion product that lands in the 2.4 GHz band Suggestion –Add mandatory RF band-pass filtering to WLAN radios –Receiver blocking test to FCC approval May 2013 Slide 12Veli-Pekka Ketonen (7signal) 2.4 GHz2.3 GHz2.2 GHz2.5 GHz2.6 GHz2.1 GHz2.0 GHz1.9 GHz1.8 GHz1.7 GHz DCS-1800 (EUR, US) PCS-1900 (EUR) UMTS-1900 (US) UMTS-2100 (EUR) UMTS-1700 (US) Distance appr. 300MHz => Harmonic distortion lands at 300MHz distance from source WLAN signal High power mobile base station signal High power mobile base station signal Ghost signal (noise) => Signal-to-noise ratio degrades in WLAN receiver and data transfer suffers Verified to happen in live network conditions

13. doc.: IEEE 802.11-13/0545r0 Submission #7. Support for legacy devices (802.11b/a) seriously degrades benefits of new standards Protection mode is “contagious” and highly inefficient Benefits of new standards are limited if legacy devices are overprotected. Important especially in consumer grade equipment. Suggestions –Better industry defaults 802.11b not supported 802.11a not supported –Improvements to protection mode –Prevent/limit protection mode spreading with required minimum signal levels May 2013 Slide 13Veli-Pekka Ketonen (7signal)

14. doc.: IEEE 802.11-13/0545r0 Submission #8. Lacking interoperability may take down whole network performance Introduction of new radio devices increased average retry rates to about 70%. Max network capacity came down at least 50% May 2013 Slide 14Veli-Pekka Ketonen (7signal) Suggestions –More exact requirements needed for client-AP interoperability –Live network performance management capabilities need to improve. All scenarios cannot ever be tested upfront.

15. doc.: IEEE 802.11-13/0545r0 Submission #9. Modest access point antenna solutions Omni-antennas with significant vertical coverage are widely used RF energy goes where it should not go and antennas try to receive it from directions where are no clients Lacking antenna sophistication –More gain towards users would benefit uplink quality –Lack of antenna directivity creates more interference Suggestions –Down-tilt beam patterns Fixed, “normal” antennas Electrically adjustable, like in mobile networks –Wider use of beam steering May 2013 Slide 15Veli-Pekka Ketonen (7signal)

16. doc.: IEEE 802.11-13/0545r0 Submission #10. Performance Management is completely missing With WLAN networks, commonly accepted fact is that: “It is not necessary to continuously know what kind of service end users get from the network. If we manage to make it work once, there is no need to look after performance for several months/a year. It will take care of itself automatically. We will troubleshoot when end users complain” This approach fundamentally prevents WLAN becoming a reliable media Mobile operators/telecom industry are used to manage networks based on Key Performance Indicators, KPIs. These are covered also in standards. This is a good practice that should be brought to WLAN networks Beyond technology providing the required solutions, data and services, even bigger change is required in attitudes. May 2013 Veli-Pekka Ketonen (7signal)Slide 16

17. doc.: IEEE 802.11-13/0545r0 Submission 2-10x improvement available with these already Results: Controller Automation vrs First Manual Optimization Round University campus, dense WLAN network 2.4 GHz downlink throughput improvementImprovement –Area 1 7Mbit/s vrs. 25Mbit/s (+250%) –Area 2 5Mbit/s vrs. 15Mbit/s (+200%) –Area 3 8Mbit/s vrs. 16Mbit/s (+100%) 2.4 GHz uplink throughput improvement –Area 1 7Mbit/s vrs. 20Mbit/s (+180%) –Area 2 10Mbit/s vrs. 25Mbit/s (+150%) –Area 3 12Mbit/s vrs. 20Mbit/s (+65%) 2.4 GHz downlink Voice Quality (MOS grade, max 4.0) improvement –Area 1 2.6Mbit/s vrs. 3.5Mbit/s (+0.9 MOS) –Area 2 2.9Mbit/s vrs. 3.8Mbit/s (+0.9 MOS) –Area 3 2.5Mbit/s vrs. 3.5Mbit/s (+1.0 MOS) 2.4 GHz uplink Voice Quality (MOS grade, max 4.0) improvement –Area 1 3.5Mbit/s vrs. 3.8Mbit/s (+0.3 MOS) –Area 2 3.5Mbit/s vrs. 3.9Mbit/s (+0.4 MOS) –Area 3 2.5Mbit/s vrs. 3.5Mbit/s (+1.0 MOS) 2.4 GHz jitter daily averages before vrs. after –Area 19% vrs. 1% (- 89%) –Area 2 9% vrs. -90%) –Area 37% vrs. 1% (-85%) Hourly minimum measured downlink throughput values increase 10X –Area1, Area 30.2 Mbit/s vrs. 2.5 Mbit/s ( ~ 1100%) 17 Veli-Pekka Ketonen (7signal.) Slide 17

18. doc.: IEEE 802.11-13/0545r0 Submission May 2013 Slide 18 Areas looking for improvements based on data from current installations 1. Too aggressive rate control 2. Automated channel control algorithms need clear improvements 3. Already available radio settings are not utilized since their impacts are not understood 4. Interference due to lacking channel coordination and Bluetooth devices 5. Too dense beacons load air unnecessarily 6. Mobile networks interfere 2.4 GHz band WLAN’s through 3 rd harmonic distortion 7. Support for legacy devices (802.11b/a) seriously degrades benefits of new standards 8. Lacking interoperability may take down whole network performance 9. Modest access point antenna solutions 10. Performance Management is completely missing Veli-Pekka Ketonen (7signal)

19. doc.: IEEE 802.11-13/0545r0 Submission References [1] Nokia Networks/Jussi Nervola: Optimization of EGPRS Link Adaptation, 2007/01/16 –M.Sc. Thesis seminar presentation –http://www.netlab.tkk.fi/opetus/s38310/06-07/Kalvot%2006-07/nervola_160107.ppthttp://www.netlab.tkk.fi/opetus/s38310/06-07/Kalvot%2006-07/nervola_160107.ppt Network statistics from 7signal network optimization and performance assurance reports May 2013 Slide 19 Veli-Pekka Ketonen (7signal)

20. doc.: IEEE 802.11-13/0545r0 Submission ADDITIONAL SUGGESTIONS May 2013 Slide 20 Veli-Pekka Ketonen (7signal)

21. doc.: IEEE 802.11-13/0545r0 Submission Other input for SG work May 2013 Veli-Pekka Ketonen (7signal)Slide 21 TopicDescription Client power controlIn dense networks AP’s have often lower power levels than clients. High amount of high power clients with a lot of traffic (data, control or management) take a lot of air time and cause unnecessary interference. Prevent hiding SSID’sHidings SSID’s increases utilization. All beacons are anyway sent, just without SSID name. Without SSID names in beacons, Clients need to probe continuously. Remote possibility of hiding SSID names as it provides so little value. Alternatively implement so that beacons are not sent at all is SSID is hidden. Improve DFS implementationDFS operation is commonly reported to be overly sensitive and prevents using DFS channels in many areas. This needs to change with.ac and.hew to allow efficient use of 5 GHz band. Improve client roaming behaviorSome clients use excessive off channel scanning. This sometimes results as QoS Null frame flood that takes air time and increase congestion. Use adaptive beaconingWhen there are no relevant clients nearby for a longer time, transmit beacons less often. This reduces overall spectrum load in the surrounding areas remarkably. Currently APs beaconing (usually 1 Mbit/rate & 100ms interval) consume a lot of spectrum day and night for no good purpose. Consider ways to work around the power save mode operation. Use dynamic fragmentationClient traffic could start with a smaller packets (with fragmentation) at higher data rate (less airtime, less interference). Once rate control has a good grip of proper rates for that client packet flow, fragmentation could be gradually removed. Rate control is slow and needs some time to adapt. In addition to lowering retries, utilization and interference, this would allow some time for rate control to work better. Beacons and probes should use higher data rates Beacons (and probes/probe responses) should not be transmitted with the low data rates. By using higher data rate there, the nearby clients (the only ones relevant to beaconing process) can receive them and the devices further away not. Rate control still should be able to move to lower data rates when really needed. This would be easy way to reduce utilization further away from AP. Cloud control: Radio Link Test and Radio Link Control interfaces to APs For automated central coordination of a large group of individual APs, there should be two new functionalities in APs. Radio Link Test interface: Without authenticating to AP/network, an authorized (password) wireless device should be able to run some basic active measurements against the AP. These include at least FTP, UDP and ICMP traffic flows. Radio Link Control interface: Without authenticating to AP/network, an authorized (password) wireless device should be able to control basic Radio settings in AP. These include at least radio channel and power level. Cloud control: With these interfaces, especially dense consumer AP installations (& SMB) could be centrally controlled and managed to provide optimum quality for everyone. Improved radio MAC based device type classification Current MAC address based device identification should be expanded. Currently only manufacturer names are recorded. This would allow developing better automated/by-default-on QoS control algorithms to infrastructure. Automated “closed loop” radio network control, adaptive/SON Optimally, wireless network should reconfigure automatically (SON, Self Organizing Networks). So far in WLAN results are modest. Channel change algorithms degrade performance, rather than improve it. Networks have to make decisions and learn from impact of changes continuously. Accurate data of end user experience is needed for this to work properly. Comprehensive data collection and analytics should drive this process.

22. doc.: IEEE 802.11-13/0545r0 Submission DATA COLLECTION BACKGROUND May 2013 Slide 22 Veli-Pekka Ketonen (7signal)

23. doc.: IEEE 802.11-13/0545r0 Submission Background 7signal utilizes its products to optimize and operate critical WLANs (hospitals, universities, enterprises, manufacturing) This process includes continuous collection and analysis of large amount of performance data, consisting of over 600 different metrics. Optimization takes place by reconfiguration and pre-emptive changes on network based on the data. Data includes –Automated client device tests, providing L1-L7 data –Passive L1-L2 packet statistics of all 802.11 air traffic –RF environment data –Spectrum analysis data May 2013 Slide 23Veli-Pekka Ketonen (7signal)

24. doc.: IEEE 802.11-13/0545r0 Submission 7signal Sapphire consists of three elements Monitor Measure Record Report Alarm Analyze Troubleshoot Verify Sonar test servers are located in in close proximity to application servers Centrally located Carat manages Eyes, provides reports and alarms. Includes Analyzer software One Eye unit manages 4-7access points (indoors) Radio analysis, radio packet capture and end-to-end application measurement Slide 24

25. doc.: IEEE 802.11-13/0545r0 Submission 7signal Sapphire data covers all layers 1-7 Synthetic tests (L1-L7) FTP, PING, HTTP, DHCP, SIP, VOIP Association/authentication/IP address/test success rates, delays, throughput, latency, jitter, packet loss, MOS, data rates, failure codes 60 performance indicators, separately for each AP/SSID/Sonar pair Synthetic tests (L1-L7) FTP, PING, HTTP, DHCP, SIP, VOIP Association/authentication/IP address/test success rates, delays, throughput, latency, jitter, packet loss, MOS, data rates, failure codes 60 performance indicators, separately for each AP/SSID/Sonar pair RF analysis (L1-L2) Access point settings and capabilities, signal levels, channels, noise levels 40 performance indicators, separately for each AP, channel, antenna beam RF analysis (L1-L2) Access point settings and capabilities, signal levels, channels, noise levels 40 performance indicators, separately for each AP, channel, antenna beam Traffic analysis (L2) Radio frame header analysis for traffic flow between clients and access points Data rates, retry rates, air congestion, roaming, frame size, device vendor, statistics for all 802.11 frame types, reason codes and status codes 500 performance indicators, separately for each client, SSID, AP, band, antenna beam Traffic analysis (L2) Radio frame header analysis for traffic flow between clients and access points Data rates, retry rates, air congestion, roaming, frame size, device vendor, statistics for all 802.11 frame types, reason codes and status codes 500 performance indicators, separately for each client, SSID, AP, band, antenna beam Spectrum analysis (L1) High resolution (280kHz) spectrum analysis for ISM band Historical data over months, interference source analysis with beam steering, compass direction data on beams Spectrum analysis (L1) High resolution (280kHz) spectrum analysis for ISM band Historical data over months, interference source analysis with beam steering, compass direction data on beams Troubleshooting tests (L1-L7) Remote, manual process for troubleshooting purposes Full array of tests may be scheduled manually to each Eye Eyes may be assigned to perform the additional tests without interrupting automated monitoring process Troubleshooting tests (L1-L7) Remote, manual process for troubleshooting purposes Full array of tests may be scheduled manually to each Eye Eyes may be assigned to perform the additional tests without interrupting automated monitoring process Full packet capture (L1-L2) Remote, manual process for troubleshooting Full blown remote packet capture and easy export to packet level analyzer like Wireshark, in case individual radio packet header content information is needed Eyes may be assigned to perform the test without interrupting automated monitoring process Full packet capture (L1-L2) Remote, manual process for troubleshooting Full blown remote packet capture and easy export to packet level analyzer like Wireshark, in case individual radio packet header content information is needed Eyes may be assigned to perform the test without interrupting automated monitoring process Slide 25

26. doc.: IEEE 802.11-13/0545r0 Submission 7signal Sapphire Eye, the data collection device A “turbo charged” client device At times active like end users, at times fully passive Beam steering technology with low noise amplifiers Integrated compass 802.11 a/b/g/n support High resolution ISM band spectrum analyzer High RF performance design; maximized coverage, minimized quantity, typically 4-7 AP’s per unit Standard PoE Neutral design and white color Attaches easily to ceiling grid, alternatively wall or pole Indoor and outdoor versions Data analyzed at device, only key results to database Minimal load to network, small test packets at determined intervals Slide 26

27. doc.: IEEE 802.11-13/0545r0 Submission 7signal view on QoS, similar to end users 300x 200x 100x WLAN radio Network 900x 5x Access Point WLAN controller LAN wired network Core switching network 100x Core Router 100x Application servers Site broadband connection LAN switch 2x 8x Core switch Server racks Servers 10x End user terminals 7signal Sonar software 7signal Eyes Active end-to-end quality of service assessment from end user perspective End user device quality of service in radio network Radio network and spectrum analysis Slide 27

28. doc.: IEEE 802.11-13/0545r0 Submission Network Optimization Flow Utilize especially “Active/Synthetic” Eye measurements (“automated end user”) Optimize until target levels have been achieved 1. Ensure that WLAN/LAN/WAN is capable providing high quality service “Passive”, client and AP level measurements Optimize network and configure clients to achieve this 2. Make sure that all clients can utilize the network properly Follow Key Performance Indicators (KPIs) and SLA tables and take actions Proactive corrections are needed when metrics degrade 3. Maintain performance at the target level over the time Slide 28