1. Bottom-Up Evaluation of 802.11 Performance Testing Dr. Michael D. Foegelle ETS-Lindgren September 2004 Foegelle, ETS-Lindgren doc.: IEEE 802.11-04-1156-01-000t Submission

2. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 2 Overview Introduction Layer Analysis Basic Operational Testing Environmental Effects Summary & Conclusion

3. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 3 Introduction Top-down approach of defining usage cases and working down to resulting metrics and needed tests is a useful starting point. However, tendency to focus on usage case scenarios is hindering determination of wireless performance parameters needed in order to progress towards development of equivalent laboratory techniques.

4. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 4 Introduction This presentation will use a bottom-up approach to identify and evaluate critical contributions of the wireless portion of the DUT, concentrating primarily on the physical layer/radio effects.

5. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 5 Layer Analysis In the ISO 7-layer network model, only the bottom two layers (Data Link and Physical) have anything to do with the device being wireless as opposed to any other network device.

6. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 6 Layer Analysis The tests necessary to evaluate the upper layers should not differ from testing performed on non-wireless network devices. –Only indirect contribution of lower levels make this any different from non-wireless devices.

7. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 7 Layer Analysis Effects of environment (location, interference, etc.) directly affect performance of physical layer, which in turn affects the MAC layer and so on, all the way up the chain. Analysis of environmental effects on the physical layer, starting at the radio, will show that it is possible to determine well defined laboratory techniques to duplicate these effects.

8. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 8 Basic Operational Testing Before we talk about performance testing, there’s a level of testing required to verify that the device is actually operational. Radio functionality –Ability to generate required signals and modulations at expected levels. –Ability to detect/decode input signals in a nominal range.

9. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 9 Basic Operational Testing Additional PHY layer functionality –Ability to encode packets and handle shared media. MAC layer functionality –Ability to establish connection, perform handshaking, and transfer data. Some level of standardized functionality testing is assumed prior to attempting performance testing.

10. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 10 Environmental Effects: Distance The simplest environmental effect to evaluate is that of distance on direct line-of-sight communication. Signal level drops as distance increases or as signal passes through intervening obstructions. –Relative phase relationship of signal remains constant over time. –No actual degradation of signal content.

11. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 11 Environmental Effects: Distance Implies evaluation of basic radio performance (transmit power and sensitivity) –Ability to detect a coherent signal –Ability to demodulate/decode a given complexity of modulation (data rate control) at a given signal level –“Clean” signal – No interference Secondary effect of propagation delay unlikely to be significant for range of 802.11 –~0.33  S at 100 meters

12. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 12 Environmental Effects: Distance This testing can easily be performed as a conducted test using a calibrated radio and a reference 802.11 interface. –Reference interface provides known good communication behavior –Calibrated radio provides traceable: Power and signal quality measurements of DUT transmit signal at various data rates. Variable forward power received by the DUT to test sensitivity and data rate control/feedback.

13. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 13 Environmental Effects: Multipath Multipath effects can be sub-divided and investigated separately. Multipath fading in its traditional sense refers to the reduction in received signal due to the location of the receive antenna in a field null. –Null caused by the summation of one or more out-of-phase signals with similar amplitudes. –Dynamic fading refers to the effect seen when motion causes the receiver to move through nulls.

14. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 14 Environmental Effects: Multipath The signal level effects of fading (i.e. similar path- length, out-of-phase signals) would have a similar effect on performance as that for distance. –Since fading (null location) is frequency dependent, spread spectrum signals offer some rejection to fading. –Accurate test for fading would entail use of variable notch filters (fading simulators) to create a frequency dependent fade behavior. Testing for dynamic fading would entail determining the device’s response to dynamically changing faded signals.

15. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 15 Environmental Effects: Multipath The term fading refers to audio behavior of traditional amplitude modulation. –As signal amplitude fades, so does audio amplitude. Sound fades out. –At audio frequencies, modulation effects of multipath are negligible. At high data rates, modulation rates approach path lengths (11 MHz chip rate ~ 27 meters), so modulation corruption due to varying multipath lengths becomes an issue.

16. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 16 Environmental Effects: Multipath Worst cases of modulation corruption likely for two similar strength, in-phase signals with considerably different path lengths. –E.g. LOS through wall (lossy) and NLOS from distand reflector –Results in strong signal for portions of some bits, nulls for others (dependent on phase modulation at given bits). SS demodulation techniques are designed to decode through multipath effects, but suitable test cases are desirable.

17. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 17 Environmental Effects: Multipath Again, behavior can be simulated through cabled test. –Brute force: split signal and use delay lines and attenuators to create a true multi-path input. –Custom phase locked transmitters capable of generating identical signals with desired delay and amplitude variations. Note that multipath tests need only be receiver side tests.

18. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 18 Environmental Effects: Antenna In general, the antenna is a modifier to the RF performance previously discussed. –An antenna simply integrates (or adds up) the various signals directed towards it, with a weighting factor as a function of both propagation and polarization directions. Directionality can change magnitude of multipath effect seen by radio by minimizing contribution of other paths, but does not change radio’s innate ability to handle a given multipath effect. –Does interact with circuitry and nearfield environment, changing performance (see 11-04-0675) Radiated over-the-air tests need only verify radio TX/RX performance with antenna loading and interaction. -- No need to perform all tests OTA.

19. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 19 Environmental Effects: Antenna Diversity performance should be able to be determined primarily from conducted test of a dual/multi-port device using simulated multipath signals. –Antenna/OTA performance is a modifier to the signal levels at each antenna. –Statistical problem related to antenna positions and input signals. –Would be possible to simulate an OTA multipath for diversity (fading elimination) by using dual measurement antenna feeds –Reverberation environments are available to simulate worst case multipath behavior quickly.

20. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 20 Environmental Effects: Antenna Smart antennas add a new level of complexity to antenna testing. –Antenna pattern can be measured for each possible smart configuration or at specific points in a continuum to create a statistical envelope. –More difficult to interpret performance as a function of variation, however. Time period and automated decision process of antenna adjustment interacts with other behaviors. Integrated antenna/circuitry may make cabled tests difficult.

21. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 21 Environmental Effects: Network Once an 802.11 device becomes part of a wireless network with multiple endpoints, there are a variety of interactions relating to both the PHY and MAC layers that should be evaluated. –These include physical effects: Out of band signals and tolerances Collisions, cross-talk, etc. Hidden endpoints & network overlap. –And protocol related effects: Timing, holdoff, etc. Multi-point access control Handshaking

22. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 22 Environmental Effects: Network There are a number of “network stress tester” type products on the market designed to simulate multiple devices on a network. –These products typically simulate a multitude of MAC layers operating through one radio. Single radio approach can’t simulate real world collisions and other RF interaction behaviors. Some level of synchronization already exists above the PHY layer to allow unit to operate. A multi-radio device is needed to properly simulate real-world network behavior.

23. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 23 Environmental Effects: Interference Interference testing may be more of a functional test than a performance test. –Any properly operating radio/PHY layer should expect to have similar susceptibility to a given level and type of interference. –Outside interference is not a predictable event. Can simulate a wide variety of interference, but the resulting metric is not likely to represent any particular real-world behavior. –As much EMC as operational or performance related.

24. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 24 Summary & Conclusion This is an incomplete list intended to get the task group thinking in terms of the input parameters to overall performance metrics. It’s not necessary to set up real-world networks to simulate real-world effects. Most tests can be done with a cabled connection to appropriately designed test and measurement equipment or systems. Any number of the metrics discussed by the TG may be evaluated, as needed, but they should be evaluated in terms of the factors that contribute to variation in each level of the 802.11 layer structure.

25. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 25 Summary & Conclusion Many metrics may have the same interaction with the available input parameters. It may not be necessary to test all conceivable metrics as a function of these parameters. We need to avoid getting caught up in upper level metrics that may not change due to the use of a wireless network as opposed to any other network. We should pick a starting point and develop some experimental test plans around currently available equipment for simulating the wireless effects.

26. doc.: IEEE 802.11-04-1156-01-000t Submission September 2004 Foegelle, ETS-LindgrenSlide 26 Summary & Conclusion Point-to-point behavior is easy to understand and should be an initial starting point for radio/PHY evaluation. Network stress testers are a good starting point for MAC level testing. Additional effects and interactions may be defined and investigated as we proceed, building on what we learn with the simpler tasks. New test equipment or combined rack systems will need to be developed to carry this effort forward.