Discussion on Evaluation Metrics for 802.19.1 April 2009 doc.: IEEE 802.19-09/xxxxr0 Dec. 2009 Discussion on Evaluation Metrics for 802.19.1 Authors: Name Company Address Phone E-mail Tuncer Baykas NICT Yokosuka Japan tbaykas@ieee.org M.Azizur Rahman -- Stanislav Filin Zhou Lan Junyi Wang Yohannes Alemseged Chin Sean Sum Gabriel Villardi Junyi Song Chen Sun Ha Nguyen Tran Hiroshi Harada Notice: This document has been prepared to assist IEEE 802.19. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein. Tuncer Baykas, NICT Tuncer Baykas, NICT

April 2009 doc.: IEEE 802.19-09/xxxxr0 Dec. 2009 Abstract This presentation discusses evaluation metrics which can be used in 802.19.1 standardization process and included in evaluation methodology document. Tuncer Baykas, NICT Tuncer Baykas, NICT

Dec. 2009 Possible Metrics Complexity: What is the complexity per device and what is the setup complexity? Optimum throughput versus achieved throughput: Does the system find optimum throughput or stop in a local maxima? Total number of systems it can handle: Is it scalable? Stability: Under which circumstances, systems fluctuate? Information Requirement: How much information is necessary? Coexistence Overhead: How much radio resources is required by the coexistence system? (excluding the need of sharing the spectrum)(e.g. guard time, ) Protection from non802.19.1 systems: How much protection against non802.19.1 systems? Fairness: How fair is the system? Tuncer Baykas, NICT

Possible Metrics from [1] Dec. 2009 Possible Metrics from [1] Medium occupancy: For each network it is the total time that that network is transmitting divided by the total simulation time. Successful medium occupancy: For each network it is the total time that that network is transmitting, and those transmissions are received properly at the intended destination, divided by the total simulation time Hidden node probability: For each network it is the probability of experiencing hidden node interference. Exposed node probability: For each network it is the probability of experiencing exposed node therefore missing a transmission chance. Tuncer Baykas, NICT

Dec. 2009 Possible Metrics The uplink and downlink packet error rates (PER): Median and %90 percent PER should be provided. The uplink throughput and the downlink throughput: For each of the link in the network. For throughput it is useful to focus not only on the median value but also the 10% value, since that is the throughput value at which 90% of the all stations exceed that throughput. Therefore, only 10% of the stations have a throughput less than the 10% throughput value. The uplink latency and downlink latency: for each of the link in the network (from top-of-the-MAC to top-of-the-MAC) For latency it is useful to focus not only on the median value but also the 90% value, since this is the latency value at which 90% of the stations have a latency that is less than this value. Therefore, only 10% of the stations have a latency more than the 90% latency value. Tuncer Baykas, NICT

Conclusions We presented possible evalation metrics for 802.19.1. Dec. 2009 Conclusions We presented possible evalation metrics for 802.19.1. Next step should creating scenarios to evaluate different proposals. Tuncer Baykas, NICT

Dec. 2009 References [1] S. Shellhammer Coexistence metrics in the 3650 MHz frequency band IEEE P802.19-07/0020r3 (A set of coexistence metrics are proposed for evaluation of coexistence of 802.11y and 802.16h in the 3650 MHz frequency band) Tuncer Baykas, NICT