1. DBLA: D ISTRIBUTED B LOCK L EARNING A LGORITHM F OR C HANNEL S ELECTION I N C OGNITIVE R ADIO N ETWORKS Chowdhury Sayeed Hyder Department of Computer Science & Engineering Michigan State University - Chowdhury Sayeed Hyder, and Li Xiao

2. Outline Background ◦ Cognitive Radio Network Channel Selection Problem Distributed Block Learning Algorithm ◦ Decision Period ◦ Channel Ranking ◦ Channel Switching Simulation Results ◦ Regret ◦ Switching cost wowmom 2012 2

3. Background Figure: Current Spectrum Allocation in US Figure: Underutilized Spectrum Ref: Akyildiz, I., W. Lee, M. Vuran, and S. Mohanty, “NeXt Generation/ Dynamic Spectrum Access/ Cognitive Radio Wireless Networks: A Survey”, Computer Networks 2006 wowmom 2012 3

4. Background Current Status ◦ Spectrum Scarcity ◦ Underutilized spectrum Cognitive radio (CR) ◦ Adapt its transmission and reception parameters (frequency, modulation rate, power etc.) Cognitive Radio Network ◦ Two types of user  Primary user or licensed user (PU)  Secondary user or opportunistic user (SU) ◦ Requirements  SU cannot affect ongoing transmission of PUs  Must vacant the spectrum if PU arrives wowmom 2012 4

5. Problem Statement Channel Selection Problem ◦ Unknown PU activity ◦ Time varying channel condition ◦ Channel switching is not free! Learning algorithm (exploration exploitation) Our goal is to design a distributed learning algorithm that minimizes regret, minimizes switching cost, and adapts to time varying channels. wowmom 2012 5

6. Problem Statement 6 wowmom 2012 The expected regret following policy ρ ^ Difference in reward between optimal channel selection and channel selection by any learning algorithm Switching regret

7. The expected reward following optimal policy ρ The expected reward following centralized policy ρ cent The expected reward following distributed policy ρ dist Problem Statement 7 wowmom 2012

8. Problem Statement Switching regret ◦ # number of switching x unit switching cost ◦ Defined as the number of packets could have been transmitted within the time if it did not switch that channel. ◦ Unit switching cost switching delay Estimated packet transmission time 8 wowmom 2012 Ref: Y. Xiao and F. Hu, Cognitive Radio Networks, CRC press, 2008 =

9. Problem Statement 9 wowmom 2012 The expected regret following centralized policy ρ cent The expected regret following distributed policy ρ dist

10. Distributed Block Learning Algorithm Formulate the channel selection problem as multi arm bandit problem with multiple play and switching cost. Present a distributed ‘block’ approach where each user selects channel independently ◦ Decision period (when) ◦ Channel Ranking (on what) ◦ Channel Switching (why) ◦ Channel Adaptation (how) 10 wowmom 2012

11. Decision Period Block and frame: ◦ Timeslots are arranged in blocks, blocks are in frames. ◦ Block length increases linearly, frame length increases exponentially with frame number ◦ All blocks in a frame are of equal length 11 wowmom 2012

12. Channel Ranking Channel ranking based on ◦ Time average statistics  What we already got from the channel ◦ Upper bound statistics  What we expect from the channel 12 wowmom 2012

13. Channel Switching Only one channel is compared with the current channel (round robin) at the decision period Channel switching rule ◦ If the candidate channel has higher expectation than the current one. ◦ If the current channel is not in the top rank 13 wowmom 2012

14. Channel Adaptation Opportunity cost ◦ Increase the expectation of other channels if the idle rate of the current channel is not consistent with its overall idle rate. ◦ Increases the probability of switching 14 wowmom 2012

15. Simulation NS2 Channels’ idle probability follows Bernoulli distribution Number of channels: 9 Number of users: 4-8 Time slots: 50000 Unit switching cost: 0.5 15 wowmom 2012

16. Results (Regret) 16 wowmom 2012 Normalized Regret vs. time (with and without switching cost) ρ rand: A. Anandkumar, N. Michael, and A.Tang. “Opportunistic Spectrum Access with Multiple Users: Learning Under Competition, INFOCOM 2010 DBLA outperforms RAND in terms of regret minimization

17. Results (Scalability) 17 wowmom 2012 In the case of RAND, regret increases exponentially while in the case of DBLA, Rate of change in regret is almost linear.

18. Results (switching) 18 wowmom 2012 Regret vs. switching cost# of Switching vs. # of users DBLA has much less regret and less number of switching compared to RAND

19. Results (adaptability) 19 wowmom 2012 Channels idle probability changes at each 10000 slots

20. Conclusion & Future Work Learning algorithm to rank channels which ◦ minimizes regret ◦ minimizes switching ◦ is scalable ◦ adapts to dynamic channel condition Future Work ◦ More realistic channel model ◦ Theoretical proof analysis for upper bound wowmom 2012 20

21. Questions ?