1. Understanding the Performance of Thin-Client Gaming 12011/5/11CQR 2011 / Yu-Chun Chang Yu-Chun Chang 1, Po-Han Tseng 2, Kuan-Ta Chen 2, and Chin-Laung Lei 1 1 Department of Electrical Engineering, National Taiwan University 2 Institute of Information Science, Academia Sinica

2. Outline Introduction Experiment methodology – Experiment setup – Performance metric extraction Performance evaluation Conclusion & future work 2011/5/112CQR 2011 / Yu-Chun Chang

3. Introduction (1/2) 3 Client Server User’s inputs Display updates Thin-client system 2011/5/11CQR 2011 / Yu-Chun Chang

4. Introduction (2/2) Motivation – To understand which performance metric is more sufficient for thin-client gaming Frame rate, frame delay, frame loss, and etc Challenges – Most thin-client products are proprietary Image compression, data-transmission protocol and display update mechanism 42011/5/11CQR 2011 / Yu-Chun Chang

5. Our focus 52011/5/11 QoE Perf. Metric Network Condition Server Client Thin-client program User Network Condition Server CQR 2011 / Yu-Chun Chang QoE Perf. Metric

6. Outlines Introduction Experiment methodology – Experiment setup – Performance metric extraction Performance evaluation Conclusion & future work 2011/5/116CQR 2011 / Yu-Chun Chang

7. Experiment Methodology 72011/5/11

8. Why Use Ms. Pac-Man? Move Pac-Man to eat pills and get the score Control through thin-client applications and move Pac-Man in the game of server – Good network condition: score ↑ – Bad network condition: score ↓ Score  Quality of Experience 2011/5/11CQR 2011 / Yu-Chun Chang8

9. Ms. Pac-Man & Bot 9 Ms. Pac-Man – Save score after the pacman ran out of 3 lives Bot: ICE Pambush3 (published in IEEE CIG 2009) – Java-based controller to move the pacman – Capture the screen of the game and determine the position of the pacman, ghosts, and pills NumberScore Pill22010 Power pill450 Ghost4200 (after eating power pills) 2011/5/11CQR 2011 / Yu-Chun Chang

10. Three thin-client systems – LogMeIn – UltraVNC – TeamViewer Network conditions 10 Network conditionSettings Network delay0 ms, 100 ms, 200 ms Network loss rate0%, 2.5%, 5% Bandwidth Unlimited, 600 kbps, 300 kbps 2011/5/11CQR 2011 / Yu-Chun Chang

11. Performance metric – Display frame rate – Frame distortion (MSE: Mean Square Error) Record game play as video files in 200 FPS 112011/5/11CQR 2011 / Yu-Chun Chang

12. Outlines Introduction Experiment methodology – Experiment setup – Performance metric extraction Performance evaluation Conclusion & future work 2011/5/1112CQR 2011 / Yu-Chun Chang

13. 132011/5/11 Thin Clients are Different! CQR 2011 / Yu-Chun Chang

14. Visual Difference Really Matters! 142011/5/11CQR 2011 / Yu-Chun Chang

15. Statistical Regression 15 Regression Model QoE (score) Independent factors Display frame rate Frame distortion 2011/5/11CQR 2011 / Yu-Chun Chang

16. Frame-Based QoE Model Linear model QoE = 16 Adjusted R-squared: 0.72 2011/5/11CQR 2011 / Yu-Chun Chang

17. Frame-Based QoE Model 17

18. Which Performance Metric is More Sufficient? QoE degradation – Optimal user’s QoE – user’s QoE predicted by model Frame rate is more sufficient! 2011/5/1118

19. Frame Rate and Network Conditions 19 Network Condition 2011/5/11 QoE Perf. Metric Server Client Thin-client program User CQR 2011 / Yu-Chun Chang

20. The Frame Rate Prediction Model Frame rate = app1, app2: dummy variables – LogMeIn : app1 = 1, app2 = 0 – TeamViewer : app1 = 0, app2 = 1 – UltraVNC : app1 = 0, app2 = 0 d: delay, l: loss rate, b: bandwidth dl, dt, du : delay of LogMeIn, delay of TeamViewer, delay of UltraVNC 2011/5/1120CQR 2011 / Yu-Chun Chang

21. The Frame Rate Prediction Model 2011/5/1121 Adjusted R-squared: 0.85 CQR 2011 / Yu-Chun Chang Delay of LogMeIn Delay of UltraVNC Bandwidth of LogMeIn Bandwidth of UltraVNC

22. Predicted Frame Rate 2011/5/1122CQR 2011 / Yu-Chun Chang Network delay Bandwidth

23. Which Thin-Client is Better? 23 Network Conditions 2011/5/11 QoE Perf. Metric Server Client Thin-client program User CQR 2011 / Yu-Chun Chang

24. Network-Based QoE Model QoE = 2011/5/11 Adjusted R-squared: 0.81

25. The Thin-Client with Best Performance o symbol: empirical network condition – 300 records collected by PingER project 2011/5/1125CQR 2011 / Yu-Chun Chang

26. Conclusions & Future Work Display frame rate and frame distortion are both critical to gaming performance on thin-clients LogMeIn performs the best among the three implementations we studied Future work – Add more thin-clients to see comparisons of performance – Design a generalizable experiment methodology for thin- client gaming with different game genres 2011/5/1126CQR 2011 / Yu-Chun Chang

27. Thank you for your attention! 2011/5/11CQR 2011 / Yu-Chun Chang27