Experiences with Client-based Speculative Remote Display John R. Lange & Peter A. Dinda Department of Electrical Engineering and Computer Science Northwestern University Sam Rossoff Department of Computer Science University of Victoria Prescience Lab

2 Speculative Remote Display Remote display interactivity fundamentally constrained by network latency –Critical for WANs Speculation can improve interactivity –Can be done without server support Significant predictability exists in remote display protocol streams Speculation can be introduced into remote display clients –User-controlled tradeoff between latency and display errors –User studies currently give mixed results for tradeoff

3 Outline Speculative Remote Display Predictability study of Remote Display protocol streams –RDP and VNC Design and implementation of a client- based speculative remote display system –VNC-SRD User study of VNC-SRD –Examination of user-controlled speculation Conclusions

4 Remote Display Systems Historically popular in many forms –VNC, RDP, THINC, Citrix, X –Thin clients, remote administration, mobile users, etc Implemented at all levels of the stack –Virtual hardware, device drivers, window system extensions –Large variance in semantic properties of the protocols Lots of work has gone into optimizing Remote Displays –Predominantly to decrease bandwidth requirements

5 Latency vs. Bandwidth Optimization Large amount of work in optimizing bandwidth usage of Remote Displays –Compression/encoding, caching, protocol design Bandwidth is important… But so is interactivity User interaction implies waiting for server updates –Network latency is a critical component

6 Prospects for Speculative Remote Display User Events Screen Events Can we predict screen events well? If so, can we speculatively execute screen events before server responds? (Windows RDP)

7 Optimizing Latency Through Speculation Speculation can improve interactivity –Display screen events before they arrive RTT is no longer limiting factor UI Interaction intuitively appears to be predictable –Menus, dialog boxes, etc –Often tied to specific user action

8 Questions Is there predictability? How do we design and implement such a tool –Client –Client/Server –Predictor –… Can users tolerate display errors? Can users control tradeoff between display errors and interactivity?

9 Analysis of Predictability Effectiveness of speculation requires predictable event sequences Examined predictability with state-limited kth order Markov models –Simple –General prediction symbols Unique event signatures as symbols –Can provide multi step ahead predictions

10 Trace Collection Instrumented versions of rdesktop and VNC User studies –Northwestern students & faculty participants 5 users for each protocol –Single study model Full screen display of remote Windows machine Private local network Standardized task sequence Event traces from typical Windows workloads –Word, PowerPoint, Web browsing –15 minute periods –VNC: 12K-24K user events, 8.5K-17K screen events –RDP: 47K-77K user events, 712K-1M screen events

11 Analysis of Predictability Generated symbols from event sequences Ran symbol streams through Markov model –Model is continuously being updated, just as in system we later built –Screen->Screen, Screen+User->Screen –Different model orders considered –Different constraints on state space size considered Analyzed one-step-ahead predictability –Percentage of predictions that were correct Predictor Accuracy –Percentage of total events correctly predicted Protocol Predictability

12 RDP Prediction Accuracy Screen+User -> Screen

13 RDP Predictability Screen+User -> Screen

14 VNC Prediction Accuracy Screen+User->Screen

15 VNC Predictability Screen+User->Screen

16 Implications RDP displays a high degree of predictability –Screen events are highly correlated RDP results are much better than VNC –Most likely due to number of messages in an update –Differences in protocol semantics –RDP screen event rate much higher than VNC Errors are possible –Users will experience inconsistent screen states –Implementation must correct mis-predictions –Can users tolerate this?

17 User Control Errors will occur, but… Sensitivity to errors varies… –Between applications –Between users Let user decide tradeoff between aggressive use of predictions to reduce latency, and increasing display errors Can users make that decision? –What is the best way to present the control to users?

18 Why not RDP/SRD? Implementation simplicity –VNC has much less client state –VNC has fewer event types –VNC execution is straightforward If it works with VNC then it surely works with RDP

19 VNC-SRD Implementation of SRD inside VNC client –~2300 LOC (C++) added to VNC client –~850 LOC (Perl) Markov predictor implementation User-driven prediction application policy –Allows user to adjust amount of speculation Generic predictor interface –Event signature-based predictions Error correction and rollback

20

21 Prediction Application Policy System predicts event signatures –Constructs events from signatures Events directed to one of two queues –Applied Queue Applied events currently displayed to the user Held until validated by arrival of server events –Pending Queue Predictions waiting to be displayed Events applied to screen at a given rate –Rate: predicted server event rate –Rate * (desired latency reduction) = # of steps ahead

22 User Control Users/workloads have varying error tolerance –Allow direct user control VNC-SRD adds a user adjustable scale to the interface –User specifies desired network latency reduction

23 Error Handling Speculation can result in wrong display –System must determine and correct errors Correction –VNC-SRD fetches entire modified region All applied predictions are rolled back Local rollback is straightforward Validation –VNC-SRD validates prediction against next event Error on mismatch –Conservative approach Out of order predictions may be displayed correctly All errors have equal weight

24 User Study Students from Northwestern and University of Victoria –16 total users 4 network scenarios –Varied latency and drop rate 300ms, 100ms, 50ms, 10ms + 10% drop rate –3 randomly presented to each user 3 Application workloads –Word, PowerPoint, and Web Browsing –Performed in each of the network scenarios –5 minute runs Predictor state cleared for each run

25 Purpose of Study Determine whether users can tolerate screen artifacts due to speculation Determine whether users can trade off using our interface

26 Study Questionnaire Users rated the system after each scenario –Ability to find a comfortable predictor setting –Responsiveness at the most comfortable setting –Acceptability of display errors at the most comfortable setting –Scale: 1 (very bad) to 10 (very good)

27 User Study Caveats Few predictions were made –Study runs not long enough to sufficiently seed predictor –VNC has low predictability Ratings should thus be taken with a large grain of salt Future work planned to improve this

28 Able to find a comfortable setting?

29 Are Display Errors Acceptable?

30 Is the System Responsive?

31 Results We still do not know for sure whether users… –Can tolerate the screen artifacts –Can trade off between screen artifacts and latency More work needed

32 Alternative Design Approaches Speculative support on the server –Synchronized predictors Predictions used for other purposes –Cache hints –Update regions –Update ordering Other prediction algorithms/symbols –Time series predictions of event coordinates

33 Future Work Exploration of predictability in Remote Display protocols –How far ahead can predictions be made? –What prediction algorithm should be used? –To what degree can SRD decrease latency? –Comparison of more protocols Exploration of different SRD designs

34 Conclusions Remote display interactivity fundamentally constrained by network latency –Critical for WANs Speculation can improve interactivity –Can be done without server support Significant predictability exists in remote display protocol streams Speculation can be introduced into remote display clients –User-controlled tradeoff between latency and display errors –User studies currently give mixed results for tradeoff

35 Prescience Lab – Empathic Systems Project – John Lange –