1. Light-Weight Stream Synchronization Framework for Multimedia Collaborative Applications Agustín J. González Department of Electronics Engineering Federico Santa María University Valaparaíso, Chile Antibes, July 2000 Hussein Abdel-Wahab Department of Computer Science Old Dominion University Virginia, U.S.A.

2. 2 Outline Introduction Synchronization Model Relaxing the synchronization condition Adaptive algorithm for intra-stream synchronization Inter-stream synchronization algorithm Results Summary

3. 3 Introduction Processing times and network delays are not deterministic. This creates the need for synchronization mechanisms that faithfully reconstruct the temporal relationship between events. Main characteristics of our solution: *It depends on one-way messages only =>No need of feedback! *These clocks might be off. *No need of globally synchronized clocks. *It supports policies to handle each stream sematics.

4. 4 Synchronization Model: Time Model Virtual Observer Networ k

5. 5 Synchronization Model: Synchronization Condition “Seen” by virtual observer “Seen” by receiver Synchronization condition: Virtual delay cte!

6. 6 Relaxing the Synchronization Condition Synchronization condition: Virtual delay There is no perfect value for cte. It must be adapted as end- to-end delay changes. Large values of cte avoid late packets, but more delay is added and more buffering is required. Tradeoff: Adjust “virtual delay” to achieve a given % of late packets

7. 7 Intra-stream Synchronization Adjust “virtual delay” to achieve a given % of late packets Estimator for % of late packets “l”: Synchronization Buffer On arriving: Compute virtual delay such that l = Desired late packet rate Algorithm: On leaving buffer: if (virtual delay shortens) Downward delay adjustment(actual delay); if (virtual delay enlargers) Upward delay adjustment(actual delay); Deliver packet for playout with actual delay;

8. 8 Intra-stream Synchronization Policies We can’t just play faster to reduce de actual delay. Think on the audio samples that will accumulate on the audio device. With video there is no problem. It might even improve quality when using inter-frame compression. Policies tell the algorithm how to change the delay taking into consideration the media stream semantics. Stream dependent policies are applied when adjusting the delay.

9. 9 Inter-stream Synchronization: Model Global synchronization model v/s Differentiated synchronization model Synchronizes streams coming from one virtual observer Synchronizes streams coming from anywhere with worst case delay

10. 10 Inter-stream Synchronization Audio Sync Data Sync Video Sync Inter-stream coordinator Virtual delay Max. virtual delay Inter-media synchronization is achieved by rendering all stream from a site with a common delay. We use a coordinator object where each media registers its delay and retrieves the current maximum.

11. 11 Intra-stream Synchronization Results: Audio Actual delay (After policy) Virtual delay

12. 12 Intra-stream Synchronization Results: Video Actual delay (After policy) = Virtual delay

13. 13 Inter-stream Synchronization Results

14. 14 Summary A framework for stream synchronization was presented. We introduce the idea of Virtual Observer to define the temporal relationship to be reconstructed at the receiver. The time model includes all delays from Virtual Observer’s perception to receiver’s. The use of policies for adjusting the delay allows the algorithms to fulfill the peculiarities of each media stream. A per-sender inter-stream synchronization approach was introduced - called differentiated synchronization model. It does not required globally synchronized clock. The effectiveness of the algorithms was tested with real data collected from the Internet.