1. Error Control in Multimedia Streaming

2. What’s the difference Traditional data: Text  Packet loss is intolerable while delay is acceptable Multimedia data: Real-time video  Some visual quality degradation is often acceptable while delay must be bounded  This feature introduces many new error control mechanisms applicable to video application, but not applicable to traditional data.

3. Categories FEC Retransmission Error-resilient encoding Error concealment

4. FEC Forward Error Correction Add redundant information so that original information can be reconstructed in the presence of packet loss. More effective when the average number of consecutively lost packets is small.

5. Retransmission May miss the play-out time, if the retransmission needs too much time which exceeds the allowable delay So we need to know if the retransmission will be available before the deadline, then decide retransmit or not.

6. Error-resilient Objective Enhance robustness of compressed video to packet loss Executed by the source to enhance robustness Difference with FEC prevent error propagation or limit the scope of the damage(caused by packet losses)

7. Error-resilient: example MDC—multiple description coding Compressed into multiple streams (descriptions) Each description provides acceptable visual quality More combined descriptions provide a better visual quality Advantage Robustness to loss, enhanced quality Disadvantage Trade off between compression efficiency and the reconstruction quality from one description

8. Error concealment Performed by the receiver when packet loss has already occurred The receiver conceal the lost data and make the presentation less displeasing to human eyes Example Error resilient and conceal are more about compression and the nature of the original data.

9. An architecture for error control mechanisms

10. FEC--categories Channel coding Source coding Joint source/channel coding

11. FEC—Channel coding The channel encoder places the k packets into a group and then create additional packets from them so that the total number of packets in the group becomes n. A user must receive K (K>=k) packets. These K packets will be enough for user to reconstruct all the original k packets.

12. FEC--Channel coding(contd.)

13. FEC--Channel Coding(contd.) Pro Can scale to arbitrary number of receivers in a large multicast group. Allows the network and receivers to discard some of the packets which cannot be handled due to limited source. Con Increase the transmission rate Increase delay Not adaptive to varying loss characteristics (can use feedback to improve it)

14. FEC—channel coding(contd.) EEP(equal error protection) All the bits of the compressed video stream are treated equally. Not good for compressed video stream--MPEG UEP(unequal error protection) Example: Priority Encoding Transmission Allow user to set different levels of error protection for different segments of the video stream.  Transport MPEG video which has an inherent priority hierarchy (I-,P-,and B-frames), give more redundancy to I-frame, but less to B-frames.

15. Source coding Source coding-based FEC(SFEC) The nth packet contains both nth GOB (group of blocks) and redundant information about the (n-1)th GOB ( a much more compressed version) The we can reconstruct (n-1)th GOB if it is lost and we receive nth GOB. (will has coarser quality) Channel coding can perfectly recover, while SFEC can only recover an acceptable one. Lower delay (needn’t to accumulate k blocks)

16. FEC—joint source/channel coding Shannon’s Separation Principle Both source coding and channel coding can be done independently. But it is only valid upon the assumption of allowing infinitely long codewords (infinitely long delay)

17. FEC—JSCC(contd.) channel capacity-limited video commu. system: spending more bits on the source means not enough channel protection, which leads to channel errors, received video quality is bad; spending more bits on the channel means enough protection and no transmission errors, but then you have overcompressed the source material and received video quality is again bad. There is a trade-off and balance point where the channel capacity is optimally allocated between source and channel to achieve the best received video quality.

18. FEC—JSCC(contd.) Source codingJoint source/channel coding

19. FEC—JSCC(contd.) Task 1: finding an optimal rate allocation between source coding and channel coding for a given channel loss characteristic Task 2: designing a source coding scheme to achieve its target rate Task 3: designing channel codes to match the channel loss characteristic and achieve the required robustness

20. Delay-constrained retransmission An approach in Transport layer The conventional retransmission scheme When packets are lost, the receiver sends feedback to notify the source, the the source retransmits the lost packets So the retransmitted packet arrives at least 3 one- way trip time after the transmission of original packet. It only works if the one-way trip time is short with respect to the maximum allowable delay.

21. Categories Unicast Receiver-based Sender-based Hybrid control Multicast

22. Unicast—receiver-based Receiver-based control Objective: minimize the requests of retransmission that will not arrive timely for display Algorithm When receiver detects the loss of packet N: Ask sender to retransmit packet N Current time Slack term, could include tolerance of error in estimating RTT. Scheduled display time for packet N

23. Receiver-based(contd.)

24. Unicast—sender-based Sender-based control Objective: suppress retransmission of packets that will miss their display time at the receiver. Algorithm: When the sender receives request of retrans. for packet N: retransmit packet N to receiver Estimated one-way trip time Estimate of Td(N)

25. Sender-based(contd.)

26. Unicast—hybrid control Receiver-based control Objective: both of the objectives of receiver-based and sender-based Receiver makes decisions on whether to send retransmission requests while the sender makes decisions on whether to discard retransmission requests. Higher performance at the cost of higher complexity.

27. Multicast Retransmission has to be restricted within closely located multicast members. Because one-way trip times between these members tend to be small, and the neighbor members have the packet which is lost. Basic idea: this closely located multicast member will be the designated receiver (DR) in a tree-based hierarchical framework.

28. Logical tree with DR

29. Local recovery for multicast

30. LVMR Layered Video Multicast with Retransmission Use a statically configured logical tree Improve efficiency by not asking for retransmissions of those packets which cannot be recovered in time. Improve response time by sending immediate NAKs and multicasting retransmissions. Use buffers to not only absorb the jitter but also to increase the likelihood of getting retransmitted packets before deadline.

31. Adaptive Playback Points with Extended Control time playback point of frame i is In LVMR, playback point p0 is extended by an additional amount which is referred to as control time. We have, is the maximum jitter in the network. So there is at least a budget of time units after the arrival of a frame to recover any missing fragments. Trade-off between latency and reliability. LVMR—contd. P0:the playback point of frame 0 R: frame rate (frame/second)

32. STORM STructure-Oriented Resilient Multicast Use a dynamically changing logical tree When a receiver first joins a multicast group, it uses an expanding ring search to look for parent nodes(DR). Selects its parent based on the packet loss rate of the candidate parent nodes Receiver periodically computes the ratio of the number of successful repairs from a parent to the number of NACKs sent to the parent. If this ratio falls below a threshold, start to find a new parent. Let the receivers decide on their own the trade-off between latency and reliability.

33. Conclusion FEC,retransmission, error-resilience are performed at both the source and the receiver side, while error concealment is carried out only at the receiver side. So it is used at the last stage. They are combined to get a better performance. Transport perspective: Retransmission, FEC Compression perspective: Error-resilience, error concealment, FEC Use feedback to improve the performance

34. Main Reference A. Albanese, J.Blomer, J.Edmonds, M. Luby, and M. Sudna, “Priority encoding transmission,” IEEE trans. On information Theory, vol.42,no.6,pp.1737-1744, Nov.1996 J-C. Bolot and T.Turletti, “Adaptive error control for packet video in the Internet,” in Proc. IEEE Int. Conf. On Image Processing, pp.25-28, Lausane, Switzerland, Sept. 1996 Dapeng Wu,Yiwei Thomas Hou,etc, Streaming Video over the Internet: Approaches and Directions IEEE Transaction on Circuits and System for Video Technology, Vol11,No1,February 2001 Dapeng Wu, Yiwei Thomas Hou,etc, Transporting real-time video over the Internet:challenges and approaches, Proceedings of the IEEE, vl.88,no. 12, Dec.2000 X. Li, S. Paul, P.Pancha, and M.H. Ammar, “Layered video multicast with retransmissions (LVMR): evaluation of error recovery schemes, “ in Proc. IEEE Int. Workshop on Network and Operating System Support for Digital Audio and Video, pp.161-172, May 1997.