Error control in video Streaming

Introduction Development of different types of n/ws such as internet, wireless and mobile networks has created new applications and communications. Multimedia and video communication Eg) youtube(a video sharing website)

Compression is an essential component of digital video. Uncompressed digital video requires - huge storage - more bandwidth

Video Communication Systems 4 End-to-End Video Transmission

The figure is about the packet based video transmission system. Packet losses may happen in the lossy channel. Compressed video is very sensitive to packet loss.

Video Conferencing over Packet- Switching Networks. Packet Loss ¡¡ Compressed video frame is packetized and transmitted. ¡¡ Packet loss can occur quite frequently due to congestion and user mobility.

To solve the problem of packet errors or packet losses the application can use Forward error correction (FEC) Automatic repeat request (ARQ)

Conventional downloading application use ARQ for robust file transfer(FTP). Two problems * receivers may need to large buffers. * users need to wait for a long time before starting playback.

Error control Techniques in video Transport level error control – Error detection and correction through FEC – Retransmission of lost packets Error resilient encoding – Adding redundancy in the bit stream to help the decoder recover from transmission error Error concealment – Recover lost/damaged regions based on image/video characteristics and human visual system properties at the decoder

Encoder-decoder-network interactive error control – Joint source/channel coding Ex: layered coding with unequal error propagation – Feedback-based adaptive encoding Ex. Reference picture selection, Selective intra update – Path diversity Different bit streams sent through separate paths

Channel coding for video multicast Providing a service such as video delivery for multiple users. Four different approaches for this.  Virtual multicast  Multicast  Broadcast  Peer to peer

Virtual multicast Group of users Server starts a separate unicast for each client. Requires more resources Lack of efficiency So max amount of client is restricted.

Multicast Clients can share the same channel. Server put the data or stream on channel once, the clients can receive it. If feedback channel is available, the users can send feedback, therefore the server can monitor the transmission quality.

Broadcast In contrast to video multicasting, video broadcasting server is responsible for large no. of users, up to millions in tv stations. So server cannot monitor the receivers quality or channel conditions

Peer to Peer Difference between general peer to peer and latest peer to peer media streaming system is that in the data sharing mode among peers, the former uses the open-after-downloading mode while the later uses the play- while-downloading.

Conventional error control techniques for multicasting. FEC ARQ Hybrid ARQ RPS Rate Distortion optimization and Collision Distortion optimization

Forward error correction. Forward Error Correction  send parity packets over a PTDD period. Periodic frame Parity Packet for periodic frame

FEC FEC is a technique used for controlling errors in data transmission over unreliable or noisy communication channels.controlling errorsdata transmission A number of FEC algorithms  Hamming code,  Reed-Solomon code  Bose-Chandhuri-Hocquenghem code.

Error detecting assume the list of numbers that you wish to transmit might be 7, 3, 8, 10, 12, and 21. 6, 7, 3, 8, 10, 12, 21, and 61. The person at the receive end would look at the first number and then ensure that the correct number of additional numbers is received and then verify that the number at the end of the sequence that is the sum of the transmitted numbers. They allow you to determine if the transmission was accurate, but they don’t allow you to correct the errors.

Error correction two terms  binary –base 2 numbers  Parity Parity is a term that is used in binary communication systems to indicate whether the numbers of 1's in a transmission is even or odd. If the number of 1's is an even number, then parity is said to be even and conversely for odd parity.

Hamming code 2^k>=n+k+1 n – no. of bits k – parity bits

Error-resilient Video Transmission 22 System Overview

Features  MBs are grouped in data units and entropy coding used  Error Control before transmission over lossy channel Forward Error Correction(FEC) Backward Error Correction (BEC) Prioritization Methods Combinations of above

 Erroneous and missing data detected and localized  Decoder gets correct data units or error indication  Error concealment applied at positions where no data received  Report loss of data units to encoder

Design Principles 25 error-resilience tools decrease compression efficiency Main goal:  Shannon’s separation principle: compression separated with transport  In low delay situations, error-free transport is impossible System Design Principles  1. Loss correction below codec layer  2. Error detection  3. Prioritization methods  4. Error recovery and concealment  5. Encoder-decoder mismatch avoidance

Error Concealment Basic Idea  Decoder should generate a representation for lost area  Match as close as possible to the lost info  Within manageable complexity Techniques  Spatial Error Concealment  Temporal Error Concealment  Hybrid Concealment  Other Techniques

Spatial Error Concealment 27 Based on assumption of continuity of natural scene content in space Use pixel values of surrounding available MBs Estimate of lost pixel: αβγ are weighing factors  Determine relative impact of vertical, Horizontal, upper, lower… Disadvantage  Blurred reconstruction

Temporal Error Concealment Rely on the continuity of a video sequence in time Use temporally neighboring areas to conceal lost regions Previous Frame Concealment (PFC)  Use previous corresponding data to copy to current frame  Only good when little motion  Widely used due to simplicity

Hybrid Concealment 29 When only apply spatial concealment  Concealed regions are significantly blurred When only use temporal error concealment  Significantly discontinuities in the concealed regions Hybrid temporal-spatial technique applied  MB mode info of reliable and concealed neighbors decide which concealment method to use

Hybrid (cont.) For intra-coded images  Only use spatial concealment For inter-coded images  Use temporal concealment when more than half of the available neighbor MBs are inter- coded  Otherwise, use spatial concealment Referred to as Adaptive temporal and spatial Error Concealment (AEC)

Selected Results 31 Performance of different error concealment strategies