09/24/02ICIP20021 Drift Management and Adaptive Bit Rate Allocation in Scalable Video Coding H. Yang, R. Zhang and K. Rose Signal Compression Lab ECE Department University of California, Santa Barbara

09/24/02 ICIP20022 Outline n Introduction n ROPE for scalable coding n R-D optimized mode selection n Simulation results n Conclusions

09/24/02 ICIP20023 Introduction n Scalable video coding n Drift problem n Multicast scenario & existent framework n Point-to-point scenario & proposed framework n Proposed coding approach

09/24/02 ICIP20024 Scalable video coding and drift problem n Scalable video coding –Error resilience –Multiple QoS n Drift problem –Whether to use enhancement layer information for prediction If used better prediction improve coding gain If lost mismatch / error error propagation

09/24/02 ICIP20025 Scalable video coding and drift problem H.263 and MPEG4 favor no-drift system. n Drift management –Goal: Achieve a good trade-off. –Key : Accurately measure and thus effectively control the amount of incurred error propagation. EI I EP P P

09/24/02 ICIP20026 Multicast scenario & existent framework n Independent channels with different capacities –Some receivers have only access to the base layer, while others have access to both. –A coarse but acceptable base layer video quality is necessary. –Bit rates of different layers are determined by channel capacities. n Existent coding framework: EI I EP P P

09/24/02 ICIP20027 Point-to-point scenario & proposed framework n Only one channel is considered. –Scalable coding only provides error resilience. –An acceptable base layer video quality is not necessary. –Bit rates of the layers don’t need to be specified before encoding. n Proposed coding framework: n Research Purposes: –How much we can gain by using the proposed framework. –Investigate the importance of accurate end-to-end distortion estimation in effective management of drift. EI I EP P P

09/24/02 ICIP20028 Proposed coding approach n Macroblock(MB) based SNR scalable video coding n Objective: To minimize the expected end-to-end distortion given the packet loss rate and the total bit rate. n Drift management and adaptive bit rate allocation are fulfilled via R-D optimized coding mode selection for each MB. n To accurately estimate the end-to-end distortion, ROPE is adopted.

09/24/02 ICIP20029 Proposed coding approach n Coding mode selection is an efficient means to optimize the tradeoff between coding efficiency and error resilience. EI I EP P P Intra: Stop error propagation & most bits. Inter B B: No new error & less bits. Inter E B: New error & least bits.

09/24/02 ICIP ROPE for Scalable Coding n Recursive Optimal per-Pixel Estimate (ROPE): –Take account of all the relevant factors as quatization, packet loss and error concealment. –Accurate & low complexity. n Adapt ROPE to scalable coding: –All the data of one frame is transmitted in one packet. –The channel is modeled as a Bernoulli process with packet loss only in the enhancement layer.

09/24/02 ICIP Overall expected decoder distortion of pixel i in frame n: Original value, Encoder reconstruction values, Decoder reconstruction values Packet loss rate of the enhancement layer, Quantized prediction residues

09/24/02 ICIP n Intra: Calculation of and n Inter B B: n Inter E B:

09/24/02 ICIP n Intra: Calculation of and assuming upward error concealment n Upward: n Inter E E:

09/24/02 ICIP RD Optimized Coding Mode Selection n Unconstrained minimization: –J can be independently minimized for each MB. –The coding mode and quantization step size of each MB are jointly selected. n Joint optimization: –Global optimal but with non-trivial complexity. For simple retransmission,..

09/24/02 ICIP RD Optimized Coding Mode Selection n Sequential optimization: –Sub-optimal but with low complexity –For the base layer: –For the enhancement layer:

09/24/02 ICIP Simulation Results n UBC H.263+ codec with two-layer scalability. Mean luminance PSNR: average first over the frames and then over the packet loss patterns. n QCIF sequences: Carphone and Salesman, first 150 frames, frame rate: 30 f/s, total bit rate: 300 kb/s. n 50 packet loss patterns. n Assuming simple retransmission:

09/24/02 ICIP (a) QCIF “Carphone”(b) QCIF “Salesman” Fig.1 PSNR Performance of different coding frameworks n Gain of “B&E drift” over “E drift” : 0.78 dB ~ 2.80 dB Gain of “E drift” over “no drift”: 0.65 dB ~ 2.59 dB n Sequential opt. captures much of the gain of the joint opt., while their complexity ratio is approximately 1:13.

09/24/02 ICIP (a) QCIF “Carphone”(b) QCIF “Salesman” Fig.2 Performance of different distortion estimation methods n “ROPE-RD” always largely outperforms “QDE-RD”. n At high packet loss rates, “QDE-RD” performs even worse than “no drift”.

09/24/02 ICIP Conclusions In the context of point-to-point video transmission over lossy networks: n Decoder drift due to prediction and packet loss should be controlled but not altogether disallowed. n Bit rates of different layers should be adaptively allocated for each frame. n Reaping the full benefits of drift management and adaptive bit rate allocation requires accurate estimation of end-to- end distortion.

09/24/02 ICIP Thank you!