1. 4/24/2002SCL UCSB1 Optimal End-to-end Distortion Estimation for Drift Management in Scalable Video Coding H. Yang, R. Zhang and K. Rose Signal Compression Lab ECE Department University of California, Santa Barbara

2. 4/24/2002 SCL UCSB2 Outline n Introduction n ROPE for scalable coding n R-D optimized mode selection n Simulation results n Conclusions

3. 4/24/2002 SCL UCSB3 Introduction n Scalable video coding n Drift problem n Multicast scenario & existent framework n Point-to-point scenario & proposed framework n Proposed coding approach

4. 4/24/2002 SCL UCSB4 Scalable video coding and drift problem n Scalable video coding –Error resilience –Multiple QoS n Drift problem –Whether to use enhancement layer information for prediction If used better prediction improve coding gain If lost mismatch / error error propagation

5. 4/24/2002 SCL UCSB5 Scalable video coding and drift problem n Drift management –Goal: Achieve a good trade-off. –Key : Accurately measure and thus effectively control the amount of incurred error propagation. H.263 and MPEG4 favor no-drift system. EI I EP P P

6. 4/24/2002 SCL UCSB6 Multicast Scenario & existent framework n Independent channels with different capacities –Some receivers have only access to the base layer, while others have access to both. –A coarse but acceptable base layer video quality is necessary. n Existent coding framework: EI I EP P P

7. 4/24/2002 SCL UCSB7 Point-to-point Scenario & proposed framework n Only one channel is considered. –Scalable coding only provides error resilience. –An acceptable base layer video quality is NOT necessary. n Proposed coding framework: n Research Purposes: –How much we can gain by using the proposed framework. –Investigate the importance of accurate end-to-end distortion estimation in effective management of drift. EI I EP P P

8. 4/24/2002 SCL UCSB8 Proposed coding approach n Macroblock(MB) based SNR scalable video coding n Objective: To minimize the expected end-to-end distortion given the packet loss rate and the total bit rate. n Drift management is fulfilled via R-D optimized coding mode selection for each MB. n To accurately estimate the end-to-end distortion, ROPE is adopted.

9. 4/24/2002 SCL UCSB9 Proposed coding approach n Coding mode selection is an efficient means to optimize the tradeoff between coding efficiency and error resilience. EI I EP P P Intra: Stop error propagation & most bits. Inter B B: No new error & less bits. Inter E B: New error & least bits.

10. 4/24/2002 SCL UCSB10 ROPE for Scalable Coding n Recursive Optimal per-Pixel Estimate (ROPE): –Take account of all the relevant factors as quatization, packet loss and error concealment. –Accurate & low complexity. n Adapt ROPE to scalable coding: –All the data of one frame is transmitted in one packet. –Channel is modeled as a Bernoulli process with packet loss only in the enhancement layer.

11. 4/24/2002 SCL UCSB11 Overall expected decoder distortion of pixel i in frame n: Original value, Encoder reconstruction values, Decoder reconstruction values Packet loss rate of the enhancement layer, Quantized prediction residues

12. 4/24/2002 SCL UCSB12 n Intra: Calculation of and n Inter B B: n Inter E B:

13. 4/24/2002 SCL UCSB13 n Intra: Calculation of and assuming upward error concealment n Upward: n Inter E E:

14. 4/24/2002 SCL UCSB14 RD Optimized Mode Selection n Unconstrained minimization: –J can be independently minimized for each MB. –The coding mode and quantization step size of each MB are jointly selected. n Sequential optimization: 1. 2.,.

15. 4/24/2002 SCL UCSB15 Simulation Results n UBC H.263+ codec with two-layer scalability Mean luminance PSNR: average first over the frames and then over the packet loss patterns n Base layer bit rate: 75 kb/s, enhancement layer bit rate: 225kb/s, frame rate: 30 f/s, 150 frames, 50 packet loss patterns.

16. 4/24/2002 SCL UCSB16 (a) QCIF “Carphone”(b) QCIF “Miss_am” Fig.1 PSNR Performance of different coding frameworks

17. 4/24/2002 SCL UCSB17 (c) QCIF “Foreman”(d) QCIF “Salesman” Fig.1 (continued) n Gain of “B&E drift” over “E drift” : 0.92 dB ~ 2.83 dB Gain of “E drift” over “no drift”: 1.25 dB ~ 2.20 dB n With p increased, gain of “B&E drift” over “E drift” is unchanged, while gain of “E drift” over “no drift” is diminished quickly.

18. 4/24/2002 SCL UCSB18 (a) QCIF “Carphone”(b) QCIF “Miss_am” Fig.2 Performance of different distortion estimation methods

19. 4/24/2002 SCL UCSB19 (c) QCIF “Foreman” (d) QCIF “Salesman” Fig.2 (continued) n “ROPE-RD” always outperforms “QDE-RD”. n In most cases, “QDE-RD” performs even worse than “no drift”.

20. 4/24/2002 SCL UCSB20 Conclusions In the context of point-to-point video transmission over lossy networks: n Decoder drift due to prediction and packet loss should be controlled but not altogether disallowed. n Reaping the full benefits of drift management requires accurate estimation of end-to-end distortion.