Advance in Scalable Video Coding Proc. IEEE 2005, Invited paper Jens-Rainer Ohm, Member, IEEE

謝俊瑋 NTU CSIE, CMLab 2 Outline  Introduction  Scalability in Existing Standard  Principles of Scalable Predictive Coding  Drift control  Interframe Wavelet Coding  Conclusion

謝俊瑋 NTU CSIE, CMLab 3 Introduction  Scalable video coding is attractive due to the capability of reconstructing lower resolution or lower quality signals from partial bit streams.  A simple and flexible solution for transmission over heterogeneous network.  Allow simple adaptation for a variety of storage devices and terminals.

謝俊瑋 NTU CSIE, CMLab 4 Scalability in Existing Standard  MPEG-2  First general-purpose video compression standard which also include tools providing scalability.  Layered coding, support spatial, temporal, SNR scalability.  Number of layers is restricted to maximum of 3.

謝俊瑋 NTU CSIE, CMLab 5 Scalability in Existing Standard(2)  MPEG-4  More flexible scalability tools, including spatial and temporal scalability within a more generic framework.  SNR scalability with fine granularity and scalability at level of video objects.  AVC can in principle be run in different temporal scalability modes.

謝俊瑋 NTU CSIE, CMLab 6 Principles of Scalable Predictive Coding  Prediction should not use any decoded information from higher layers, otherwise drift effect would occur.  However the rate-distortion performance toward higher rates will be worse than single- layer coder.

謝俊瑋 NTU CSIE, CMLab 7

8 Drift control  It is possible to track the drift within the local loop of the encoder that would occur in a decoder only receiving the base-layer information.  Drift compensation  Drift clipping  Drift leaking

謝俊瑋 NTU CSIE, CMLab 9 Drift compensation  It assume that the decoder side is not aware of the drift compensation made.  Usual MC decoder loop could be used without any modifications.  The encoder have to find the balance between the penalties for the base and enhancement decoding.

謝俊瑋 NTU CSIE, CMLab 10 Drift clipping  The drift is dynamically limited if a maximum value D max is reached.  A good choice for D max is approximately by the base-layer quantizer step size.

謝俊瑋 NTU CSIE, CMLab 11 Drift leaking  The accumulation of drift is limited by multiplying D ’ = a * D (a < 1)  The best selection of drift coefficient is dependent on the operational target and sequence characteristic.

謝俊瑋 NTU CSIE, CMLab 12

謝俊瑋 NTU CSIE, CMLab 13 Interframe Wavelet Coding  To overcome the limitation which are caused by drift problem, it would be desirable to discard the temporal recursion.  Motion-Compensated Temporal Filtering

謝俊瑋 NTU CSIE, CMLab 14 Spatiotemporal wavelet decomposition

謝俊瑋 NTU CSIE, CMLab 15 Spatiotemporal wavelet decomposition (2)

謝俊瑋 NTU CSIE, CMLab 16 Lift structure

謝俊瑋 NTU CSIE, CMLab 17 Operation of Motion Compensation

謝俊瑋 NTU CSIE, CMLab 18

謝俊瑋 NTU CSIE, CMLab 19 Conclusion  The fully open loop property of MCTF provides high flexibility in bit stream scalability.  Combination of MCTF with the new drift- controlled prediction strategies is also a promising path.  Decoder could integrate additional signal synthesis element whenever the received information is incomplete.

謝俊瑋 NTU CSIE, CMLab 20 Conclusion (2)  When low delay is required, the update step must be omitted, or the number of temporal wavelet decomposition levels must be low.  Seamless transition between intraframe and interframe coding methods.