MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin

2 Reference S. Chattopadhyay, S. M. Bhandarkar, K. Li, “FGS-MR: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding,” ACM NOSSDAV W. Li, “Overview of Fine Granularity Scalability in MPEG-4 Video Standard,” IEEE Trans. on Circuits and Systems for Video Technology, Vol. 11, No. 3, pp , Mar H. Radha, M. van der Schaar, and Y. Chen, “The MPEG-4 fine- grained scalable video coding method for multimedia streaming over IP,” IEEE Trans. on Multimedia, vol.3, pp. 53–68, Mar

3 Outline Introduction MPEG-4 Fine Grained Scalability Motivation FGS-AQ vs. FGS-MR Experimental Results Conclusion

4 Introduction MPEG4 Fine Grained Scalability (FGS) profile for streaming video Base Layer Bit Stream must exist at the decoder has coarsely quantized DCT coefficients provides the minimum video quality Enhancement Layer Bit Stream can be absent at the decoder contains encoded DCT coefficient differences provides higher quality can be truncated to fit the target bit rate

5 FGS Encoding Block Diagram

6 Motivation Base Layer video quality is usually not satisfactory in order to provide a wide range of bit rate adaptation MPEG4 FGS Adaptive Quantization (FGS-AQ) for Base Layer video does not provide good rate- distortion (R-D) performance parameter overhead at the decoder Proposed FGS-MR no parameter overhead to transmit transparent the codec better rate-distortion performance

7 Outline Introduction MPEG-4 Fine Grained Scalability Motivation FGS-AQ vs. FGS-MR FGS-AQ FGS-MR MR-Mask Creation MR-Frame Experimental Results Conclusion

8 FGS Adaptive Quantization (AQ) Goals To improve visual quality To better utilize the available bandwidth Method Define different quantization step sizes for different transform coefficients within a macro-block (low freq. DCT coeff. => small step size) for different macro-blocks (different quantization factors) Disadvantages R-D performance degrades due to FGS-AQ parameter overhead

9 Proposed Multi-Resolution FGS (FGS-MR) Goal To improve the visual quality To better utilize the available bandwidth No transmission overhead and hence maintaining the R-D performance Method Apply a low-pass filter on “visually unimportant” portion of the original video frame before encoding.

10 Two Equivalent Operations Apply a low-pass filter on the spatial domain of an image Truncate DCT coefficients in the corresponding transform domain of an image

11 FGS-MR Process (Step 1) MR-Mask creation Use Canny edge detector to detect edges Weight Mask an weight parameter w i, j for each pixel p(i, j) of an image, 0 ≦ w i, j ≦ 1 w i, j = 1, if p(i, j) is on the edge 0 < w i, j ≦ 1, if p(i, j) is near edge w i, j = 0,if p(i, j) is in non-edge region

12 Original (5.12Mbps)

13 MR-Mask

14 FGS-MR Process (Step 2) MR-Frame Creation V I = (I-W) V L +W V H V F = Iteration( V I, G(σ I )) Note V I contains abrupt changes in resolution V F is a smooth version of V I Parameters V o : original video frame V L : low resolution frame from the convolution of V o and G(σ L ) V H : high resolution frame from the convolution of V o and G(σ H ) V I : intermediate video frame V F : final multi-resolution frame I: matrix with all entries as 1 W: MR-mask weight matrix G(σ): Gaussian filter with standard deviation of σas LPF σ L >σ H

15 Original (5.12Mbps)

16 FGS-AQ (0.17Mbps, PSNR = 22.77dB)

17 FGS-MR (0.17Mbps, PSNR = 26.5dB)

18 Determine Parameters σ L, σ H, and σ I to control the bit rate W (weight matrix) to control the quality of the encoded video frame Figure of merit function: δ=Q/C Q = 2^( PSNR(σ L, σ H, σ I )/10 ) or PSNR = 10log(Q) C: compression ratio The authors empirically determine the parameters σ L = 15, σ L = 3, and varying σ I

19 Outline Introduction MPEG-4 Fine Grained Scalability Motivation FGS-AQ vs. FGS-MR FGS-AQ FGS-MR Experimental Results Rate Distortion Resource Consumption Conclusion

20 Experiments Video 1 320x240, fps = 30 A single person walking in a well lighted room Video 2 176x144, fps = 30 A panning view across a poorly lighted room. No moving object

21 Rate Distortion Performance Vary σ I from 3 to 25 to adjust the target bit rate

22 Power Consumption Energy used and hence power consumed by wireless network interface card (WNIC): T: time duration S: data size b: the bit rate of streaming video B: available BW E R : energy used by WNIC during data reception E s : energy used by WNIC when sleeping

23 Power Consumption Comparison

24 Conclusion The rate distortion performance of FGS-MR is better than FGS-AQ. FGS-MR can be seamlessly integrated into existing MPEG4 codec. My comment Processing time issue of FGS-MR Empirical determined filter parameters