1. Sub pixel motion estimation for Wyner-Ziv side information generation Subrahmanya M V (Under the guidance of Dr. Rao and Dr.Jin-soo Kim)

2. ContentsContents WZ encoder WZ decoder Motion Estimation ◦ Half-pel motion estimation ◦ Quarter-pel motion estimation Motion Compensation WZ Encoder and Decoder Implementations Results Future Tasks References

3. WZ Encoder WZ Encoder

4. Encoder is simple in complexity  H.264 Intra Key frames  Calculate WZ frame residual  Quantization  LDPCA Encoder  Low-Density Parity-Check Accumulate codes are used for encoding. [10]  Rate Control No motion estimation!

5. WZ Decoder WZ Decoder [E. Peixoto, R. L. de Queiroz, and D. Mukherjee 2008]

6. WZ Decoder(Contd) WZ Decoder(Contd) Complex Decoder  Decode H.264 Intra Key frames  Side Information Generation  Estimation of frame to be decoded  Previously decoded frames are used for estimation  Motion estimation is done here  LDPCA Decoder  Theoretically WZ codec can achieve the same efficiency as regular encoder-decoder pair

7. Motion Estimation Forward ME between key frames Backward ME between key frames Derive MVs for WZ frame using Key frame MVS [E. Peixoto, R. L. de Queiroz, and D. Mukherjee 2008]

8. Half-pel motion estimation

9. Half-pel motion estimation(Contd) 3 half-pixel positions. Interpolation is done using six tap filter [6] ◦ H33 = [F13 + -5 * F23 + 20 * F33 + 20 * F43 + -5 * F53 + F63 + 15] >> 5 ◦ G33 = [F31 + -5 * F32 + 20 * F33 + 20 * F34 + -5 * F35 + F36 + 15] >> 5 ◦ D33 = [H31 + -5 * H32 + 20 * H33 + 20 * H34 + -5 * H35 + H36 + 15] >> 5

10. Quarter-pel motion estimation

11. Quarter-pel motion estimation(Contd) 12 quarter pel positions ◦ q1 = ( F33 + G33 + 1 ) >> 1 ◦ q2 = ( G33 + F34 + 1 ) >> 1 ◦ q3 = ( F33 + H33 + 1 ) >> 1 ◦ q4 = ( H33 + G33 + 1 ) >> 1 ◦ q5 = (G33 + D33 + 1 ) >> 1 ◦ q6 = ( G33 + H34 + 1 ) >> 1 ◦ q7 = ( H33 + D33 + 1 ) >> 1 ◦ q8 = ( D33 + H34 + 1 ) >> 1 ◦ q9 = ( H33 + F43 + 1 ) >> 1 ◦ q10 = ( H33 + G43 + 1 ) >> 1 ◦ q11 = ( D33 + G43 + 1 ) >> 1 ◦ q12 = ( G43 + H34 + 1 ) >> 1

12. Motion Compensation Interpolation of pixels based on motion vector Filters and Interpolation scheme used for sub-pel MC is same as that of ME Bidirectional motion compensation

13. WZ Encoder and Decoder Implementation Integrated LDPCA encoder/decoder with JM encoder/decoder Side information using simple averaging of key frames Side information using ME  16x16, 8x8 and 4x4 block sizes  Full-pel and half-pel  Bi-directional prediction

14. ResultsResults Reconstructed Frames with simple Averaging(1 st column), Full-Pel ME(2 nd Column), Half-Pel Me(3 rd Column)

15. Results(PSNR)Results(PSNR) PSNR of the reconstructed frame with various SI frame generation schemes.

16. Future Tasks Quarter-pel motion estimation and motion compensation Increase the number of WZ frames between Key frames

17. ReferencesReferences 1. E. Peixoto, R. L. de Queiroz, and D. Mukherjee, “Mobile video communications using a Wyner-Ziv transcoder,” Proc. SPIE 6822, VCIP, 68220R Jan. 2008. 2. A. Aaron, E. Setton and B. Girod, "Towards practical Wyner-Ziv coding of video," Proceedings. 2003 International Conference on Image Processing, ICIP 2003., vol.3, pp. III-869-872 14-17 Sept. 2003 3. K. R. Rao and J. J. Hwang, Techniques and standards for image, video, and audio coding, Prentice Hall PTR, 1996. 4. Jin-Soo Kim, "Brief overview of Wyner-Ziv CODEC" (Private Communication) 5. A. Aaron, D. Varodayan, and B. Girod, “Wyner-Ziv residual coding of video,” Proc. International Picture Coding Symposium, Beijing, P. R. China, April 2006. 6. T. Wiegand and G.J Sullivan, “The H.264/AVC video coding standard”, IEEE SP Magazine, vol. 24, pp. 148-153, March 2007. 7. G. J. Sullivan, P. Topiwala, and A. Luthra, "The H.264/AVC advanced video coding standard: Overview and introduction to the fidelity range extensions", SPIE Conf. on applications of digital image processing XXVII, vol. 5558, pp. 53-74, Aug. 2004. 8. S.K. Kwon, A. Tamhankar, and K.R. Rao “Overview of H.264/MPEG-4 Part 10” J. VCIR, Vol. 17, pp. 186-216, April 2006, Special Issue on “Emerging H.264/AVC Video Coding Standard,”.

18. References(Contd)References(Contd) 9. A. Wyner and J. Ziv, "The rate-distortion function for source coding with side information at the decoder," IEEE Trans., Information Theory, vol.22, pp. 1-10, Jan. 1976. 10. D. Varodayan, A. Aaron and B. Girod, "Rate-adaptive distributed source coding using low-density parity-check codes," Conference Record of the Thirty-Ninth Asilomar Conference on Signals, Systems and Computers, 2005, pp. 1203-1207, Oct. 28 – Nov. 1, 2005. 11. Z. Li and E.J. Delp, "Wyner-Ziv video side estimator: conventional motion search methods revisited," IEEE International Conference on Image Processing, 2005. ICIP 2005, vol.1, pp. I-825- 8, 11-14 Sept. 2005. 12. L. Liu and E. J. Delp, "Wyner-Ziv video coding using LDPC codes," Proceedings of the 7th Nordic Signal Processing Symposium, 2006. NORSIG 2006. 13. D. Kubasov, K. Lajnef and C. Guillemot, "A hybrid encoder/decoder rate control for Wyner-Ziv video coding with a feedback channel," IEEE 9th Workshop on Multimedia Signal Processing, 2007. MMSP 2007., vol., pp.251-254, 1-3 Oct. 2007. 14. C. Brites and F. Pereira, "Encoder rate control for transform domain Wyner-Ziv video coding," IEEE International Conference on Image Processing, 2007. ICIP 2007., vol.2, pp.II -5-II - 8, Sept. 16 2007-Oct. 19 2007. 15. A. Roca, et al, "Rate control algorithm for pixel-domain Wyner-Ziv video coding," Proc. SPIE, vol. 6822, 68221T (2008). 16. D. Mukherjee, “Optimal parameter choice for Wyner-Ziv coding of Laplacian sources with decoder side information,” HP Labs Technical Report HPL-2007-34, 2007. 17. Z. Wang, et al, "Image quality assessment: From error visibility to structural similarity," IEEE Trans., Image Processing, vol.13, pp. 600-612, April 2004.