1. An Early Block Type Decision Method for Intra Prediction in H.264/AVC Jungho Do, Sangkwon Na and Chong-Min Kyung VLSI Systems Lab. Korea Advanced Institute of Science and Technology (KAIST)

2. Table of contents Introduction Intra prediction Early block type decision – Block type decision point – Rate-distortion cost Experimental results Conclusion Reference 2SiPS 2009

3. Introduction Two prediction methods in H.264/AVC – Inter prediction Using temporal correlation between the earlier frame and the current frame – Intra prediction Using spatial correlation between neighboring blocks in the current frame 9 prediction modes 3SiPS 2009

4. Introduction Why do we focus on Intra prediction (IP)? – Rate-distortion optimization (RDO) in Intra prediction also improves an overall rate-distortion performance. – More block types and prediction modes in Intra prediction than inter prediction Constraint: 560 cycles for 1920x1080@30fps (133MHz) Total: 640 cycles 4SiPS 2009

5. Introduction Why do we focus on Intra prediction (IP)? – Rate-distortion optimization (RDO) in Intra prediction also improves an overall rate-distortion performance. – More block types and prediction modes in Intra prediction than inter prediction Constraint: 560 cycles for 1920x1080@30fps (133MHz) Total: 640 cycles 5SiPS 2009 IPDCTQQ -1 IDCT Reco n. IPDCTQQ -1 IDCT Reco n. IPDCTQQ -1 IDCT Reco n. Pipeline for one prediction mode

6. Introduction Rate-distortion cost function for RDO [9] M.G. Sarwer and L.M. Po, “Bit rate estimation for cost function of 4x4 intra mode decision of h.264/avc,” ICME, 2007, pp. 1579 – 1582. 6SiPS 2009

7. Intra prediction Nine prediction modes – Associated with the direction of prediction 7SiPS 2009

8. Early block type decision Motivation – Spatial correlation between sub-blocks and macroblock (MB) The prediction mode of neighboring blocks have the similar direction. – We assume that R-D cost of neighboring blocks also have the similar cost based on the above hypothesis. R-D cost of macroblock can be substituted by R-D cost of sub-blocks for mode decision. 8SiPS 2009

9. Encoding order Conventional encoding order – 8x8 blocks are performed after all 4x4 blocks are performed. 9 4x4 C8x8 4x44x44x44x44x44x44x44x44x44x44x44x44x44x44x4 8x8 16x16 16 SiPS 2009

10. Encoding order Proposed encoding order – To use spatial correlation, the prediction of 8x8 block is performed after the predictions of the corresponding four 4x4 blocks are finished. Three possible decision points 10SiPS 2009

11. 1 st block type culling 16x16 block type – Probability of 16x16 block to be decided as the best mode when 4x4 block is selected by proposed early block type decision – 16x16 block type is bypassed when 4x4 block type is selected. 11 QP162024283236 Probability(%)00.160.280.260.40.2 11SiPS 2009

12. 2 nd block type culling Block type decision at 1/4MB point – If J RD,4x4(1/4) < J RD,8x8(1/4) Prediction of only 4x4 block type is performed evaluated after 1/4MB. – Otherwise 8x8 block type is taken. 12 16x16 block SiPS 2009 4x4 8x8

13. 2 nd block type culling Block type decision at 1/4MB point – If J RD,4x4(1/4) < J RD,8x8(1/4) Prediction of only 4x4 block type is performed evaluated after 1/4MB. – Otherwise 8x8 block type is taken. 13 16x16 block SiPS 2009 4x4 8x8

14. 2 nd block type culling Block type decision at 2/4MB and 3/4MB 14SiPS 2009 4x4 8x8 4x4 8x8

15. Block type decision point Which block type decision point is the best? – In terms of R-D performance and the number of reconstruction loops according to the positions of early block type decision method. 15 1/4MB2/4MB3/4MB R-D performance ∆ PSNR (dB)-0.039-0.016-0.009 ∆ BR (%)0.9450.5360.476 Cycle counts of reconstruction loops 4x4 block type 300360420 8x8 block type 464528592 SiPS 2009

16. Block type decision point Which block type decision point is the best? – In terms of R-D performance and the number of reconstruction loops according to the positions of early block type decision method. 16SiPS 2009 Constraint

17. Block type decision point Proposed early block type decision – 2/4 MB point is effective in terms of distortion, bit- rate and computational complexity (the number of cycles for MB process). 17 SiPS 2009 Constraint

18. R-D cost function Original cost function 18SiPS 2009

19. R-D cost function Proposed R-D cost for sub-block – Assumption: the best mode of chroma block is decided independently regardless of luma mode. 19SiPS 2009 The best mode of luma block is decided independently. The difference of R CBP between 4x4 block type and 8x8 block type is frequently very small. The value of R MB is fixed as ‘intra’ for all block types.

20. R CBP Coded_block_pattern (CBP) variable 20 Coded_block_pattern indicates which 8x8 blocks(6) in the MB contain nonzero coefficients 8 LumaChroma 000000 SiPS 2009

21. R CBP Comparison of R CBP 21SiPS 2009

22. R-D cost function Simplified R-D cost function – Proposed R-D cost function enables evaluation of R header to be performed at the early decision point (2/4MB) because R Header can be derived from R L mode. 22SiPS 2009

23. Experimental results R-D performance comparison 23SiPS 2009 Full search Proposed

24. Conclusion Computational complexity problem in RDO- enabled Intra prediction An early decision at 2/4MB point for RDO- enabled Intra prediction – Block type decision based on spatial correlation – R-D cost computation is reduced by 90.1% with 0.93% bit-rate increase and 0.039 dB PSNR decrease compared to full search. 24SiPS 2009

25. Thank you SiPS 200925

26. Reference [1] F. Pan, X. Lin, S. Rahardja, K.P. Lim, and Z.G. Li, “A directional field based fast intra mode decision algorithm for h.264 video coding,” ICME, 2004, vol. II, pp. 1147–1150. [2] A.C. Tsai, J.F. Yang, andW.G. Lin, “Effective subblockbased and pixel-based fast direction detections for h.264 intra prediction,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 18, pp. 975–982, July 2008. [3] Y.-K. Lin and T.-S. Chang, “Fast block type decision algorithm for intra prediction in h.264 frext,” ICIP, 2005, vol. I, pp. 585–588. [4] T. Zhang, G. Tian, and S. Goto, “A frequency-based fast block type decision algorithm for intra prediction in h.264/avc high profile,” IAPCCAS, 2008, vol. II, pp. 1292–1295. [5] W. Lee, Y. Jung, S. Lee, and J. Kim, “High speed intra prediction scheme for h.264/avc,” Consumer Electronics, IEEE Transactions on, vol. 53, pp. 1577–1582, Nov 2007. [6] G. Jin, J.-S. Jung, and H.-J. Lee, “An efficient pipelined architecture for h.264/avc intra frame processing,” ISCAS, 2007, vol. II, pp. 1605–1608. [7] Y.-K. Lin, C.-W. Ku, D.-W. Li, and T.-S. Chang, “A 140-mhz 94k gates hd1080p 30-frames/s intra- only profile h.264 encoder,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 19, pp. 432–436, Mar 2009. [8] C.K. Huang and L.L. Youn, “An h.264/avc full-mode intra-frame encoder for 1080hd video,” ICME, 2008, pp. 1037–1040. [9] M.G. Sarwer and L.M. Po, “Bit rate estimation for cost function of 4y4 intra mode decision of h.264/avc,” ICME, 2007, pp. 1579 – 1582. [10] I.E. Richardson, “Draft itu-t recommendation and final draft international standard of joint video specification(itu-t rec.h.264 -iso/iec14496-10 avc),” Joint Video Team, 2003. [11] Z. Kun, Y. Chun, L. Qiang, and Z. Yuzhou, “A fast block type decision method for h.264/avc intra prediction,” ICACT, 2007, vol. I, pp. 673–676. [12] “Joint video team (jvt) reference software version 14.0,” http://iphome.hhi.de/suehring/tml/download/. http://iphome.hhi.de/suehring/tml/download/ [13] I.E. Richardson, “H.264 and mpeg-4 video compression: video coding for next generation multimedia,” Chichester, U.K.: Wiley, 2003. 26SiPS 2009

27. Appendix SiPS 200927

28. Introduction H.264/AVC coding flow 28 Entropy Coding Scaling & Inv. Transform Motion- Compensation Control Data Quant. Transf. coeffs Motion Data Intra/Inter Coder Control Decoder Motion Estimation Transform/ Scal./Quant. - Input Video Signal Split into Macroblocks 16x16 pixels Intra-frame Prediction De-blocking Filter Output Video Signal SiPS 2009

29. Introduction Why we focus on Intra prediction – Computational bottleneck in rate-distortion (R-D) optimized encoder due to many block types and prediction modes Constraint: 560 cycles for 1920x1080@30fps (133MHz) Total: 640 cycles (= 15*16 + 60*4 + 160) – One 4x4 block: 15 cycles – One 8x8 block: 60 cycles – One 16x16 block: 160 cycles 29SiPS 2009

30. Early block type decision Flow 30 SiPS 2009