Study and Optimization of the Deblocking Filter in H Study and Optimization of the Deblocking Filter in H.265 and its Advantages over H.264 By: Valay Shah Under the guidance of: Dr. K. R. Rao

EE-5359 : Project Proposal Presentation HEVC Overview HEVC – a buzz word in compression standards Also known as H.265 50% bit rate reduction compared to H.264 (same picture quality) Price – more complex video coding algorithm with computationally more expensive tools[2] HEVC test codec – HM (HEVC test Model) 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation What’s Different? Picture partitioning A Coding Tree Block (CTB) of upto 64×64 spatial dimension in HEVC against 16×16 Macro-Block (MB) in H.264 Minimum size of Coding Unit (CU) in HEVC is 8×8 versus 4×4 in H.264 In-loop deblocking filtering Sample Adaptive Offset (SAO) filtering Challenges: HEVC takes more time (approx. 40%) and hence more power consumption [2] 2/21/2013 EE-5359 : Project Proposal Presentation

Deblocking Filter (DBF) Focusing on – Deblocking filter to reduce the processing time In-loop deblocking filtering in HEVC: Similar to H.264, operated within inter-prediction loop Simplified design in regard to its decision making, hence makes it viable for parallel processing In HEVC the processing order of the deblocking filter is [1]: Horizontal filtering for vertical edges for the entire picture first Followed by vertical filtering for horizontal edges 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Parallel Deblocking Salient Features [2]: - Larger deblocking removes data dependency between the edges in one direction (H.264 uses 4×4 deblocking filter size versus 8×8 used by HEVC) - Hence, the vertical and horizontal filtering could be parallelized fully 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation HEVC DBF Procedure[2] Advantages: Allows parallel deblocking Disadvantages: Increases the processing time since some data needs to be re-fetched 2/21/2013 EE-5359 : Project Proposal Presentation

Modified DBF Procedure[2] The principle of performing vertical filtering first followed by the horizontal filtering is kept intact The difference lies in selection of the blocks as shown in the figure on left 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Proposed Work Try to implement modified architecture in HEVC software code to get the performance enhancement Implement a low complexity offsets perceptual optimization for deblocking filtering [3] Optimize the skipping mode technique in order to decrease edge processing thereby reducing the power consumption Compare the HEVC performance with H.264 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Expected Results 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation References [1] G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649-1668, Dec. 2012. [2] M. Li et al, “De-blocking Filter Design for HEVC and H.264/AVC/AVC”, PCM 2012, LNCS 7674, pp. 273–284, 2012. [3] M. Naccari et al, “Low Complexity Deblocking Filter Perceptual Optimization For The HEVC Codec”, 18th IEEE International Conference on Image Processing, pp. 737-740, 2011. [4] A. Norkin et al, “HEVC Deblocking Filter”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1746-1754, Dec. 2012. [5] A. J. Honrubia, J. L. Martínez and P. Cuenca, “HEVC: A Review, Trends and Challenges”, Instituto de Investigación en Informática de Albacete, Spain. [6] T. Wiegand et al, “High Efficiency Video Coding (HEVC) Standarization”, IEEE Transactions on Circuits and Systems for Video Technology, Dec. 2010. [7] C. Man-Yau and S. Wan-Chi, “Computationally-Scalable Motion Estimation Algorithm for H.264/AVC Video Coding”, IEEE Transactions on Consumer Electronics, vol. 56, pp. 895-903, 2010. 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation References-contd. [8] R. Jianfeng, N. Kehtarnavaz, and M. Budagavi, “Computationally Efficient Mode Selection in H.264/AVC Video Coding, IEEE Transactions on Consumer Electronics, vol. 54, pp. 877-886, 2008. [9] Dr. K. R. Rao, “High Efficiency Video Coding”, Chapter 5 – soon to be published. [10] P. List et al, “Adaptive deblocking filter”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 614-619, 2003. [11] K. Xu and C. S. Choy, “A Five-Stage Pipeline, 204 Cycles/MB, Single-Port SRAM-Based Deblocking Filter for H.264/AVC”, IEEE Transactions on Circuits and Systems, vol. 18(3), pp. 363–374, 2008. [12] F. Tobajas et al, “An Efficient Double-Filter Hardware Architecture for H.264/AVC De-blocking Filtering”, IEEE Transactions on Consumer Electronics, Vol. 54(1), Feb. 2008. [13] Y. C. Lin et al, “A Two-Result-Per-Cycle De-Blocking Filter Architecture for QFHD H.264/AVC Decoder”, IEEE Transactions on VLSI Systems, vol. 17(6), June 2009. [14] D. Zhou et al, “A 48 Cycles/MB H.264/AVC De-blocking Filter Architecture for Ultra High Definition Applications”, IEICE Transactions Fundamentals E92-A (12), Dec. 2009. [15] JM software download for H.264/AVC: http://iphome.hhi.de/suehring/tml/ [16] HM codec download for H.265: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/ 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Implementation Steps Enable/Disable Deblocking Filter LoopFilterDisable parameter – under Deblock Filter – alone would not do the job of disabling the deblocking filter Have to use combination of following deblocking filter parameters: DeblockingFilterControlPresent LoopFilterDisable LoopFilterBetaOffset_div2 LoopFilterTcOffset_div2 2/21/2013 EE-5359 : Project Proposal Presentation

Results – Total Encoding Time # of Frames Seq. 1 Seq. 2 Seq. 3 Seq. 4 1 57.19 56.878 61.418 62.119 116.486 120.05 16.426 16.63 2 113.662 117.297 124.379 124.754 230.396 232.534 32.62 32.682 5 284.295 287.661 307.445 309.894 574.205 575.407 80.902 81.229 10 568.371 568.028 614.891 614.391 1154.293 1149.987 162.459 161.94 100 5712.818 5701.007 6044.842 6056.827  - - Total time taken in sec for encoding where columns in yellow (i.e.: 2, 4, 6 & 8) denotes results where deblocking filter was disabled whereas columns in green (i.e.: 3, 5, 7 & 9) denotes results where the deblocking filter was enabled and its parameters were optimized. 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Results – Bit-Rate # of Frames Seq. 1 Seq. 2 Seq. 3 Seq. 4 1 5816 5825.2 10667.04 10679.04 7966.08 7960.8 1932.72 1934.4 2 5801.6 5810.4 10677.12 10687.68 7948.08 7949.52 1902.24 1902.48 5 5762.24 5769.84 10653.12 10661.38 7910.112 7915.68 1836.768 1837.536 10 5714 5721.68 10612.85 10619.33 7895.712 7900.08 1818.144 1818.816 100 5910.58 5917.288 9837.552 9844.315  - -  - Bit-rate in kbps for encoding where columns in yellow (i.e.: 2, 4, 6 & 8) denotes results where deblocking filter was disabled whereas columns in green (i.e.: 3, 5, 7 & 9) denotes results where the deblocking filter was enabled and its parameters were optimized. 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Results – PSNR # of Frames Seq. 1 Seq. 2 Seq. 3 Seq. 4 1 57.19 56.878 61.418 62.119 116.486 120.05 16.426 16.63 2 113.662 117.297 124.379 124.754 230.396 232.534 32.62 32.682 5 284.295 287.661 307.445 309.894 574.205 575.407 80.902 81.229 10 568.371 568.028 614.891 614.391 1154.293 1149.987 162.459 161.94 100 5712.818 5701.007 6044.842 6056.827  - - PSNR in dB of the encoded image where columns in yellow (i.e.: 2, 4, 6 & 8) denotes results where deblocking filter was disabled whereas columns in green (i.e.: 3, 5, 7 & 9) denotes results where the deblocking filter was enabled and its parameters were optimized. 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Conclusions It is apparent from the results that the bit-rate have increased by optimizing the deblocking filter parameters. Hence, there are two benefits of applying the deblocking filter: (i) it will help remove the blocking artifacts from the reconstructed image and (ii) it will help increase the bit-rate of the signal 2/21/2013 EE-5359 : Project Proposal Presentation

EE-5359 : Project Proposal Presentation Next Steps To include the Quantization Parameter (QP) to see how does it affect the performance since it affects Beta and tC (deblocking parameters) To vary all the deblocking parameters to get the best results in terms of PSNR, total encoding time and bit-rate 2/21/2013 EE-5359 : Project Proposal Presentation