1. Analysis of Motion Estimation Algorithm (HEVC), using Multi-core processing Shiba Kuanar Shiba.kuanar@mavs.uta.edu 1000449352

2. OVERVIEW OF HEVC ALGORITHEM SEARCH RESULTS REFERENCES CONTENTS:

3. High Efficiency Video Coding (HEVC) is the newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards—in the range of 50% bit-rate reduction for equal perceptual video quality. Video Coding Standards: Always have a trade between 1) Picture Quality 2) Compression Ratio 3) Computation complexity HEVC

4. Fig 1:HEVC Encoder[2]

5. Partitioning – into non overlapping macro blocks Prediction - Forward/Backward based on current, past and future frame Error signal - transformed and quantized Entropy coded HEVC

6. Figure 2: HEVC decoder block diagram [2]

7. Fig 3: Partitioning of a Macro block CTB->CBs andTBs Solid lines indicate CB boundaries and dotted lines indicate TB boundaries Corresponding QUAD TREE

8. Motion estimation consumes more than 84% coding complexity and time to encode Block Based ME algorithms are proposed to reduce the computation time. Algorithms will be discussed the current proposal “Based on search position number reduction” “Fast Full algorithm” Proposed work is to analyze the existing algorithm and try to reduce motion estimation computation time using multicore programming. HM13.0 – latest HEVC software. Block Matching Criterions used in BBME - Mean of Square Error - Mean of absolute difference (No multiplication) - Matching Pixel Count. Motion Estimation(ME)

9. Fig 4: BBME Algorithm for a) 2DLOG, b) TSS and c) NTSS search

10. Figure 5: Various Search Patterns: a) Diamond search, b) Square Search, c) Horizontal Hexagonal search, d) Vertical Hexagonal Search, e)Rotating Hexagon Type - 1 and f) Rotating Hexagon Type - 2 [22] a) Diamond search b) Square Search c) Horizontal Hexagonal d) Vertical Hexagonal e) Rotating Hexagon Type – 1 f) Rotating Hexagon Type – 2

11. In HEVC video coding, motion estimation (ME) plays a vital role in temporary redundancy reduction between the frames. In BBME, the current frame is divided into N ×N pixel size macro blocks (MBs) and for each MB a certain area of the reference frame is searched, to minimize a block difference measure (BDM). The block difference measure is usually a sum of absolute differences (SAD) between the current and the reference frame MB. The displacement within the search area which gives the minimum BDM value is called a motion vector (MV). MVs together with transformed and quantized block differences (residua) are entropy coded into the video bit stream. If maximum search range (SR) within the reference frame is set to d, then (2d+ 1)² [fig 6] search points (SPs) have to be evaluated, when the full search (FS) strategy is employed. To decrease such a huge computational burden, many fast BBME algorithms have been proposed. On the current proposal only two algorithms are going to be analyzed 1) Algorithm based on the search position number and 2) algorithm based on reduction of fast full search. Block based Motion Estimation

12. Figure 6: d - search range, with a reference frame. (2d+ 1)² search points (SPs)

13. Full Search Algorithm Process (4 stages) 1) Prediction stage – Algorithm uses motion vector of previously coded neighboring blocks to predict initial block. 2) Search patterns are employed to find the “Global minimum point” 3) Early termination where the ME algorithm uses a “threshold” to terminate the search process (Basically Iterative process) 4) ME algorithm refines the motion vector if it does not satisfy the early termination criteria

14. Results and Plots (Measures) The simulation will be conducted using HM Software 13.0 with difference video sequences [28], search range, block sizes and number of frames using GPU multicore computing. PSNR (dB), bitrate (kbps) plots and BD (Bjontegaard Delta) [17] results will calculated for different algorithms using various search patterns

15. AVC: Advanced Video Coding CABAC: Context Adaptive Binary Arithmetic Coding CB: Coding Block CSVT: Circuits and Systems for Video Technology CTB: Coding Tree Block CTU: Coding Tree Unit CPU: Central Processing Unit CU: Coding Unit CUDA: Compute Unified Device Architecture DCT: Discrete Cosine Transform GPU: Graphic Processing Unit HEVC: High Efficiency Video Coding ISO: International Organization for Standardization ITU-T: International Telecommunication Union – Telecommunication Standardization Sector Acronyms

16. MPEG: Moving Picture Experts Group OPENMP: Open Multiprocessing PB: Prediction Block PCM: Pulse Code Modulation PU: Prediction Unit SIMD: Single Instruction Multiple Data TB: Transform Block TU: Transform Unit VCEG: Video Coding Experts Group VBSME: Variable Block Size Motion Estimation Acronyms

17. Reference thesis 1 Thesis by S.Gangavati on “Complexity reduction of H.264 using parallel programming” which describes significant speed-up in encoding time on GPU using CUDA and CPU combined than on CPU by data and task parallelization, 2012. http://www-ee.uta.edu/Dip/Courses/EE5359/Sudeep_Thesis_Draft_2.pdf 2 Thesis proposed by Pratik Meheta on “Complexity reduction for intra mode selection in HEVC using OpenMP” http://www- ee.uta.edu/Dip/Courses/EE5359/Pratik_Mehta_ThesisProposal.pdfhttp://www- ee.uta.edu/Dip/Courses/EE5359/Pratik_Mehta_ThesisProposal.pdf References [1] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. CSVT, vol. 22, pp.1649-1668, Dec.2012. References

18. [2] G. J. Sullivan et al “Standardized Extensions of High Efficiency Video Coding (HEVC). J. Sel. IEEE Journal of topics in Signal Processing” vol. 7, pp.1001-1016, Dec. 2013 [3] F. Bossen et al, “HEVC complexity and implementation analysis”, IEEE Trans. CSVT, vol. 22, pp.1685-1696, Dec.2012. [4] T.L Silva et al,”HEVC intra coding acceleration based on tree inter-level mode correlation”, SPA 2013, Poznan, Poland Sep.2013. [5] A. Saxena and F. Fernanades, “Mode dependent DCT/DST for intra prediction in block based image/video coding”, IEEE ICIP, pp. 1685-1688, Sept. 2011. [6] H. Zhang and Z. Ma, ”Fast intra prediction for high efficiency video coding ”, Pacific Rim Conf. on Multimedia, PCM2012, Singapore, Dec.2012. [7] C.C. Chi et al, “Parallel scalability and efficiency of HEVC parallelization approaches”, IEEE Trans. CSVT, vol. 22, pp.1827-1838, Dec.2012. References

19. [8] Introduction to parallel computing https://computing.llnl.gov/tutorials/parallel_comp/#Whatishttps://computing.llnl.gov/tutorials/parallel_comp/#Whatis [9] T. Wiegand, et al, Working Draft 3 of High Efficiency Video Coding, JCTVC-E603, JCTVC of ISO/IEC and ITU-T. Geneva, Swiss, Mar. 2011. [10] T. Wiegand and G.J. Sullivan, "Overview of the H.264/AVC video coding standard," IEEE Trans. on Circ. Sys. for Video Tech., vol. 13, no. 7, July 2003. [11] Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, "Comments on Motion Estimation Algorithms in Current JM Software (JVT-Q089)", Joint Video Team Document, 17th Meeting: Nice, FR, 14-21 Oct., 2005. [12] N. Purnachand, L.N. Alves and A.Navarro, "Improvements to TZ search motion estimation algorithm for multiview video coding", IEEE IWSSIP 2012, Vienna, Apr. 2012. [13] HM Reference Software 13.0 https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-13.0rc1/ [14] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.itsudparis.eu/jct/doc_end_user/current_document.php?id=7243 ​ http://phenix.itsudparis.eu/jct/doc_end_user/current_document.php?id=7243 [15] J. Ascenso, C. Brites and F. Pereira, "Improving Frame Interpolation with Spatial Motion Smoothing for Pixel Domain Distributed Video Coding", in Proc. EURASIP Conference on Speech and Image Processing, Multimedia Communication and Services, Slovak Republic, June-July 2005. [16] W. Hong, “Coherent Block-Based Motion Estimation for Motion-Compensated Frame Rate Up-Conversion", IEEE International Conference on Consumer Electronics, pp. 165-166, Jan.2010. References

20. [17] G. Bjontegaard, "Calculation of average PSNR difference between RD curves", VCEG-M33, 2001. [18] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012. [19] P. Hanhart et al, “ Subjective quality evaluation of the upcoming HEVC video compression standard”, SPIE Applications of digital image processing XXXV, vol. 8499, paper 8499 -30, Aug. 2012. [20] M. Horowitz et al, “Informal subjective quality comparison of video compression performance of the HEVC and H.264/MPEG - 4 AVC standards for low delay applications”, SPIE Applications of digital image processing XXXV, vol. 8499, paper 8499 - 31, Aug. 2012. [21] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012. [22] L.N.A. Alves, A. Navarro, " Fast Motion Estimation Algorithm for HEVC ", Proc IEEE International Conf. on Consumer Electronics -ICCE Berlin, Germany, vol.11, pp. 11 - 14, Sep., 2012 References

21. [[23] X. Wang et al, Paralleling Variable Block Size Motion Estimation of HEVC on Multicore CPU plus GPU platform, IEEE International Conference on Image Processing (ICIP 2013), Melbourne, Australia, Sep.15-18, 2013.Paralleling Variable Block Size Motion Estimation of HEVC on Multicore CPU plus GPU platform [24] “K.R. Rao, D.N. Kim, J.J. Hwang, Video Coding Standards” –Springer 2014. [25] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms-a survey,” Journal of Opto-Electronics Review, vol. 21, pp 86-102, Mar. 2013. [26] A. Abdelazim,W. Masri and B. Noaman "Motion estimation optimization tools for the emerging high efficiency video coding (HEVC)", SPIE vol. 9029, Visual Information Processing and Communication V, 902905, Feb. 17, 2014, doi:10.1117/12.2041166 [26] Software repository for HEVC - http://hevc.hhi.fraunhofer.de/http://hevc.hhi.fraunhofer.de/ [27] Video test sequences - http://forum.doom9.org/archive/index.php/t-135034.html or http://media.xiph.org/video/derf/http://forum.doom9.org/archive/index.php/t-135034.html http://media.xiph.org/video/derf/ [28] HM Software Manual - https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/ References

22. THANK YOU