1. PERFORMANCE COMPARISON OF DAALA AND HEVC By Rohith Reddy Etikala rohith.Etikala@mavs.uta.edu 1001049552

2. TABLE OF CONTENTS Acronyms and Abbreviations Goal of the Project Overview of High Efficiency Video Coding (HEVC) – HEVC Encoder and Decoder – Features of HEVC Partitioning Prediction Transform and Quantization Entropy Coding In-loop Filters Overview of DAALA Metrics used – Peak Signal to Noise Ratio (PSNR) – Bjontegaard Delta Bitrate (BD-BR) and Bjontegaard Delta PSNR (BD-PSNR) Profiles used Test sequences used Results Conclusions References

3. Acronyms AVC: Advanced Video Coding BD-BR: Bjontegaard Delta Bitrate. BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. CABAC: Context Adaptive Binary Arithmetic Coding. CTB: Coding Tree Block. CTU: Coding Tree Unit. CU: Coding Unit. DCT: Discrete Cosine Transform. fps: frames per second. HEVC: High Efficiency Video Coding. HM: HEVC Test Model. IEC: International Electro-technical Commission. ISO: International Organization for Standardization. ITU-T: International Telecommunication Union- Telecommunication Standardization Sector. JCT: Joint Collaborative Team. JCT-VC: Joint Collaborative Team on Video Coding. MAD: Mean Absolute Difference. MC: Motion Compensation. MPEG: Moving Picture Experts Group. MSE: Mean Square Error. PB: Prediction Block. PSNR: Peak Signal to Noise Ratio. QP: Quantization Parameter SAO: Sample Adaptive Offset. TB: Transform Block. TU: Transform Unit. TF: Time Frequency. VCEG: Video Coding Experts Group.

4. Goal of the Project The main goal of this project is to implement/build the reference software for both DAALA and HEVC, use various profiles and test sequences [31] with varying resolutions and compare the performance with different metrics such as PSNR and BD-PSNR [9].Also, computational complexity of both the systems will be compared.

5. Overview of HEVC The High Efficiency Video Coding (HEVC) [12] is the latest video standard developed by Joint Collaborative Team on Video Coding (JCT-VC), a group of video coding experts from ITU-T Video Coding Experts Group and SO/IEC Moving Picture Experts Group (MPEG). As the demand for HD video (4K and 8K) increased, there is a need for stronger coding efficiency than H.264/AVC. Also, there is increased use of parallel processors. So, HEVC [25] has been introduced to support increased video resolution and parallel processing. HEVC obtains about 50% reduction in bit rate when compared to its predecessor H.264/AVC [1] at the same visual quality.

6. HEVC Codec Block Diagram of HEVC CODEC [11].

7. HEVC Encoder HEVC Encoder Diagram In-loop filters [32]

8. HEVC Decoder HEVC Decoder Block Diagram[21]

9. HEVC Tiles and Slices HEVC Slice and Tile Structure [3]. [21]

10. HEVC Block structure HEVC divides a picture into coding tree units(CTU) of 16x16,32x32, or 64x64 samples. The CTU can further be divided into smaller blocks using a quad tree structure, called a coding unit(CU), can further be split into prediction units(PU) and is also a root of transform quad tree.Each of the child nodes of the transform quad tree defines a transform unit(TU).The size of transforms used in the prediction error coding can vary from 4x4 to 32x32 samples. [25]

11. HEVC CTU Partitioning and Code Tree Structure CTU partitioning and code tree structure [18].

12. HEVC Intra Prediction modes Modes and directional orientations for intra prediction in HEVC [4].

13. HEVC Motion Estimation Process Illustration of Motion Estimation Process [27].

14. Other HEVC Features Transform and Quantization Residual CU is transformed by using block transforms such as integer DCT of sizes 32x32, 16x16 and 4x4. Then the transformed data is quantized [25]. Entropy Coding Context Adaptive Binary Arithmetic Coding (CABAC) is used to encode quantized transform coefficients, motion vector data and header data [25]. In-loop Filters HEVC employs in-loop filters such as deblocking and Sample Adaptive Offset (SAO) [3] to smoothen block transitions and remove ringing artifacts.

15. Overview of DAALA Daala is a video codec developed by Mozilla foundation, Xiph.Org foundation and other contributors [36]. The goal of this project is to provide a free implementation and performance superior to HEVC. Daala uses Lapped transforms rather than DCT. Perceptual vector quantization. Overlapping-block motion compensation. Chroma plane prediction from luma planes Multisymbol arithmetic encoding Time-frequency resolution switching.

16. Lapped Transform Illustration of one possible decomposition of a lapped transform, consisting of the DCT with pre-filters (P) and post-filters (P -1 ) straddling block boundaries [40].

17. Contd.. Pre-filter illustration in daala [40]

18. Prediction in the frequency domain Illustration of the daala intra predictor for the 4x4 block case [40].

19. Time/Frequency Resolution Switching Splitting and merging of blocks in daala [40]

20. Building DAALA Codec

21. DAALA Encoder and Decoder executables

22. DAALA Encoder arguments

23. DAALA Commands D:\daala\win32\Visual Studio\VS2010\Debug>encoder_example.exe -o deadline.ogg -l 50 deadline_cif.y4m File 'deadline_cif.y4m' is 352x288 29.970 fps 420 video. Compressing... 0:00:01.66 video: 450kbps done. D:\daala\win32\Visual Studio\VS2010\Debug>decoder_example.exe deadline.ogg -o deadline_ogg.y4m Encoded by Xiph's experimental encoder library Apr 13 2016 Ogg logical stream 60cb is Daala 352x288 29.97 fps video 50 frames Done.

24. HEVC Command D:\hm\bin\vc2013\Win32\Release>TAppEnco der.exe -c D:\hm\cfg\encoder_randomaccess_main.cfg - wdt 352 -hgt 288 -fr 30 -f 50 -i deadline_cif.y4m

25. Metrics Used Peak Signal to Noise Ratio (PSNR). Given a noise-free m x n monochrome image I and its noisy approximation K, MSE is defined as: PSNR in dB is defined via the Mean Square Error (MSE). Here, MAX I is the maximum possible pixel value of the image. The PSNR is defined as:

26. Contd… Bjontegaard Delta Bitrate (BD-BR) and Bjontegaard Delta PSNR (BD-PSNR): To objectively evaluate the coding efficiency of video codecs, Bjontegaard Delta PSNR (BD-PSNR) was proposed. Based on the rate-distortion (R-D) curve fitting [8][9], BD-PSNR provides a good evaluation of the R-D performance SSIM The structure similarity (SSIM) index is a method for measuring the similarity between two images. The SSIM index is a full reference metric; in other words, the measurement of image quality based on an initial uncompressed or distortion-free image as reference. SSIM is designed to improve on traditional methods like peak signal-to-noise ratio (PSNR) and mean square error (MSE), which have proven to be inconsistent with human eye perception. Complexity The complexity in terms of encoding time for both HEVC and DAALA will be compared in addition to PSNR, SSIM and BD-PSNR.

27. Profiles For HEVC, HM 16.9 [17] main profile is used for the project. Official Git repository code is used for daala.

28. Test Sequences Used[31] akiyo_qcif_176x144_30.yuv

29. Contd… foreman_cif_352x288_30.yuv

30. Results akiyo_qcif_176x144_30.yuv, Number of frames encoded = 300 QPPSNR in dBBit rate in kbps Encoding time in seconds 161.6020783.7592294.692 1548.6436105.4296150.640 3039.410622.2072117.047 5126.48263.7752109.659 Table 1: Results for akiyo_qcif_176x144_30.yuv sequence in HEVC random access configuration. HEVC Results: foreman_cif_352x288_30.yuv, Number of frames encoded = 300 QPPSNR in dBBit rate in kbps Encoding time in seconds 159.468711463.28802491.648 1544.96481682.77441399.865 3036.5274173.5560741.002 5125.90337.4681523.429 Table 2: Results for foreman_cif_352x288_30.yuv sequence in HEVC random access configuration.

31. Contd… DAALA Results: akiyo_qcif_176x144_30.yuv, Number of frames encoded = 300 Quality Factor (q) PSNR in dBBit rate in kbps Encoding time in seconds 153.7389739.260169.112 15026.59163.09670.406 25026.66502.58569.337 30023.92682.59768.095 Table 3: Results for akiyo_qcif_176x144_30.yuv sequence in DAALA. foreman_cif_352x288_30.yuv, Number of frames encoded = 300 Quality Factor (q) PSNR in dBBit rate in kbps Encoding time in seconds 151.96159151.6481269.923 15027.091632.067471.161 30024.381116.683469.474 51122.636512.287443.492 Table 4: Results for foreman_cif_352x288_30.yuv sequence in DAALA.

32. Contd…

34. Conclusions From this project it is observed that as the quality factor for the DAALA increases, bit-rate decreases, PSNR decreases and encoding time decreases. Similarly, for HEVC, as quantization parameter increases, bit-rate decreases, PSNR decreases and encoding time decreases. These results are expected as increasing the quality factor in DAALA or quantization parameter in HEVC will perform coarse quantization, which reduces the bitrate, encoding time and PSNR Over the next few years, DAALA plans to achieve high PSNR than HEVC at a lower bit-rate while maintaining almost same visual quality [41].

35. References [1] I.E.G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002. [2] K.R. Rao, D.N. Kim and J.J. Hwang, “Video Coding Standards: AVS China, H.264/MPEG-4 Part 10, HEVC, VP6, DIRAC and VC-1”, Springer, 2014. [3] G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012. [4] F. Bossen et al, “HEVC Complexity and Implementation Analysis”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1685-1696, Dec. 2012. [5] J. R. Ohm et al, “Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC)”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1669-1684, Dec. 2012. [6] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE International Conference on Multimedia and Expo (ICME), pp. 685-690, July 2010. [7] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC- L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.it- sudparis.eu/jct/doc_end_user/current_document.php?id=7243 [8] J. Vanne et al, “Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1885-1898, Dec. 2012. [9] G. Bjontegaard, “Calculation of Average PSNR Differences between RD Curves”, document VCEG- M33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001. [10] D. Grois et al, “Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/ MPEG-AVC Encoders”, available on: http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf [11] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html [12] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html [13] HEVC white paper-Ittiam Systems: http://www.ittiam.com/Downloads/en/documentation

36. Contd… [15] HEVC white paper-Elemental Technologies: http://www.elementaltechnologies.com/lp/hevc- h265-demystified-white-paper [16] Test Sequences: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ [17] Access to HM 16.9 Reference Software: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.9/ [18] Detailed Overview of HEVC/H.265 by Shevach Riabtsev https://app.box.com/s/rxxxzr1a1lnh7709yvih [19] Access the website for HEVC Chapter 1 for more references http://www.uta.edu/faculty/krrao/dip/Courses/EE5359/ [20] Access to HM 16.9 Software Manual: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.9/doc/software-manual.pdf [21] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012. [22] S. Jayaraman, S. Esakkirajan and T. Veerakumar, “Digital image processing”, McGraw Hill Education (India) Private Ltd., New Delhi, 2009. [23] G. Correa et al, “Performance and computational complexity assessment of high efficiency video encoders”, IEEE Trans. on Circuits and Systems for Video Technology, Vol.22, pp.1899-1909, Dec.2012. [24] L.-K.Liu and E.Feig, “A block based gradient descent search algorithm for block motion estimation in video coding,” IEEE Trans. on Circuits and Systems for Video Technology,Vol. 6, No. 4, pp. 419-422, Aug. 1996. [25] V. Sze and M. Budagavi, “Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial)”, IEEE International Symposium on Circuits and Systems (ISCAS), Melbourne, Australia, June 2014, available on http://www.rle.mit.edu/eems/publications/tutorials/ [26] D. Grois, B. Bross and D. Marpe, “HEVC/H.265 Video Coding Standard (Version 2) including the Range Extensions, Scalable Extensions, and Multiview Extensions, IEEE ICIP, Quebec City, Canada, 27 – 30 Sept. 2015. This tutorial is for personal use only [Password: a2FazmgNK ] https://datacloud.hhi.fraunhofer.de/owncloud/public.php?service=files&t=8edc97d26d46d4458a9c1a17964bf881

37. Contd… [27] M. J. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms – a survey,” Opto-Electronic Review, Vol. 21, pp 86-102, March2013. [28] W. Hong, “Coherent Block-Based Motion Estimation for Motion-Compensated Frame Rate Up-Conversion", IEEE International Conference on Consumer Electronics, pp. 165-166, Jan. 2010. [29] L.N.A. Alves and A. Navarro, " Fast Motion Estimation Algorithm for HEVC ", Proc. IEEE International Conf. on Consumer Electronics -ICCE Berlin, Germany, vol.11, pp. 11 - 14, Sep. 2012. [30] A. Abdelazim, W. Masri and B. Noaman "Motion estimation optimization tools for the emerging high efficiency video coding (HEVC)", SPIE vol. 9029, Visual Information Processing and Communication V, 902905, Feb. 17, 2014. [31] Video test sequences - http://forum.doom9.org/archive/index.php/t-135034.html or http://media.xiph.org/video/derf/ [32] G. J. Sullivan et al “Standardized Extensions of High Efficiency Video Coding (HEVC)”, IEEE Journal of selected topics in Signal Processing, vol. 7, pp.1001-1016, Dec. 2013. [33] M. Wien, “High efficiency video coding: Tools and specification”, Springer, 2015. [34] I.E.G. Richardson, “Coding video: A practical guide to HEVC and beyond”, Wiley, 2016. [35] V.Sze, M.Budagavi and G.J.Sullivan (Editors), “High Efficiency Video Coding (HEVC) –Algorithms and Architectures”, Springer, 2014. [37] Daala info: https://xiph.org/daala/ [38] Daala codec Git repository: https://git.xiph.org/?p=daala.git [39] Z.Wang et al, “Image quality assessment: From error visibility to structural similarity”, IEEE Transactions on Image Processing, vol.13, pp. 600- 612, April 2004. [40] Daala Codec demo - http://people.xiph.org/~xiphmont/demo/daala/demo1.shtml [41] T.J. Daede et al, " Daala: A Perceptually-Driven Next Generation Video Codec," in Proc. IEEE Data Compression Conference (DCC'16), Snowbird, UT, March 29- April 01, 2016.

38. Thank you