1. IMPLEMENTATION AND PERFORMANCE ANALYSIS of Dirac VIDEO CODING STANDARD AND COMPARISON WITH AVS CHINA Under the guidance of Dr. K R. Rao Electrical Engineering Department The University of Texas at Arlington By Saumya Raval (1000746720) Saumya.raval@mavs.uta.edu Multimedia Processing (EE 5359) Project Report

2. Project Objective This project will give an overview of the working, performance and hardware requirements of Dirac and AVS-China codecs. The objective of this project is to analyze the performance of the baseline profiles of the Dirac and AVS China video codecs based on various factors like video quality, bit rates, compression ratio, etc. Also using sample videos, factors such as PSNR, MSE and SSIM [24] will be derived for two standard formats at various bit rates.

3. Introduction A software or a device that enables video compression and decompression is known as a video codec [1]. The need for video coding standards arose with the increased commercial interest in video communications. Video coding standards Dirac and AVS China are the latest standards adopted by Broadcasting Corporation BBC and China standards organization respectively [2].

4. Basic processing of Video files Figure 1: Block diagram of general video coding and decoding process [36]

5. Dirac [10] Dirac[10] is a video compression system developed by the British Broadcasting Corporation (BBC) utilizing motion compensation and wavelet transforms. Dirac video codec applications span from mobile, internet, Ultra HDTV to film and video production. The Dirac encoder architecture is shown in Figure 2. The decoder shown in Figure 3 performs the inverse operations.

6. Figure 2: Dirac encoder block diagram [10]

7. Figure 3: Dirac decoder block diagram [2]

8. Wavelet Transform Wavelet filters split the signal into 4 frequency sub-bands namely LL (Low-Low), LH (Low- High), HL (High-Low) and HH (High-High) In further stages the LL is decomposed. Daubechies wavelet filters are used to transform and divide the data in sub-bands

9. Figure 4: Decomposition of bands into sub-bands by applying wavelet transforms [3]

10. AVS China [5] AVS video codec is developed by the audio video coding standard working group of China[5]. AVS China comprises of four different profiles namely Jizhun, Jiben, Shenzan and Jiaqiang of which the Jiben profile (basic profile) is defined in AVS Part 7 for mobile applications [5].

11. ProfilesKey applications Jizhun profileTelevision broadcasting, HDTV, etc. Jiben profileMobility applications, etc. Shenzhan profileVideo surveillance, etc. Jiaqiang profileMultimedia entertainment, etc. Table 1: Applications of the various profiles of AVS China [5]

12. Table 2: Summary of profiles in AVS-Video [5]

13. Various profiles of AVS China [18] AVS-Video Jizhun profile (base profile) - First profile of AVS-Part2 - Focuses on digital video applications like commercial broadcasting and storage media, including high-definition applications. - Preferable for high coding efficiency on video sequences of higher resolutions, at the expense of moderate computational complexity. AVS-video Jiben profile (basic profile) - Mobility video applications featured with smaller picture resolution - Ability on error resilience is needed due to the wireless transporting environment AVS-Shenzhan profile (extended profile) - Solutions of standardizing the video surveillance applications AVS-Jiaqiang profile (enhanced profile) - Movie compression for high-density storage - Relatively higher computational complexity

14. Parts of AVS China [2] The following are the parts of AVS China. 1.System 2.Video 3.Audio 4.Conformance Test 5.Reference software 6.Digital media rights management 7.Mobile video 8.Transmit AVS via IP (Internet protocol) network 9.AVS file format 10.Mobile speech and audio coding

15. Figure 5: AVS China encoder block diagram [6]

16. Figure 6: AVS China decoder block diagram [5]

17. Layered Structure of AVS China [28] Figure 7: Layered data structure [28]

18. Coding tools in AVS China 8x8 Intra Predictions [18] – Decoded information in the current frame as the reference of prediction – Five luminance four chrominance – Four 8x8 luminance blocks can be predicted using one of the five intra-prediction modes – Prediction of the most probable mode is according to the intra- prediction modes of neighboring blocks Figure 8 : Neighbor pixels in luminance intra prediction [18]

19. Table 3: Contest based most probable intramode decision table [40]

20. Five luminance prediction modes are illustrated in Fig. 9. DC mode (mode2), diagonal down left (mode3) mode and diagonal down right mode (mode 4) and a three-tap low-pass filter (1,2,1) Figure 9: Five luminance intra prediction modes [18]

21. Inter prediction [5] – Derived from the decoded frames – Precision of motion vector in inter prediction is up to 1/4 pixel – Sub-pixel interpolation in AVS-video is called as two steps four taps (TSFT) interpolation [27] and three kinds of filters are applied – Filter of (-1, 5, 5, -1) to get the half-pixel reference pixel values as the first step and a filter of (1, 7, 7, 1) is applied for quarter-pixel reference pixel values either horizontally or vertically as the second step – Exception of the second step is that for quarter-pixel reference pixel values of e, g, p, r, a diagonal bilinear filter is used

22. Figure 10 : Interframe prediction block sizes

23. Figure 11: Position of integer pixels, 1/2 pixels and 1/4 pixels [18]

24. Experimental Results The software which has been used to perform Dirac is Dirac 1.0.2 [11] obtained from the official Dirac website [11], for AVS China Part 2 it is RM 9.0.2 [38] and for Part 7 it is RM 3.3.7 [39]. Microsoft Visual C++ 2008 express edition [34] has been used to run the code and build the project for all the codecs. After building the project, code will generate two application files namely encode.exe and decode.exe. We run these two files using appropriate and necessary parameters and obtain the final result which is a decoded file. The original file and decoded file are than evaluated using MSU video quality measurement tool. The values of PSNR, MSE and SSIM are obtained from it.

25. Dirac QCIF sequence: coastguard_qcif.yuv Height: 176, Width: 144 Total no. of frames: 300 Frames used: 100 Original File size: 3713 KB (Kilobytes) Frame Rate = 30 fps (Frames per Second) Quality Factor (QF) Compressed File Size (KB) Compression Ratio Bitrate (KBps)PSNR (dB)SSIMMSETime (sec) 029.7125:172.45622.340.511378.870.92 339.993:197.25426.3350.69151.2174.29 571.652:1174.40830.840.86753.47281.02 821317:1520.2237.1330.96612.5882.97 104508.25:11097.00941.760.9874.3381.58 Lossless2119.681.75:15163.3241001092

26. Original Sequence QF = 0 QF = 5QF = 10 Figure 12: Video quality at different QF values File: coastguard_qcif.yuv, QCIF format

27. CIF sequence: foreman_cif.yuv Height: 352, Width: 288 Total no. of frames: 300 Frames used: 100 Original File size: 14,850 KB Frame Rate: 30 fps QF Compressed File Size(KB) Compression Ratio Bitrate (KBps)PSNR (dB)SSIMMSETime (sec) 0107139:1261.15525.6740.738176.05385.803 3144103:1351.84230.7480.8654.729396.647 521370:1518.84934.6750.92122.162401.199 854227:11319.33739.7090.9686.951418.026 10120812:12953.93643.1350.9823.159429.424 Lossless78442:119100.71310010440.438

28. Original SequenceQF = 0 QF = 5QF = 10 Figure 13: Video quality at different QF values File: foreman_cif.yuv, CIF format

29. AVS China Part 2 QCIF sequence: coastguard_qcif.yuv Height: 176, Width: 144 Total no. of frames: 300 Frames used: 100 Original File size: 3713 KB Frame Rate = 30 fps Quantization Parameter (QP) Compressed File Size (KB) Compression Ratio Bitrate (KBps)PSNR (dB)SSIMMSETime (sec) 016282.25:16391.7260.530.990.0550.824 165786.5:12815.7944.640.962.2347.281 3212729:1950.9634.140.9225.0142.79 4820.9175:1192.9824.130.62250.7329.831 635.74638:133.718.860.34844.7127.601

30. Original Sequence QP = 0 QP = 32QP = 63 Figure 14: Video quality at different QP values File: coastguard_qcif.yuv, QCIF format

31. CIF sequence: foreman_cif.yuv Height: 352, Width: 288 Total no. of frames: 300 Frames used: 100 Original File size: 14,850KB Frame Rate = 30 fps QP Compressed File Size (KB) Compresson Ratio Bitrate (KBps)PSNR (dB)SSIMMSETime (sec) 056522.5:115734.9760.70.990.05210.896 16135211:13509.9745.160.981.97189.529 3223945:1542.8837.210.9412.35175.071 4856.6177:1118.4130.540.8457.32125.977 6319.1547:137.0224.250.68243.98117.542

32. Original SequenceQP = 0 QP = 32QP = 63 Figure 15: Video quality at different QP values File: foreman_cif.yuv, CIF format

33. AVS China Part 7 QCIF sequence: coastguard_qcif.yuv Height: 176, Width: 144 Total no. of frames: 300 Frames used: 100 Original File size: 3713KB Frame Rate = 30 fps QP Compressed File Size (KB) Compression Ratio Bitrate (KBps)PSNR (dB)SSIMMSETime (sec) 024371.5:16014.02580.9990.115.207 1611573.2:12855.745.240.9821.9411.642 324059:1996.0434.920.93120.928.863 4895.739:1235.0127.120.68125.925.597 6328132:168.7619.930.43660.125.179

34. Original SequenceQP = 0 QP = 32QP = 63 Figure 16: Video quality at different QP values File: coastguard_qcif.yuv, QCIF format

35. CIF sequence: foreman_cif.yuv Height: 352, Width: 288 Total no. of frames: 300 Frames used: 100 Original File size: 14,850KB Frame Rate = 30 fps QP Compressed File Size (KB) Compression Ratio Bitrate (KBps)PSNR (dB)SSIMMSE Time (sec) 0 77822:1 19145.5558.070.990.1 51.6 16 30005:1 7389.6345.860.981.68 39.446 32 90116.5:1 2214.2637.310.9412.05 31.429 48 28253:1 695.229.870.8366.94 21.522 63 125119:1 307.721.930.65416.32 20.329

36. Original SequenceQP = 0 QP = 32QP = 63 Figure 17: Video quality at different QF values File: foreman_cif.yuv, CIF format

37. Graphs File: coastguard_cif.yuv; Format: CIF 4:2:0

38. File: foreman_cif.yuv; Format: CIF 4:2:0

39. File: coastguard_qcif.yuv; Format: QCIF 4:2:0

40. File: foreman_qcif.yuv; Format: QCIF 4:2:0

41. File: coastguard_qcif.yuv; Format: QCIF 4:2:0

42. File: foreman_qcif.yuv; Format: QCIF 4:2:0

43. File: coastguard_qcif.yuv; Format: QCIF 4:2:0

44. File: foreman_qcif.yuv; Format: QCIF 4:2:0

45. The diagram for the computational time is based on the total time taken to encode 100 frames. The graph shows X-axis as picture quality. The X- axis signifies that picture quality is increasing as move right on X-axis. The picture quality taken in consideration for computational time is as below. For Dirac as we increase the QF, the quality increases. But for AVS China as we increase QP from 0 to 63, the quality of picture decreases. Hence for X-axis AVS China QP has been taken in reverse order. X-axisDiracAVS Part 2AVS Part 7 1063 2348 3532 4816 51000

46. MSU Video quality measurement tool [30] Fig 18: Snapshot of MSU tool

47. Conclusion It can be concluded from the results that Dirac stands out in terms of performance with respect to compression ratio, quality and applications over AVS China part 7 and part 2. The graphs and tabulations clearly show that the PSNR, MSE and SSIM [13] of the video sequences improve as the bit rate increases, while the bit rate is varied using the quantization parameter. Also from the computaional time shown we can see that Dirac gives better performance but takes the highest time to encode whereas AVS China Part 7 gives least performance but also takes least time to encode. Hence depending on the type of application computational time can also be a critical parameter that needs to be taken into consideration.

48. Abbreviations and Acronyms AIF – Adaptive Interpolation Filter AU - Access Unit AVC – Audio Video Coding AVS - Advanced Video Standard AVS-M - Audio Video Standard for mobile BBC - British Broadcasting Corporation CIF - Common Intermediate Format FPS – Frame per second HD - High Definition ICT - Integer Cosine Transform I-Frame - Intra Frame IEC - International Electrotechnical Commission IMS IP - Multimedia Subsystem IP – Internet Protocol ISO – International Organization for Standardization ITU-T International Telecommunication Union JVT - Joint Video Team KBps – Kilo bytes per second MB - Macroblocks MBPAFF - Macro Block Pair Adaptive Field Frame MPEG - Moving Picture Experts Group MSE – Mean Square Error NAL - Network Abstraction Layer PAFF - Picture Adaptive Field Frame P-Frame - Predicted Frame PIT – Pre-scaled Integer Transform PSNR – Peak Signal to Noise ratio QCIF - Quarter Common Intermediate Format QF – Quality Factor QP - Quantization Parameter RTP – Real-time Transport Protocol SD - Standard Definition SSIM – Structural Similarity Metric TV - Television VCEG - Video Coding Experts Group VCL – Video Coding Layer VLC - Variable Length Coding

49. References [1] T. Sikora, “Digital video coding standards and their role in video communications”, Signal processing for multimedia, J.S. Byrnes (Ed.), IOS press, pp. 225-251, 1999. [2] K. R. Rao and D. N. Kim, “Current video coding standards: H.264/AVC, Dirac, AVS China and VC-1,” IEEE 42nd southeastern symposium on system theory (SSST), pp. 1-8, March 2010. [3] K. Onthriar, K. K. Loo and Z. Xue, “Performance comparison of emerging Dirac video codec with H.264/AVC,” IEEE International conference on digital telecommunications, ICDT 2006, vol. 6, Page: 22, Issue: 29- 31, Aug. 2006. [4] X. F. Wang and D. B. Zhao, “Performance comparison of AVS and H.264/AVC video coding standards,” Journal of computer science and technology, Vol. 21, No. 3, pp.310-314, May 2006. [5] L. Yu, S. Chen and J. Wang, “Overview of AVS video coding standards,” Signal processing: image communication, Vol. 24, Issue 4, pp. 247-262, April 2009. [6] L. Fan et al, “Overview of AVS video standard”, IEEE International conference on multimedia and expo (ICME), Vol. 1, pp. 423 - 426, June 2004. [7] T. Borer and T. Davies, “Dirac video compression using open technology,” BBC EBU technical review, July 2005. [8] T. Borer, “Dirac coding: tutorial and implementation,” EBU networked media exchange seminar, EBU, Geneva, 22-23 June 2009. [9] Dirac specification, Version 2.2.3, Available: http://diracvideo.org/download/specification/dirac-spec-latest.pdfhttp://diracvideo.org/download/specification/dirac-spec-latest.pdf [10] BBC research on Dirac: http://www.bbc.co.uk/rd/projects/dirac/technology.shtmlhttp://www.bbc.co.uk/rd/projects/dirac/technology.shtml [11] Dirac video download source reference: http://diracvideo.org/download/http://diracvideo.org/download/ [12] YUV video sequences source: http://trace.eas.asu.edu/yuv/http://trace.eas.asu.edu/yuv/ [13] Z. Wang, et al, “Image quality assessment: From error visibility to structural similarity”, IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612, April 2004. [14] A. Ravi and K.R. Rao, “Performance analysis and comparison of the Dirac video codec with H.264/MPEG-4 part 10 AVC”, International journal of wavelets, multi-resolution and information processing, Vol. 09, Issue: 04, pp: 635-757, Jan. 2010. [15] L. Fan, “Mobile multimedia broadcasting standards”, Springer Publication, ISBN 978-0-387-78263-8, 2009. [16] AVS-China official website: http://www.avs.org.cnhttp://www.avs.org.cn

50. [14] A. Ravi's Thesis under Dr. K.R. Rao, “Performance analysis and comparison of the Dirac video codec with H.264/MPEG-4 part 10 AVC”, International journal of wavelets, multi-resolution and information processing, Vol. 09, Issue: 04, pp: 635-757, January 2010. [15] L. Fan, “Mobile multimedia broadcasting standards”, Springer Publication, ISBN 978-0-387-78263-8, 2009. [16] AVS-China official website: http://www.avs.org.cnhttp://www.avs.org.cn [17] Dirac video codec - A programmer's guide: http://dirac.sourceforge.net/documentation/code/programmers_guide/toc.htm [18] L. Yu et al, “Overview of AVS-Video: tools, performance and complexity,” SPIE VCIP, Vol. 5960, pp. 596021-1~596021-12, Beijing, China, July 2005. [19]Dirac developer support: Wavelet transform: http://dirac.sourceforge.net/documentation/algorithm/algorithm/wlt_transform.xht http://dirac.sourceforge.net/documentation/algorithm/algorithm/wlt_transform.xht [20] T. Davies, “The Dirac algorithm”: http://dirac.sourceforge.net/documentation/algorithm/ [21] H. Eeckhaut, et al, “Speeding up Dirac’s entropy coder”, 5th WSEAS International conference on multimedia, internet and video technologies, pp. 120-125, Greece, Aug. 2005. [22] M. Tun, K. K. Loo and J. Cosmas, “Semi-hierarchical motion estimation for the Dirac video codec”, 2008 IEEE international symposium on broadband multimedia systems and broadcasting, pp. 1-6, 2008. [23] M. Tun and W. A. C. Fernando, “An error-resilient algorithm based on partitioning of the wavelet transform coefficients for a DIRAC video codec”, Tenth international conference on information visualization, 2006, IV, Vol. 5-7, pp. 615 –620, Issue : July 2006. [24] W. Gao et al, “AVS - The Chinese next-generation video coding standard”, NAB, Las Vegas, 2004. [26] Power point slides by L.Yu, chair of AVS video : http://www.ee.uta.edu/dip/Courses/EE5351/ISPACSAVS.pdfhttp://www.ee.uta.edu/dip/Courses/EE5351/ISPACSAVS.pdf [27] R. Wang, et al., “Sub-pixel motion compensation interpolation filter in AVS”, 2004 IEEE International Conference on Multimedia and Expo, pp. 93-96, 2004. [28] Project proposal on "Low Complexity AVS-China Part-2 video using data mining techniques" by Jennie Abraham: http://www-ee.uta.edu/Dip/Courses/EE5359/Thesis%20Project%20table%20docs/ jennieproposal.doc [29] A. Ravi "Performance analysis and comparison of Dirac video codec with H.264/MPEG4 part 10 AVC“, M.S. Thesis, EE Dept., University of Texas at Arlington, Aug. 2009.

51. [30] MSU video quality measurement tool : http://compression.ru/video/quality_measure/video_measurement_tool_en.htmlhttp://compression.ru/video/quality_measure/video_measurement_tool_en.html [31] S Xu and S. Wang, “Main program flow analysis in AVS-M”, 2nd International conference on networking and digital society (ICNDS), Vol. 2, pp. 621-624, May 2010. [32] F. Zhou and W. Xiong, “Realization and optimization of AVS coding algorithm ”, International Conference on system science, engineering design and manufacturing informatization (ICSEM), Vol. 1, pp. 282-285, Oct. 2011. [33] Dirac programmers’ guide: http://diracvideo.org/download/dirac-research/documentation/programmers_guide-1.0.0.pdfhttp://diracvideo.org/download/dirac-research/documentation/programmers_guide-1.0.0.pdf [34] Microsoft Visual C++ 2008 express edition: http://msdn.microsoft.com/en-us/express/future/bb421473 [35] Dirac algorithm: http://diracvideo.org/download/dirac-research/documentation/http://diracvideo.org/download/dirac-research/documentation/ [36] J.Ostermann et al, “Video coding with H.264/AVC: Tools, Performance, and Complexity”, IEEE Circuits and Systems Magazine, vol. 4, Issue:1, pp. 7 – 28, Aug. 2004. [37] S.Kwon, A. Tamhankar and K.R. Rao, “Overview of H.264 / MPEG-4 Part 10”, J. Visual Communication and Image Representation, vol. 17, pp.186-216, April 2006. [38] AVS China software: Part 2: ftp://124.207.250.92/incoming/video_codec/AVS1_P2/RM09 [39] AVS China software: Part 7: ftp://124.207.250.92/incoming/video_codec/AVS1_P7