EE 5359: MULTIMEDIA PROCESSING PROJECT PERFORMANCE ANALYSIS OF INTEGER DCT OF DIFFERENT BLOCK SIZES USED IN H.264, AVS CHINA AND WMV9. Guided by Dr. K.R. Rao Presented by: Suvinda Mudigere Srikantaiah UTA ID:

Aim and Abstract Aim: To investigate performance analysis of integer DCT of block sizes 8X8, 16X16 and 32X32 used in H.264, AVS China and WMV9. Abstract: This project discusses how the use of larger transforms, especially in high resolution videos, can provide better performance. In particular, transforms of sizes larger than 4x4 or 8x8, especially 16x16 and 32x32 are proposed because of their increased applicability to the de- correlation of high resolution video signals.

Introduction to IntDCT  Discrete cosine transform has been serving as the basic elements of video coding systems.  The integer discrete cosine transform is an integer approximation of the discrete cosine transform.  It can be implemented exclusively with integer arithmetic.  It proves to be highly advantageous in cost and speed for hardware implementations [1].

DCT to IntDCT  DCT matrix elements are real numbers and for a 16-order DCT, 8 bits are needed to represent these numbers in order to ensure perfectly negligible image reconstruction errors due to finite-length number representation  If the transform matrix elements are integers, then it may be possible to have a smaller number of bit representation and at the same time zero truncation errors.  Moreover, the resultant cosine values are difficult to approximate in fixed precision integers, thus producing rounding errors in practical applications. Rounding errors can introduce enough error into the computations and alter the orthogonality property of the transform

Definition: ICT matrix is in the form [2,3]: I = KJ where I is the orthogonal ICT matrix K is a diagonal matrix whose elements take on values that serve to scale the rows of the matrix J so that the relative magnitudes of elements of the ICT matrix I are similar to those in the DCT matrix. The matrix J is orthogonal with elements that are all integers.

Transforms used in some standards StandardTransform 1. MPEG-4 part 10/H.2648 X 8, 4 X 4 integer DCT 2. WMV-98 X 8, 8 X 4, 4 X 8, 4 X 4 integer DCT 3. AVS ChinaAsymmetric 8 X 8 integer DCT Table no.1: Transforms used in standards H.264, WMV-9 and AVS china [4].

DCT The forward Discrete Cosine Transform (DCT) of N samples is formulated by [11] for u = 0, 1,..., N - 1, where The function f(x) represents the value of the x th sample of the input signal. F(u) represents a Discrete Cosine Transformed coefficient for u = 0, 1, …, N – 1 First of all we apply this transformation to the rows, then to the columns of image data matrix

IDCT The Inverse Discrete Cosine Transform (IDCT) of N samples is formulated by: for x = 0, 1,..., N – 1, where The function f(x) represents the value of the x th sample of the input signal. F(u) represents a Discrete Cosine Transformed coefficient for u = 0, 1, …, N – 1 For image decompression we use this DCT.

DCT II  The DCT-II is probably the most commonly used form, and is often simply referred to as "the DCT" [6].  Given an input function f(i,j) over two integer variables i and j (a piece of an image), the 2D DCT transforms it into a new function F(u,v), with integer u and v running over the same range as i and j. The general definition of the transform is: where i,u = 0,1,…,M − 1; j,v = 0,1,…, N − 1; and the constants C(u) (or C(v)) are determined by where l = u,v

OVERVIEW OF CODING STANDARDS H.264, AVS CHINA AND WMV9

Int DCT in H.264:  H.264 video coding standard uses a transform for reduction of spatial correlation, quantization for bitrate control, motion compensated prediction for reduction of temporal correlation, and entropy encoding for reduction of statistical correlation.  One of the important changes in H.264 to fulfill better coding performance was the introduction of Integer transform. It is multiplier free and reduces implementation complexity.  In general, transform and quantization require several multiplications resulting in high complexity for implementation. So, for simple implementation, the exact transform process is modified to avoid the multiplications. Then the transform and quantization are combined by the modified integer forward transform, quantization, scaling.

Int DCT in AVS China  Audio Video Coding Standard (AVS) is the national standard of China. Its Enhanced Profile (EP) targets at high definition video coding.  It is expected that the use of larger transform, especially in high resolution videos, can provide higher coding gain.  The order-16 and order-32 transform proposed is an extended version of the order-8 ICT adopted in AVS.  Without significant increase in complexity, order-8 transform matrix can be extended to order-16 and order-32 transform matrix

Int DCT in WMV9  Windows Media 9 Series includes a variety of audio and video codecs, which are key components for authoring and playback of digital media.  Floating point arithmetic is ruled out on the decoder side in wmv9 for several reasons, the important ones being the need to minimize decoder complexity, and the need to implement decoders that precisely match the specification so as to avoid mismatch.  Floating point operations are not very portable across processors—their definitions usually involve some measure of tolerance, making them unsuitable for perfectly matching implementations.  It is largely accepted that low-precision integer arithmetic is a desirable feature.

EXTENDING ORDER 8 INTEGER TRANSFORM TO ORDER 16 AND ORDER 32

Dyadic symmetry

(1) Order-8 transform matrix (1) T8: Order 8 transform matrix [5].

Extending order 8 to order 16 Denoting even symmetry with ‘E’ and odd symmetry with ‘O’ about the solid line represents mirror image and negative mirror image.

(2) Order-16 transform matrix derived from order-8 transform matrix (2) T16: Order 16 transform matrix [5]. (2)

H.264 The transform matrices of order 8, 16 and 32 for H.264 are shown below. Note the Orthogonality in all three cases:

AVS China The transform matrices of order 8, 16 and 32 for AVS China are shown below. Note the Orthogonality in all three cases:

WMV9 The transform matrices of order 8, 16 and 32 for WMV9 are shown below. Note the Orthogonality in all three cases:

PERFORMANCE ANALYSIS

Performance Evaluation: In finding efficiency of integer DCT, standard images are applied as an input signal. Transforms considered will be DCT, Integer DCT of different block sizes. The following operations are performed in this project for the purpose of performance analysis:  a) Variance distribution for I order Markov process, ρ = 0.9 (Plot and Tabulate)  b) Normalized basis restriction error vs. # of basis function (Plot and Tabulate)  c) Plot fractional correlation (0< ρ <1)

Comparison of performances of 8X8 ICT NDCTH.264WMV9AVS China a) Variances of transform coefficients

b) Normalized basis restriction error versus the number of basis (Order 8) NDCTH.264WMV9AVS China

Graph 1

Graph 2

Graph 3

Comparison of performances of 16X16 ICT a) Variances of transform coefficients NDCTH.264WMV9AVS China

b) Normalized basis restriction error versus the number of basis (Order 16) NDCTH.264WMV9AVSChina

Graph 1

Graph 2

Graph 3

Comparison of performances of 32X32 ICT a) Variances of transform coefficients NDCTH.264WMV9AVSChina

b) Normalized basis restriction error versus the number of basis (Order 32) NDCTH.264WMV9AVSChina

Graph 1

Graph 2

Graph 3

References: 1. N. Ahmed, T. Natarajan, and K. R. Rao, "Discrete Cosine Transform", IEEE Trans. Computers, vol. C- 32, pp , Jan W. K. Cham and Y. T. Chan” An Order-16 Integer Cosine Transform”, IEEE Trans. Signal proc. vol. 39, issue no. 5, pp – 1208, May W. K. Cham, “Development of integer cosine transforms by the principle of dyadic symmetry,” in Proc. Inst. Electr. Eng. I: Commun. Speech Vis., vol no. 4, pp. 276–282, Aug S. Kwon, A. Tamhankar, K.R. Rao, “Overview of H.264/MPEG-4 part 10”, Special issue on “ Emerging H.264/AVC video coding standard”, J. Visual Communication and Image Representation, vol. 17, pp , Apr W. Cham and C. Fong “Simple order-16 integer transform for video coding” IEEE ICIP 2010, Hong Kong, Sept R. Joshi, Y.A. Reznik and M. Karczewicz, “ Efficient large size transforms for high-performance video coding”, SPIE 0ptics + Photonics, vol. 7798, paper , San Diego, CA, Aug M. Costa and K. Tong, “A simplified integer cosine transform and its application in image compression”, Communications Systems Research Section, TDA Progress Report pp , Nov A.T. Hinds, “Design of high-performance fixed-point transforms using the common factor method”, SPIE 0ptics + Photonics, vol. 7798, paper , San Diego, CA, Aug

9. S. Chokchaitam, M. Iwahashi and N. Kambayashi, “Optimum word length allocation of integer DCT and its error analysis”, Elsevier, Signal Processing: Image Communication vol. 19, pp. 465– 478, July C Wei, P. Hao Q. Shi, “Integer DCT-based Image Coding”, National Lab on Machine Perception, Peking University Beijing, , China. 11. P.C. Yip and K.R. Rao, “ The transform and data compression handbook,” Boca Raton, FL: CRC Press, Y. Zeng, et al “Integer DCTs and Fast Algorithms”, IEEE Trans. Signal proc. vol. 49, No. 11, Nov P. Chen, Y. Ye and M. Karczewicz, “Video Coding Using Extended Block Sizes,” ITU-T Q.6/SG16, T09-SG16- C-0123, Geneva, Jan B. Lee, et al “A 16×16 Transform Kernel with Quantization for (Ultra) High Definition Video Coding,” ITU-T Q.6/SG16 VCEG, VCEG-AK13, Yokohoma, Japan, April G. Mandyam, N. Ahmed, and N. Magotra, “Lossless image compression using the discrete cosine transform”, Journal of Visual Communication and Image Representation, Vol.8, No.1, pp , March, W.Gao, et al “AVS - The Chinese next-generation video coding standard”, Joint development lab., Institute of computing science, Chinese academy of sciences, Beijing, China. 17. S. Srinivasan, et al “Windows Media Video 9: Overview and Applications,” Signal Processing: Image Communication, vol. 9, pp , Oct

THANK YOU!!!