Implementation and Analysis of Directional Discrete Cosine Transform in H.264 for Baseline Profile Shreyanka Subbarayappa Electrical Engineering Graduate Student The University of Texas at Arlington Advisor Dr. K. R. Rao, EE Dept, UTA Committee Members Dr. W. Alan Davis, EE Dept, UTA Dr Kambiz Alavi, EE Dept, UTA Good afternoon everyone. Firstly, I would like to thank all of you for taking the time off your day for my thesis defense. My name is Vidhya Vijayakumar and my thesis is titled “Low complexity H.264 to VC-1 Transcoder”. To say a little bit about myself, I started my graduate studies at UT Arlington in Fall 08 and started working with Dr Rao right away. I did an internship with Adobe during summer and fall 09 on HTTP video streaming. After that I worked with Dr Ahmad, my co-advisor, on a research project in parallel to my thesis. I would like to take this opportunity to thank Dr Rao and Dr Ahmad for giving me an enriching and very good learning experience.

Agenda Introduction to the field of research Motivation for the research Overview of H.264 Overview of DDCT Image Quality measures H.264 JM 18.0 settings Experimental Results Conclusions Future work References April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Introduction to the field of research April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

NEED FOR IMAGE OR VIDEO COMPRESION Introduction Importance of video Need for compression High bandwidth requirements Remove inherent redundancy Need for standardization Ensures interoperability 2011-2013 Coding Efficiency Network awareness Complexity 2005 2010 1999 1994 MPEG4 H.264 1992 MPEG1 Video Conferencing H.263 2003 Mobile Phone Hand PC Mobile TV SVC HDTV MPEG2 H.265/HEC / NGVC VC-1 In present information age, visual media has gained huge importance. From sports to education to cooking to news, everything is available in visual media. This picture gives us an idea about the different multimedia applications we use (Computers, cameras, TV, DVR etc). The problem with video is that there is a huge amount of data to represent it in the raw form. Because of the high bandwidth requirements of video, video compression is required. Video compression is the process of removal of redundancy in video. Due to the varied devices that use video applications, it is necessary to follow universal format to represent them to enable interoperability. So standardization of formats is necessary. There have been numerous organization like MPEG, ITU that have proposed different compression technologies like MPEG-1, MPEG-2, MPEG-4, H.261,H.262,H.263,H.264 etc. This figure describes the evolution of various standards. Year NEED FOR IMAGE OR VIDEO COMPRESION April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Lossless or Lossy Compression Lossless compression There is no information loss, and the image can be reconstructed exactly the same as the original Applications: Medical imagery, Archiving Lossy compression Information loss is tolerable. Applications: commercial distribution (DVD) and rate constrained environment where lossless methods cannot provide enough compression ratio April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Motivation for the research April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Motivation for implementing DDCT in H.264 Broadcast Streaming Content Server Internet Link Mobile Storage H.264 ISO media file format Choice of codecs Prevalence of H.264 Need for DDCT in H.264 New concept in the transform domain Better coding gain Better image quality Implemented in the other upcoming standards like H.265 Larger applications for H.264 in communication fields, data storage and streaming. As we see, the two standards compete in every direction, be it internet streaming, mobile applications, internet streaming or storage formats. The blue ray format has mandated the support of MPEG-2, H.264 and VC-1. VC-1 is comparable to H.264 in terms of quality at the same bit rates. The advantage VC-1 has over H.264 is that it is less complex. So transcoding H.264 to VC-1 will benefit devices by using less complex VC-1 in them. While there has been past work on transcoding MPEG-2 to H.264, H.264 to MPEG-2, VC-1 to H.264, there has been none on H.264 to VC-1. An application scenario will explain the need for transcoding H.264 to VC-1 better. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Overview of H.264 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

H.264: Overview Latest block-oriented motion-compensation-based codec. Good video quality at substantially lower bit rates. Better rate-distortion performance and compression efficiency than MPEG-2 [42]. Simple syntax specifications, very flexible. Network friendly. Wide variety of applications such as video broadcasting, video streaming, video conferencing, D-Cinema, HDTV. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

H.264 – Encoder [1] This is the basic encoder structure of H.264. The difference between the input video sample and the predicted sample is transformed, quantized and is entropy encoded to form the bitstream. Prediction can be spatial intra prediction or temporal inter prediction. The encoder has a decoder loop with in to avoid drifts between the encoder and the decoder. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

H.264 – Decoder [1] April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Profiles in H.264 [1] April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Tools introduced in FRExts and their classification under the new high profiles April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Overview of D-DCT April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Outline Problem: DDCT: Implementation Complexity - To replace the DCT-like transform for intra prediction residuals in AVC and the associated zigzag scan pattern Solution: Directional Discrete Cosine Transform (DDCT) DDCT: - The transforms Properties Implementation - Transform - Quantization Scanning pattern Complexity - Computation - Memory Implementation and Analysis of Directional Discrete Cosine Transform in H.264 April 16, 2012

Conventional DCT [3] - The 2-D discrete cosine transform (DCT) of a square or a rectangular block shape is used for almost all block-based transform schemes for image and video coding. + (M 1D-DCT s of Length N) - Implemented separately through two 1-D transforms, one along the vertical direction and another along the horizontal direction. - The conventional DCT seems to be the best choice for image blocks in which vertical and/or horizontal edges are dominating. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Limitations of conventional DCT Forward 2D DCT (NXM) [3] Inverse 2D DCT (NXM) [3] x(n,m) = Samples in the 2D data domain. XC2 (k, l) = Coefficients in the 2D-DCT domain Limitations of conventional DCT It is not very efficient when the conventional DCT is applied to an image block in which other directional edges dominate. When the first 1-D DCT (vertical or horizontal) is applied, the nonzero coefficients are not well aligned across different columns (or rows). Consequently, the second 1-D DCT may produce more nonzero coefficients April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Modes Of DDCT [4] Mode 0 – Vertical Mode 1 – Horizontal Mode 2 – DC (Ignored for DDCT modes) Mode 3 – Diagonal down left Mode 4 – Diagonal down right Mode 5 – Vertical right Mode 6 – Horizontal down Mode 7 – Vertical left Mode 8 – Horizontal up April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Mode 3 DDCT- Diagonal Down Left Coefficients PIXELS Coefficients STEP 2 STEP 1 STEP 3 Step 1: (X00, X01,…… ,X32, X33)- Pixels in the 2-D spatial domain. Step 2: 1D- DCT is performed for the 4X4block in diagonal down-left position with lengths L=1, 2, 3, 4, 3, 2, 1. (A,B,C,……O,P)- coefficients in the DCT domain. Step 3: The coefficients of step2 after 1D DCT are arranged vertically as shown in the figure. Apply Horizontal 1D- DCT for lengths L=7, 5, 3 and 1 and arranged in the same pattern April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

same pattern as shown in the figure step 4. Coefficients Coefficients STEP 4 STEP 5 Step 4: Apply Horizontal 1D- DCT for lengths L=7, 5, 3 and 1, the coefficients are arranged in the same pattern as shown in the figure step 4. Step 5: After Step 4, move all 2D (4X4) Directional DCT coefficients to the left. Implement quantization followed by 2D VLC for compression/coding along zig-zag scan. This scanning helps to increase the runlength of zero (transform) coefficients leading to reduced bit rate in the 2D-VLC coding (similar to JPEG [12]). April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Mode 4 DDCT- Diagonal Down Right Coefficients PIXELS Coefficients STEP 3 STEP 1 STEP 2 Step 1: X00, X01, ….., X33 are the pixels in the 2D spatial domain. Step 2: 1D DCT is performed for the 4X4 block in diagonal down-right position with lengths L= 1, 2, 3, 4, 3, 2 and 1. Step 3: The coefficients of step 2 after 1 D DCT are arranged vertically in the same pattern as shown in step 3. Then apply horizontal 1 D DCT for lengths L = 7, 5, 3 and 1 and arrange in the same pattern. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

arranged the same pattern as shown in step4. Coefficients STEP 4 STEP 5 Step 4: Apply horizontal 1 D DCT for lengths L= 7, 5, 3 and 1. The coefficients are arranged the same pattern as shown in step4. Step 5: After step 4, move all 2D (4X4) directional DCT coefficients to the left. Implement quantization followed by 2D VLC for compression/coding zigzag scan. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Mode 5 DDCT- Vertical Right Coefficients PIXELS Coefficients STEP 1 STEP 2 STEP 3 Step 1: X00, X01, ….., X33 are the pixels in the 2D spatial domain. Step 2: 1D DCT is performed for the 4X4 block in vertical-right position with lengths L= 2,4,4,4,2. Step 3: The coefficients of step 2 after 1 D DCT are arranged vertically in the same pattern as shown in step 3. Then apply horizontal 1 D DCT for lengths L = 5, 5, 3 and 3and arrange in the same pattern. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Coefficients STEP 4 STEP 5 Step 4: Apply horizontal 1 D DCT for lengths L= 5, 5, 3 and 3. The coefficients are arranged the same pattern as shown in step 4. Step 5: After step 4, move all 2D (4X4) Directional DCT coefficients to the left. Implement quantization followed by 2D VLC for compression/coding zigzag scan as shown in step 5. This scanning helps to increase the run-length of zero (transform) coefficients leading to reduce bit rate in 2D-VLC coding (similar to JPEG). April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Mode 6 DDCT- Horizontal down Coefficients Coefficients PIXELS STEP 1 STEP 2 STEP 3 Step 1: X00, X01, ….., X33 are the pixels in the 2D spatial domain. Step 2: 1D DCT is performed for the 4X4 block in Horizontal down position with lengths L= 2, 4, 4, 4 and 2. Step 3: The coefficients of step 2 after 1 D DCT are arranged vertically in the same pattern as shown in step 3. Then apply horizontal 1 D DCT for lengths L = 5, 5, 3 and 3 and arrange in the same pattern. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Coefficients STEP 4 STEP 5 Step 4: Apply horizontal 1 D DCT for lengths L= 5, 5, 3 and 3. The coefficients are arranged the same pattern as shown in step 4. Step 5: After step 4, move all 2D (4X4) directional DCT coefficients to the left. Implement quantization followed by 2D VLC for compression/coding zigzag scan as shown in step 5. This scanning helps to increase the run-length of zero (transform) coefficients leading to reduce bit rate in 2D- VLC coding (similar to JPEG). April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Mode 7 DDCT- Vertical Left Coefficients PIXELS Coefficients STEP 1 STEP 2 STEP 3 Step 1 X00, X01, ….., X33 are the pixels in the 2D spatial domain. Step 2: 1D DCT is performed for the 4X4 block in Vertical left position with lengths L= 2,4,4,4 and 2 as shown in step 3. Step 3: The coefficients of step 2 after 1 D DCT are arranged vertically in the same pattern as shown in step 3. Then apply horizontal 1 D DCT for lengths L = 5, 5, 3 and 3 and arrange in the same pattern. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Coefficients STEP 5 STEP 4 Step 4: Apply horizontal 1 D DCT for lengths L= 5, 5, 3 and 3. The coefficients are arranged the same pattern. Step 5: After step 4, move all 2D (4X4) Directional DCT coefficients to the left. Implement quantization followed by 2D VLC for compression/coding zigzag scan as shown in step 3. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Mode 8 DDCT- Horizontal Up Coefficients PIXELS Coefficients STEP 2 STEP 3 STEP 1 Step 1: X00, X01, ….., X33 are the pixels in the 2D spatial domain. Step 2: 1D DCT is performed for the 4X4 block in Horizontal Up position with lengths L= 2, 4, 4, 4 and 2 as shown step 2. Step 3: The coefficients of step 2 after 1 D DCT are arranged vertically in the same pattern as shown in step 3. Then apply horizontal 1 D DCT for lengths L = 5, 5, 3 and 3 and arrange in the same pattern. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Coefficients STEP 4 STEP 5 Step 4: Apply horizontal 1 D DCT for lengths L= 5, 5, 3 and 3. The coefficients are arranged the same pattern as shown in step 4. Step 5: After step 4, move all 2D (4X4) directional DCT coefficients to the left. Implement quantization followed by 2D VLC for compression/coding zigzag scan as shown in step 5. This scanning helps to increase the run-length of zero (transform) coefficients leading to reduce bit rate in 2D-VLC coding (similar to JPEG [4]). April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Obtaining Mode 3 from Mode 4 Pixels Pixels Pixels Rotate pixels by –pi/2 ( counterclock wise by 90°) to get Mode 4 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Step By STEP Mode Change -π/2 = MODE 3 Pixels Coefficients Pixels MODE 4 Step 1: Rotate the pixels by –π/2 Step 2: Perform 1-D DCT with Length = 1, 2, 3, 4, 3, 2, 1 Step 3: We get the coefficients of mode 3 (Diagonal Down Left) from mode 4 (Diagonal Down Right) as shown in figure 3. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Obtaining Mode 6 from Mode 5 Pixels Pixels Pixels Rotate pixels by reflecting across the diagonal axis to get Mode 6 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Obtaining Mode 7 from Mode 5 Pixels Pixels Pixels Rotate pixels by reflecting across the horizontal axis to get Mode 7 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Obtaining Mode 8 from Mode 5 Pixels Pixels Pixels Rotate pixels by pi/2 (clockwise by 90°) to get Mode 8 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Eigen or Basis Images Mapping of a 2D data array into a 2D DCT domain implies decomposing the 2D data array into the basis images of the DCT. Computation of basis images for diagonal down left (a) The original 4X4 block with diagonal down left computation (b) The 1 D DCT of coefficients for lengths 7, 5, 3 and 1 for basis image (0, 0) (c) The 1 D DCT of coefficients for lengths 7, 5, 3 and 1 for basis image (0,1) (d) The 1 D DCT of coefficients for lengths 7, 5, 3 and 1 for basis image (3,3) April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Mode 3 Diagonal Down Left Eigen Image (1,1) matrix Computation for 4X4 block Step 1: Horizontal 1D-DCT for length =7, 5, 3, 1 Step 2: Coefficients of 1D-DCT for length =7, 5, 3, 1 Step 3: Put back the coefficients in the block form Step 4: 1 D DCT of diagonal down left with lengths = 1, 2, 3, 4, 3, 2 and 1 Step 5: Putting back the coefficients of step 4 1 D-DCT we get (1,1) basis image for 4X4 block

MODE 3 - Diangonal down left basis images for 4X4 block of an image April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

MODE 3 - Diangonal down left basis images for 8X8 block of an image April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

MODE 0 or 1 – Vertical or Horizontal basis images for 8X8 block of an image April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

MODE 5 – Vertical right basis images for 8X8 block of an image April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Computation of DDCT for an image April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Image Quality Measures April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Image Quality Measures Criteria to evaluate compression quality Two types of quality measures Objective quality measure- PSNR, MSE Structural quality measure- SSIM [29] SSIM emphasizes that the human visual system is highly adapted to extract structural information from visual scenes. Therefore, structural similarity measurement should provide a good approximation to perceptual image quality. MSE and PSNR for a NxM pixel image are defined as where x is the original image and y is the reconstructed image. M and N are the width and height of an image and ‘L’ is the maximum pixel value in the NxM pixel image. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

The SSIM index is defined as a product of luminance (l), contrast (c) and structural (s) comparison functions. where , α>0, β>0 and γ >0 are parameters used to adjust the relative importance of the three components where μ is the mean intensity, and σ is the standard deviation as a round estimate of the signal contrast. C1 and C2 are constants. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

H.264 JM18.0 [24] settings April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

4.4.2 Encoder Configuration in JM 18.0 FramesToBeEncoded = 1 #Number of Frames to be coded ProfileIDC = 66 # Profile IDC (66 = baseline, 77 = main, 88 = extended; FREXT Profiles: 100 = High, 110= High 10, 122= High 4:2:2, 244 = High 4:4:4, 44= CAVLC 4:4:4 Intra, 118 = Multiview High Profile,128 = Stereo High Profile) IntraProfile = 1 # Activate Intra Profile for FRExt (0: false, 1: true) #(e.g. ProfileIDC = 110, IntraProfile = 1 => High 10 Intra Profile) Transform8X8Mode = 0 # (0: only 4X4 transform, 1: allow using 8X8 transform additionally, 2: only 8X8 transform Transform 16X16Mode=0 #(0: no 16X16 mode, 1: allow 16X16 mode) Input YUV file: foreman_qcif.yuv Output H.264 bitstream: test.264 Output YUV file: test_rec.yuv YUV format: YUV 4:2:0 Frames to be encoded: 1 Frequency used for encoded bitstream: 30.00 fps DistortionSSIM = 1 # Compute SSIM distortion (0: disable/default, 1: enabled) April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Experimental Results and Graphs April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

QCIF and CIF CIF (Common Intermediate Format), is a format used to standardize the horizontal and vertical resolutions in pixels for sequences in video signals, commonly used in video teleconferencing systems. The CIF "image sizes" were specifically chosen to be multiples of macroblocks (i.e. 16 × 16 pixels) due to the way that discrete cosine transform based video compression/decompression is handled. So, by example, a CIF-size image (352 × 288) corresponds to 22 × 18 macroblocks QCIF means "Quarter CIF". To have one fourth of the area as "quarter" implies the height and width of the frame are halved. QCIF-size image is 176 x 144. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Bit Rate (kbps) QP (I frame) PSNR in dB MSE SSIM 5590.56 79.159 0.00079 1 5232.48 4 68.825 0.00852 3891.84 8 57.204 0.12378 0.9995 2168.16 16 48.86 0.84553 0.9965 1088.88 24 41.803 4.29364 0.9846 745.68 28 38.892 8.39157 0.9735 331.92 36 33.173 31.32035 0.9342 152.64 44 27.981 103.5018 0.8388 72 51 23.335 301.693 0.6703 Image metrics for Foreman QCIF sequence in integer DCT implementation in H.264 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Bit Rate(kbps) PSNR in dB MSE SSIM 5486.96 90.357 0.00006 1 5201.96 82.436 0.00068 3882.53 69.689 0.0014 2264.63 60.147 0.00842 0.9996 1153.84 52.976 0.55385 0.9972 686.54 41.876 2.43788 0.9925 302.53 38.653 10.2537 0.9801 142.53 34.642 50.4376 0.9208 67 30.764 110.268 0.8674 Image metrics for Foreman QCIF sequence in DDCT implementation in H.264 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

QP (I frame) Encoding Time of Int-DCT (sec) Encoding Time of DDCT (sec) 10.876 18.96 4 10.032 18.096 8 9.183 17.264 16 7.292 15.367 24 5.666 12.5437 28 4.968 11.0642 36 4.067 10.853 44 3.484 9.638 51 3.073 8.428 Encoding Time of I frame for Foreman QCIF sequence in DDCT and Int-DCT implementation in H.264 April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

PSNR v/s bit rate for DDCT and integer DCT for foreman QCIF sequence April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

MSE v/s bit rate for DDCT and integer DCT for foreman QCIF sequence April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

SSIM v/s bit rate for DDCT and integer DCT for foreman QCIF sequence April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Encoding time v/s quantization parameter for DDCT and integer DCT for Foreman QCIF sequence April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Test sequence used for simulation Bit Rate:72kbits/frame Bit Rate:67kbits /frame April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Properties of DDCT Adaptivity: Unlike AVC in which the same DCT-like transform is applied to the intra prediction errors for all intra prediction modes of the same block size (4x4, 8x8, or 16x16), DDCT assigns a different transform and scanning pattern to each intra prediction mode. These transforms and scanning patterns are designed taking into account the intra prediction direction. Directionality: Since the intra prediction mode is known, the DDCT is designed with the knowledge of the intra prediction direction. By first applying the transform along the prediction direction, DDCT has the potential to minimize the artifacts around the object boundaries. Symmetry: Although there are 22 DDCTs for 22 intra prediction modes (9 modes for 4x4, 9 modes for 8x8, and 4 modes for 16x16), these transforms can be derived, using simple operators such as rotation and/ or reflection, from only 7 different core modes. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Conclusions April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Directional DCT has a better coding gain when compared to integer DCT PSNR value for DDCT is more when compared to Integer DCT. MSE value of DDCT is less compared to integer DCT for the same bit rates. SSIM graph shows that the value obtained for different bit rates is almost 1 for DDCT when compared to Integer DCT. Foreman frame of QCIF format gives a better quality image obtained from DDCT with respect to the output obtained from Integer DCT. Drawback of DDCT Encoding time for DDCT is more when compared to integer DCT. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Future Work April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

Future work DDCT can be extended to other video standards that use integer DCT in the transform domain. It can be extended for the entire video – inter frame prediction. It can be extended to other profiles in H.264 like main and extended profiles. Only 8 modes are described in this research. These can be extended to other directional modes. Payoff between increasing complexity and improved visual quality can be investigated. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

References April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

E. Lallana and M. Uy, “The Information Age,” UNDP-APDIP, 2003. I. E.G. Richardson, “H.264 and MPEG-4 video compression: video coding for next-generation multimedia”, Wiley, 2003. N. Ahmed, T. Natarajan, and K. R. Rao, “Discrete cosine transform,” IEEE Trans. Comput., vol. C-23, pp. 90-93, Jan. 1974. K. R. Rao and P. Yip, “Discrete cosine transform: Algorithms, advantages, applications,” Boca Raton FL: Academic Press, 1990. B. Zeng and J. Fu, “Directional discrete cosine transforms - A new framework for image coding”, IEEE Trans. on Circuits and Systems for Video Technology, vol. 18, no. 3, pp. 305-313, Mar. 2008. B. Zeng and J. Fu, “A compensation techniques in directional DCT’s”, IEEE International Symposium on Circuits and Systems, pp. 521- 524, June, 2007. E. Lallana and M. Uy, “The Information Age,” UNDP-APDIP, 2003. Open source article, “Digital Revolution,” Wikipedia Foundation, http://en.wikipedia.org/wiki/Digital_Revolution K. Sayood, "Introduction to data compression,” 3rd Edition, Morgan Kaufmann Publisher Inc., 2006. R. Schafer and T. Sikora, "Digital video coding standards and their role in video communications," Proceedings of the IEEE, Vol. 83, pp. 907-923, Jan. 1995. Information technology-generic coding of moving pictures and associated audio information: ISO/IEC 13818-2 (MPEG-2) Std. Advanced video coding for generic audiovisual services, ITU-T Rec. H.264 / ISO / IEC 14496-10, Nov. 2009. I. Ahmad et al, “Video transcoding: An overview of various techniques and research issues”, IEEE Trans. on Multimedia, vol. 7, pp. 793-804, Oct. 2005. Open source article, “H.264/MPEG-4 AVC,” Wikipedia Foundation, http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

14. S. Kwon, A. Tamhankar and K. R. Rao, ”Overview of H 14. S. Kwon, A. Tamhankar and K.R. Rao, ”Overview of H.264 / MPEG-4 Part 10”, J. Visual Communication and Image Representation, vol. 17, pp.186-216, April 2006. 15. T. Wiegand and G. J. Sullivan, “The H.264 video coding standard”, IEEE Signal Processing Magazine, vol. 24, pp. 148-153, March 2007. 16. A. Puri et al, “Video coding using the H.264/ MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp: 793 – 849, Oct. 2004. 17. G. Sullivan, P. Topiwala and A. Luthra, “The H.264/AVC advanced video coding standard: Overview and introduction to the fidelity range extensions”, SPIE conference on Applications of Digital Image Processing XXVII, vol. 5558, pp. 53-74, Aug. 2004. 18. K. R. Rao and P. C. Yip, “The transform and data compression handbook”, Boca Raton,FL: CRC press, 2001. 19. T. Wiegand et al, “Overview of the H.264/AVC video coding standard”, IEEE Trans. on Circuits and Systems for Video Technology, vol. 13, pp. 560-576, Jul. 2003. 20. T. Wiegand and G. J. Sullivan “The picturephone is here: Really” IEEE spectrum, vol.48, pp.50-54, Sept.2011. 21. I. Richardson, “The H.264 advanced video compression standard”, Wiley, 2nd edition, 2010. 22. Intra prediction modes in H.264. Website: http://www.vcodex.com/files/h264_intrapred.pdf F. Kamisli and J. S. Lim, “Video compression with 1-d directional transforms in H.264/AVC”, IEEE ICASSP, pp. 738-741, Mar. 2010. 24. H.264/AVC reference software. Website: http://iphome.hhi.de/suehring/tml/download 25. Intra coding with directional DCT and directional DWT, Document: JCTVC-B107_r1 26. Directional Discrete Transform JCTV Website: http://wftp3.itu.int/av-arch/jctvc-site/2010_07_B_Geneva/JCTVC-B107.zip 27. B.Chen, H.Wang and L.Cheng, “Fast directional discrete cosine transform for image compression”, Opt. Eng. vol. 49, issue 2, article 020101, Feb. 2010. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

28. C. Deng et al, “Performance analysis, parameter selection and extensions to H.264/AVC FRExt for high resolution video coding”, J. Vis. Commun. Image R., vol. 22 (In Press), Available on line, Feb. 2011. 29. Z.Wang et al, “Image quality assessment: From error visibility to structural similarity”, IEEE Trans. on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. 30. W.Zhao, J.Fan and A.Davari, “H.264-based wireless surveillance sensors in application to target identification and tracking”, i-manager’s Journal on Software Engineering, vol.4, no. 2, Oct. 2009. 31. Website: http://web.eng.fiu.edu/fanj/pdf/J5_i-manager09h264_camera.pdf 32. W.Zhao et al, “H.264-based architecture of digital surveillance network in application to computer visualization”, i-manager’s Journal on Software Engineering, vol.4, no. 4, Apr. 2010. 33. Directional Discrete Cosine Transform theory Website: http://web.eng.fiu.edu/fanj/pdf/J8_i-mgr10architecture_camera.pdf 34. D. Marpe, T. Wiegand and G. J. Sullivan, “The H.264/MPEG-4 AVC standard and its applications”, IEEE Communications Magazine, vol. 44, pp. 134-143, Aug. 2006. 35. Website: http://iphome.hhi.de/wiegand/assets/pdfs/h264-AVC-Standard.pdf 36. F. Kamisli and J. S. Lim, “Transforms for motion compensation residual”, IEEE ICASSP, pp.789-792, Apr. 2009. 37. Z.Wang, E.P.Simoncelli and A.C.Bovik, “Multi-scale structural similarity for image quality assessment”, Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, vol. 2, Nov. 2003. 38. C.L.Chang and B.Girod, “Direction-adaptive partitioned block transform for image coding”, 15th IEEE International Conference on Image Processing, pp. 145-148, Oct. 2008. 39. H.Xu, J.Xu and F.Xu, “Lifting-based directional DCT-like transform for image coding”, IEEE Trans. on Circuits and Systems for Video Technology, vol. 17, issue 10, pp. 1325-1335, Oct. 2007. 40. J.Xu, B.Zeng and F.Wu, “An overview of directional transforms in image coding”, Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 3036-3039, Aug. 2010. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

41. MPEG-1 basics Website: http://en.wikipedia.org/wiki/Mpeg-1 44. H.261 basics Website: http://en.wikipedia.org/wiki/H.261 45. H.262 basics Website: http://en.wikipedia.org/wiki/H.262 46. H.263 basics Website: http://en.wikipedia.org/wiki/H.263 47. DFT basics Website: http://en.wikipedia.org/wiki/Discrete_Fourier_transform 48. K. R. Rao and J. J. Hwang, “Techniques and standards for image/video/audio coding”, Prentice Hall, 1996. April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264

QUESTIONS ? April 16, 2012 Implementation and Analysis of Directional Discrete Cosine Transform in H.264