1. Low Complexity H.264 to VC-1 Transcoder Vidhya Vijayakumar Electrical Engineering Graduate Student The University of Texas at Arlington Advisor Dr. K. R. Rao, EE Dept, UTA Co-Advisor Dr. I. Ahmad, CSE Dept, UTA

2. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 2 Agenda  Introduction to the field of research  Motivation for the research  Overview of H.264  Overview of VC-1  H.264, VC-1 Comparison  Overview of Transcoding  Proposed H.264 to VC-1 Transcoder  Results  Conclusions  Future work  References

3. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 3 Introduction to the field of research

4. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 4 Introduction  Importance of video  Need for compression – High bandwidth requirements – Remove inherent redundancy  Need for standardization – Ensures interoperability Year Coding Efficiency Network awareness Complexity 2005 2010 1999 1994 MPEG4 H.264 1992 MPEG1 Video Conferencing H.263 2003 Mobile Phone Hand PC Mobile TV SVC HDTV MPEG2 H.265/HEC / NGVC VC-1

5. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 5 Motivation for the research

6. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 6 Motivation for a H.264 to VC-1 transcoder  Choice of codecs –Prevalence of H.264 and VC-1  Need for transcoding –Co-existence in broadcast, streaming, mobile communication, storage etc. –VC-1 is comparable to H.264 on subjective quality –VC-1 being less complex than H.264 – Favorable for limited resource devices Broadcast Streamin g Content Server Internet Link Mobile Storage H.264 ISO media file format

7. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 7  An application scenario –Network uses H.264 and all end device have complex decoders –With transcoder, complexity shifted to a single point –Devices can use less complex VC-1 codec with transcoder H.264 to VC-1 Transcoder Motivation for a H.264 to VC-1 transcoder

8. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 8 Overview of H.264/AVC

9. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 9 H.264: Overview  Latest block-oriented motion-compensation-based codec.  Good video quality at substantially lower bit rates.  Better rate-distortion performance and compression efficiency than MPEG-2.  Simple syntax specifications, very flexible.  Network friendly.  Layered structure - consists of two layers: Network Abstraction Layer (NAL) and Video Coding Layer (VCL).  Wide variety of applications such as video broadcasting, video streaming, video conferencing, D-Cinema, HDTV.

10. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 10 H.264 - Encoder

11. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 11 Design Features Highlights  Features for enhancement of prediction –Directional spatial prediction for intra coding 9 intra 4x4 modes + 4 intra 16x16 modes + 9 intra 8x8 modes –Variable block-size motion compensation with small block size 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4 –Quarter-sample-accurate motion compensation –Multiple reference picture motion compensation –In-the-loop deblocking filtering to remove blocky artifacts  Features for improved coding efficiency –Small block-size transform – 4x4 and 8x8 integer DCT –Exact-match inverse transform –Short word-length transform –Hierarchical block transform –Arithmetic entropy coding –Context-adaptive entropy coding

12. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 12 Overview of VC-1

13. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 13 VC-1 : Overview  Informal name of the SMPTE 421M video codec  Standard initially developed by Microsoft – WMV 9  Supported standard for blu-ray discs and windows media video  Alternative to H.264/AVC  Better visual quality when compared with H.264* and MPEG-2 demonstrated in independent tests.  Prevalent codec in Microsoft’s ASF files, Silver light framework, X- Box 360 and Play station 3  Delivers HD content at bit rates as low as 6-8 Mbps  Requires less computational power  Block-oriented motion-compensation, DCT based codec  Coding tools for interlaced and progressive encoding * - results compared before the FRExts

14. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 14 VC-1Codec

15. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 15 Design Feature Highlights  Adaptive variable transform size (16 bit transforms) –8x8, 8x4, 4x8, 4x4  DC/AC intra prediction –no spatial prediction, always uses 8x8 transform size  Simple motion estimation – block sizes of 16x16 and 8x8 only, ¼ pixel accuracy  Deblocking filter –Overlap transform –In loop filtering  Multiple scanning patterns  Quantization with dead zone (uniform and non uniform quantization)  Bit plane coding  Huffman coding  Intensity compensation  Range reduction

16. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 16 Comparison of H.264 and VC-1

17. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 17 H.264 Baseline Vs VC-1 Simple FeatureH.264 BaselineVC-1 Simple Picture typeI, P Transform Size4x44x4, 4x8, 8x4, 8x8 TransformInteger DCT Intra Prediction4x4, 16x16 spatial Frequency domain DC and AC Prediction Motion Compensation Block Size 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4 16x16, 8x8 Total MB Modes7 inter + (9 + 4) intra2 inter + 1 intra Motion Vector resolution ¼ pixel In loop filterDeblockingDeblocking, Overlap transform Reference FramesSingle Entropy codingCAVLCAdaptive VLC

18. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 18 Performance Comparison Codec RankingWMV-9H.264/AVCMPEG-2 Sequence 1 – Dick Tracey11 (0)4 (-0.4) Sequence 2 – Titan12 (-0.3)3 (-0.4) Sequence 3 – Harry Potter12 (-0.2) Sequence 4 – Stuart Little 213 (-0.4)2 (-0.1) Sequence 5 – Seven13 (-0.1)1 (0) Sequence 6 – Monsters Inc12 (-0.1)3 (-0.6)

19. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 19 Overview of Transcoding

20. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 20 Transcoding  Conversion of video from one format to another −Bit rate conversion −Spatial resolution change −Temporal conversion −Format change  Architectures –Cascaded decoder & encoder –Spatial domain –Frequency domain –Hybrid domain

21. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 21 Proposed H.264 to VC-1 Transcoder

22. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 22 Choice of Profiles  The profiles chosen for H.264 is Baseline profile. The reasons for the choice of this profile are –Used in mobile applications and video conferencing –Less complex, no B pictures –Single reference frame, less memory requirements  The corresponding profile in VC-1 is simple profile which matches most of the features of the baseline profile in H.264

23. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 23 Choice of Transcoding Architecture  Differences –ME/MC block sizes H.264 supports 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4 macroblock partitions VC-1 supports 16x16 and 8x8 –Transform size and type H.264 supports 8x8 and 4x4 block sizes VC-1 supports 8x8, 8x4, 4x8 and 4x4 block sizes  Hence heterogeneous transcoding in pixel domain is chosen

24. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 24 Reference & Proposed Architecture H.264 Encoder H.264 Decoder VC-1 Encoder VC-1 Decoder Original YUV Reconstructed YUV H.264 bitstream VC-1 bitstream H.264 Encoder H.264 Decoder Reduced complexity VC-1 Encoder VC-1 Decoder Original YUV Reconstructed YUV Reusable information – MB mode, Motion vector H.264 bitstream VC-1 bitstream  Reference  Proposed

25. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 25 Methods used to reduce complexity  To achieve low complexity –Re-use of mode decisions –Re-use of macroblock partition sizes for macroblock partitioning (16x16 or 8x8) –Transform size selection (8x8, 8x4, 4x8, 4x4) from macroblock partitions –Re-use of motion vectors and thereby eliminating the motion estimation block  Value of eliminating the mode decision in VC-1 –Complex manipulations relating to RD cost of each mode is not done  Value of eliminating the motion estimation in VC-1 –Motion estimation is the most complex block in block based codecs –Up to 70% of the encoding time is taken up by motion estimation –This translates into lesser hardware costs, lesser cycles to process data and lesser power consumption in the device  Value of eliminating the transform size selection in VC-1 –Deciding the transform size involves complex calculations for each 8x8 block

26. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 26 Extracted information from incoming H.264 bitstream  H.264 bitstream contains information about  Macroblock type –P16x16 – P block type 16x16 –P16x8 – P block type 16x8 –P8x16 – P block type 8x16 –P8x8 – P block type 8x8 –I4MB – I block type 16x16 –I16MB – I block type 16x16  Macroblock sub block type –SMB8x8 – sub macroblock type 8x8 –SMB8x4 – sub macroblock type 8x4 –SMB4x8 – sub macroblock type 4x8 –SMB4x4 – sub macroblock type 4x4  Reference picture index  Motion vector x, y I frameP Frame

27. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 27 Mode decision and MV re-use Intra MB MappingInter MB Mapping  Skip as Skip  Same reference picture H.264 Intra MBVC-1 Intra MB Intra 16x16 (Any mode)Intra MB 8x8 Intra 4x4 (Any mode)Intra MB 8x8 H.264 Inter MBVC-1 Inter MBTransform size in VC-1 Inter 16x16 8x8 Inter 16x8Inter 8x88x4 Inter 8x16Inter 8x84x8 Inter 8x8 8x8 Inter 4x8Inter 8x84x8 Inter 8x4Inter 8x88x4 Inter 4x4Inter 8x84x4 H.264 Inter MBVC-1 Inter MBMotion Vector Re-use Inter 16x16 Same motion vectors for 16x16 block Inter 16x8Inter 8x8Median of motion vectors for each 8x8 block Inter 8x16Inter 8x8Median of motion vectors for each 8x8 block Inter 8x8 Same motion vectors for 8x8 block Inter 4x8Inter 8x8Median of motion vectors for each 8x8 block Inter 8x4Inter 8x8Median of motion vectors for each 8x8 block Inter 4x4Inter 8x8Median of motion vectors for each 8x8 block

28. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 28 Motion vector selection med - median

29. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 29 Motion vector selection  16x8 to 8x8 / 8x16 to 8x8 mode mapping –Since the euclidean distance is the same between 2 MV, the average is chosen  8x8 to 8x8 mode mapping –Choose MVi with minimum di

30. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 30 Proposed Algorithm N Y Update for VC1 Inter8x8 N End MB == 8x8 Start Firs t fra me Update for VC1 SKIP Update for VC1 Inter16x16 Update for VC1 Inter8x8 MB == 16x16 MB == 16x8 MB == 8x16 Y N NNN Y YYY Perform VC1 Intra MB == Skip

31. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 31 Implementation and Results

32. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 32 Implementation and testing  H.264 decoder - JM 16.1 –The decoder was modified to output the needed information (C implementation)  VC-1 Reference software from SMPTE –The encoder was modified to re-use the information (C implementation)  The simulation was carried out using 5 sequences –The sequences were chosen with a variety of motion in them –Resolutions commonly used in mobile devices were chosen SequenceSizeMotion AkiyoQCIF (176x144)Low Miss AmericaQCIF (176x144)Low ForemanCIF (352x288)Medium-High FootballCIF (352x288)Medium-High Mobile CalendarCIF (352x288)Medium

33. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 33 Results for Akiyo

34. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 34 Results for Akiyo (a) Original (b) H.264 decoded (c) Reference cascade decoded at QP 10 (d) Proposed transcoder decoded at QP 10

35. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 35 Results for Akiyo with respect to original video

36. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 36 Results for Foreman

37. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 37 Results for Foreman (a) Original (b) H.264 decoded (c) Reference cascade decoded at QP 10 (d) Proposed transcoder decoded at QP 10

38. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 38 Results for Foreman with respect to original video

39. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 39 Results for Football

40. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 40 Results for Football (a) Original (b) H.264 decoded (c) Reference cascade decoded at QP 10 (d) Proposed transcoder decoded at QP 10

41. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 41 Results for Football with respect to original video

42. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 42 Conclusions

43. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 43 Conclusions  Two dominant video codecs, H.264 and VC-1, were compared and contrasted.  A low complexity H.264 to VC-1 transcoder was developed –Efficient re-use of mode decisions, block partitions and motion vectors –Complete by pass of motion estimation in VC-1 re-encoding –Proposed transcoder achieves comparable quality to reference re- encoding transcoder –Reduced average encoding time by 80% compared to the re-encoding scheme. –Performance of VC-1 transcoder is comparable to H.264’s subjective quality at low QP values. –Performance of the transcoder falls short at high QP values.

44. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 44 Future work

45. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 45 Future work  The current proposal does not consider motion vector refinement. This can be researched in future for better performance.  The profiles considered for the input H.264/AVC bitstream is baseline profile. This can be extended to other profiles like main and high profiles. –Since main and high profiles of H.264 involve bi-directional prediction with multiple reference pictures and VC-1 allows only two reference pictures, algorithms to re-scale the motion vectors according to the appropriate reference pictures can be explored.

46. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 46 References

47. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 47 References 1. I. E.G. Richardson, “H.264 and MPEG-4 video compression: video coding for next-generation multimedia”, Wiley, 2003. 2. E. Lallana and M. Uy, “The Information Age,” UNDP-APDIP, 2003. 3. Open source article, “Digital Revolution,” Wikipedia Foundation, http://en.wikipedia.org/wiki/Digital_Revolution 4. J. Ribas-Corbera, “Windows Media 9 Series — a platform to deliver compressed audio and video for Internet and broadcast applications”, EBU Technical Review No. 293, Jan. 2003 5. K. Sayood, "Introduction to Data Compression,” 3rd Edition, Morgan Kaufmann Publisher Inc., 2006. 6. Open source article, “Rec 601,” Wikipedia Foundation, http://en.wikipedia.org/wiki/Rec._601http://en.wikipedia.org/wiki/Rec._601 7. R. Schafer and T. Sikora, "Digital video coding standards and their role in video communications," Proceedings of the IEEE, Vol. 83, pp. 907-923, Jan. 1995. 8. Information technology-generic coding of moving pictures and associated audio information: ISO/IEC 13818-2 (MPEG-2) Std. 9. Advanced Video Coding for Generic Audiovisual Services, ITU-T Rec. H.264 / ISO / IEC 14496-10, Nov. 2009. 10. VC-1 Compressed Video Bitstream Format and Decoding Process (SMPTE 421M-2006), SMPTE Standard, 2006 (http://store.smpte.org/category-s/1.htm).http://store.smpte.org/category-s/1.htm 11. A. Vetro, C. Christopoulos and H. Sun, “Video transcoding architectures and techniques: an overview,” IEEE Signal Processing Magazine, pp 18-29, March 2003. 12. I. Ahmad et al, “Video transcoding: An overview of various techniques and research issues”, IEEE Trans. on Multimedia, vol. 7, pp. 793-804, Oct. 2005 13. C. Chen, P-H.Wu and H. Chen, “MPEG-2 to H.264 transcoding,” Picture Coding Symposium, pp. 15-17, Dec. 2004. 14. Open source article, “H.264/MPEG-4 AVC,” Wikipedia Foundation, http://en.wikipedia.org/wiki/H.264/MPEG- 4_AVChttp://en.wikipedia.org/wiki/H.264/MPEG- 4_AVC 15. G.A Davidson et al, “ATSC video and audio coding”, Proc. IEEE, vol 94, pp. 60-76, Jan. 2006 (www.atsc.org). 16. G.F.-Escribano et al, “ An MPEG-2 to H.264 video transcoder in the baseline profile”, IEEE Trans. CSVT, vol. 20, pp. 763-768, May 2010

48. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 48 References 17. Jae-Beom Lee and H. Kalva, "An efficient algorithm for VC-1 to H.264 video transcoding in progressive compression," IEEE International Conference on Multimedia and Expo, pp. 53-56, July 2006. 18. Open Source Article, “VC-1 Technical Overview”, Microsoft Corporation, http://www.microsoft.com/windows/windowsmedia/howto/articles/vc1techoverview.aspx http://www.microsoft.com/windows/windowsmedia/howto/articles/vc1techoverview.aspx 19. DVD Forum, http://www.dvdforum.org/forum.shtmlhttp://www.dvdforum.org/forum.shtml 20. DV Magazine, http://www.dv.com/http://www.dv.com/ 21. Tandberg Television, http://www.tandberg.com/index2.jsphttp://www.tandberg.com/index2.jsp 22. CT Magazine, http://www.heise.de/ct/http://www.heise.de/ct/ 23. European Broadcasting Union, http://www.ebu.ch/http://www.ebu.ch/ 24. Blue-ray Disc Association - http://www.blu-raydisc.com/index.htmhttp://www.blu-raydisc.com/index.htm 25. Real Media, http://www.real.com/http://www.real.com/ 26. Envivo MPEG-4, http://www.envivio.com/http://www.envivio.com/ 27. Apple MPEG-4, http://www.apple.com/quicktime/technologies/mpeg4/http://www.apple.com/quicktime/technologies/mpeg4/ 28. M. Jeffrey, “The SMPTE 421M "VC-1" Standardization Project”, ITU-T Workshop Video and Image Coding and Applications, July 2005 29. R. Pereira, K.R. Rao and A. Kruafak, “Efficient transcoding of an MPEG bitstream to an H.264 bit stream”, University Scientific Journal series Telecommunications and Electronics, vol.11, pp. 5-31, Poland, 2008 30. S. Sharma and K.R. Rao, “Transcoding of H.264 bitstream to MPEG-2 bitstream”, IEEE APCC, pp. 391-396 Bangkok, Thailand, 18-20 Oct. 2007. 31. S. Moiron et al, “Video transcoding from H.264/AVC to MPEG-2 with reduced computational complexity”, Signal Processing: Image Communication, vol 24, pp. 637-650, Sept. 2009 32. S. Kwon, A. Tamhankar and K.R. Rao, ”Overview of H.264 / MPEG-4 Part 10”, J. Visual Communication and Image Representation, vol. 17, pp.183-216, April 2006. 33. T. Wiegand and G. J. Sullivan, “The H.264 video coding standard”, IEEE Signal Processing Magazine, vol. 24, pp. 148-153, March 2007. 34. A. Puri et al, “Video Coding using the H.264/ MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp: 793 – 849, Oct. 2004.

49. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 49 References 35. D. Marpe, T. Wiegand and S. Gordon, “H.264/MPEG4-AVC Fidelity Range Extensions: Tools, Profiles, Performance, and Application Areas”, Proc. IEEE International Conference on Image Processing 2005, vol. 1, pp. 593 - 596, Sept. 2005. 36. G. Sullivan, P. Topiwala and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions”, SPIE conference on Applications of Digital Image Processing XXVII, vol. 5558, pp. 53-74, Aug. 2004. 37. K. R. Rao and P. C. Yip, “The transform and data compression handbook”, Boca Raton,FL: CRC press, 2001. 38. T. Weigand et al, “Introduction to the Special Issue on Scalable Video Coding—Standardization and Beyond” IEEE Trans on Circuits and Systems for Video Technology, vol. 17, pp. 1034, Sept. 2007. 39. HHI presentation of the Scalable Extension of H.264/AVC, http://ip.hhi.de/imagecom_G1/savce/index.htmhttp://ip.hhi.de/imagecom_G1/savce/index.htm 40. S. Srinivasan and S. L. Regunathan, "An overview of VC-1," Visual Communications and Image Processing (VCIP), Proc. SPIE, vol. 5960, pp. 720-728, July 2005. 41. Open Source Article, “VC-1”, Wikipedia Foundation, http://en.wikipedia.org/wiki/VC-1http://en.wikipedia.org/wiki/VC-1 42. S. Srinivasan et al, "Windows Media Video 9: overview and applications", Signal Processing: Image Communication, vol. 19, Issue 9, pp. 851-875, Oct. 2004 43. Y. Huh, K. Panusopone, K.R. Rao, “Variable block size coding of images with hybrid quantization”, IEEE Trans. Circuits Systems Video Technol. 6, pp 679–685, Dec. 1996 44. J. Ribas-Corbera and D.L. Neuhoff, “Optimizing Block Size in Motion Compensation”, Journal of Electronic Imaging, vol. 7, pp.155-165, Jan. 1998 45. T.D.Tran, J.Liang and C. Tu, “Lapped Transform via Time-Domain Pre- and Post-Filtering”, IEEE Trans on Signal Processing, vol.51, no.6, pp. 1557-1571, Jun. 2003. 46. W.B.Pennebaker and J.L.Mitchell, JPEG Still Image Data Compression Standard, Van Nostrand Reinhold, 1993. 47. M. Wien, “Variable block size transforms for H.264/AVC”, IEEE Trans. Circuits Systems Video Technology, vol. 13, pp. 604–613, July 2003. 48. S. Gordon, “Adaptive Block Transform for Film Grain Reproduction in High Definition Sequences”, ”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, doc. JVT-H029, Geneva, Switzerland, 23-27 May, 2003 (available via anonymous ftp from ftp://ftp.imtc-files.org/jvt-experts/)ftp://ftp.imtc-files.org/jvt-experts/

50. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 50 References 49. Tektronics Picture Quality Analyzer PQA 300 (http://www.tek.com/site/ps/0,,25-11735-INTRO EN,00.html). 50. Microsoft, WMV-9—an Advanced Video Codec for 3GPP, 3GPP SA4 Meeting #18, document S4-030613. (available from http://www.3gpp.org/ftp/tsg sa/WG4 CODEC/TSGS4 28/Docs/)http://www.3gpp.org/ftp/tsg sa/WG4 CODEC/TSGS4 28/Docs/ 51. Nokia, Proposal to support MPEG-4 AVC / H.264/ AVC in Rel-6, 3GPP SA4 Meeting #27, document S4-030478. (available from http://www.3gpp.org/ftp/tsg sa/WG4 CODEC/TSGS4 27/Docs/)http://www.3gpp.org/ftp/tsg sa/WG4 CODEC/TSGS4 27/Docs/ 52. Z. Volta, Kompressionisten, C’T Mag. 10 (2003) 146-159, (in German, summary at http://www.heise.de/ct/03/10/146/). http://www.heise.de/ct/03/10/146/ 53. Windows Media Web site for Consumer Electronic devices, http://www.microsoft.com/windows/windowsmedia/conselec.asp. 54. M. Ravassi, M. Mattavelli and C. Clerc, “JVT/H.26L decoder complexity analysis”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, doc. JVT-D153, Klagenfurt, Austria, 22–26 July, 2002 (available via anonymous ftp from ftp://ftp.imtc-files.org/jvt-experts/).ftp://ftp.imtc-files.org/jvt-experts/ 55. H. Sun, X. Chen and T. Chiang, “Digital video transcoding for transmission and storage,” CRC Press, 2005. 56. J. Xin, C. Lin and M. Sun, “Digital Video Transcoding”, Proceedings of the IEEE, Vol 93, pp 84-96, Jan. 2005 57. B. Girod, “Overview: Video coding standards,” Stanford University coursework, http://www.stanford.edu/class/ee398b/handouts.htm http://www.stanford.edu/class/ee398b/handouts.htm 58. G. Sullivan, “Overview of international video coding standards (preceding H.264/AVC),”ITU-T VICA workshop, Geneva, July 2005. 59. M.-T. Sun, T.-D. Wu and J.-N. Hwang, “Dynamic bit allocation in video combining for multipoint conferencing,” IEEE Trans. Circuit Syst. II, vol. 45, no. 5, pp. 644-648, May 1998. 60. O. Werner, “Re-quantization for transcoding of MPEG-2 intra frames,” IEEE Trans. Image Processing, vol. 8, no. 2, pp. 179-191, Feb. 1999. 61. T. Shanabelah and M. Ghanbari, “Heterogeneous video transcoding to low spatial temporal resolutions and different encoding formats,” IEEE Trans. Multimedia, vol. 2, no. 2, pp. 101-110, Jun. 2000. 62. K.-H. Tan and M. Ganbari, “Layered image coding using the DCT pyramid,“ IEEE Trans. Image Processing, vol. 4, no. 4, pp. 512-516, Apr. 1995.

51. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 51 References 63. J.-N.Hwang and T.-D. Wu, “Motion vector re-estimation and dynamic frame-skipping for video transcoding,” Conf Rec. 32nd Asilomar Conf. Signals, Systems and Computer vol 2, pp 1606-1610, 1998. 64. J. Youn, M.-T. Sun and C.-W. Lin, “Motion vector refinement for high-performance transcoding,” IEEE Trans. Multimedia, vol. 1, no. 1, pp. 30-40, Mar 1999. 65. M.-J. Chen, M.-C. Chu and C.-W. Pan, “Efficient motion estimation algorithm for reduced frame-rate video transcoder,” IEEE Trans. Circuits Syst. Video Technol., vol. 12, pp. 269-275, Apr 2002. 66. N. Bjork and C. Christopoulos, “Transcoder architecture for video coding,” IEEE Trans. Consumer Electronics., vol. 44, pp. 88–98, Feb. 1998. 67. R. Mohan, J. R. Smith, and C. Li, “Adapting multimedia internet content for universal access,” IEEE Trans. Multimedia, vol. 1, no. 1, pp. 104–114, Mar. 1999 68. J.-N. Hwang and T.-D. Wu, “Motion vector re-estimation and dynamic frame-skipping for video transcoding,” in Conf. Rec. 32nd Asilomar Conf. Signals, System & Computer, vol. 2, 1998, pp. 1606–1610. 69. I. E. Richardson, “The H.264 Advanced Video Compression Standard”, Second Edition, Wiley, May 2010. 70. JM reference software http://iphome.hhi.de/suehring/tml/http://iphome.hhi.de/suehring/tml/ 71. VC-1 SMPTE software http://store.smpte.org/category-s/30.htmhttp://store.smpte.org/category-s/30.htm 72. A. Luthra, G. Sullivan and T. Wiegand, “Introduction to the special issue on the H.264/AVC video coding standard”, IEEE Trans. on Circuits and Systems for Video Technology, vol. 13, issue 7, pp. 557-559, July 2003. 73. J. Padia, “Complexity reduction for VP6 to H.264 transcoder using motion vector re-use”, M.S. Thesis, EE Dept, UT Arlington, May 2010. 74. Open source article, “MPEG-2,” Wikipedia Foundation, http://en.wikipedia.org/wiki/MPEG-2http://en.wikipedia.org/wiki/MPEG-2 75. Open source article, “MPEG-1,” Wikipedia Foundation, http://en.wikipedia.org/wiki/MPEG-1http://en.wikipedia.org/wiki/MPEG-1 76. Open source article, “H.261,” Wikipedia Foundation, http://en.wikipedia.org/wiki/H.261 77. Open source article, “H.263,” Wikipedia Foundation, http://en.wikipedia.org/wiki/H.263 78. Open source article, “MPEG-4 - Part 2” Wikipedia Foundation, http://en.wikipedia.org/wiki/MPEG-4_Part_2http://en.wikipedia.org/wiki/MPEG-4_Part_2 79. H. Kalva and J.B. Lee, “The VC-1 and H.264 Video Compression Standards for Broadband Video Services”, Springer, 2008. 80. H. Kalva and J.B. Lee, “The VC-1 Video Coding Standard,” IEEE Multimedia, Vol. 14, No 4, pp. 88-91, Oct.-Dec. 2007.

52. Thank you

53. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 53 H.264 - Profiles

54. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 54 Design Features Highlights  Features for enhancement of prediction –Directional spatial prediction for intra coding –Variable block-size motion compensation with small block size –Quarter-sample-accurate motion compensation –Motion vectors over picture boundaries –Multiple reference picture motion compensation –Decoupling of referencing order from display order –Decoupling of picture representation methods from picture referencing capability –Weighted prediction –Improved “skipped” and “direct” motion inference –In-the-loop deblocking filtering

55. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 55 Design Features Highlights  Features for improved coding efficiency –Small block-size transform –Exact-match inverse transform –Short word-length transform –Hierarchical block transform –Arithmetic entropy coding –Context-adaptive entropy coding

56. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 56 Design Features Highlights  Features for robustness to data errors/losses –Parameter set structure –NAL unit syntax structure –Flexible slice size –Flexible macroblock ordering (FMO) –Arbitrary slice ordering (ASO) –Redundant pictures –Data Partitioning –SP/SI synchronization/switching pictures

57. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 57 Directional spatial prediction for intra coding  Intra prediction is to predict the texture in current block using the pixel samples from neighboring blocks  Intra prediction for 4  4 (9 modes) and 16  16 blocks (4 modes) are supported in all H.264 profiles.  Intra prediction for 8x8 (9 modes) is supported in the high profiles. Intra 4x4 Intra 8x8 Intra 16x16

58. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 58 Variable block-size motion compensation  Partitioned in 2 stages  In the 1 st stage, determine first 4 modes –16  16 –16  8 –8  16 –8  8  If mode 4 (8  8) is chosen, further partition into smaller blocks for every 8  8 block –8  4 –4  8 –4  4  At most 16 motion vectors may be transmitted for a 16  16 macroblock  Sub pixel accuracy  Large computational complexity to determine the modes but efficient encoding

59. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 59 Multiple reference picture motion compensation  P Slice –More than one prior coded picture can be used as reference for MC prediction –Reference index parameter is transmitted for each MC 16  16, 16  8, 8  16 or 8  8 –For smaller blocks within the 8  8 use 1 reference index –P-Skip type is supported  B Slice –Utilize two distinct lists of reference pictures –Four different types of inter-picture predict List 0, list 1, bi-predictive, and direct –Bi-predictive weighted average of MC list 0 and list 1 –Direct prediction Inferred from previously transmitted syntax Either list 0 or list 1 prediction or bi- predictive –Similar macroblock partitioning as P slices is utilized –B Skip mode is supported P frame B frame

60. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 60 Hierarchical block transform  4x4 and 8x8 (high profile only) multiplier-free integer DCT transform  Transform coefficients perfectly invertible  Hierarchical transform (Integer DCT and Hadamard)  For macroblock coded in 16  16 Intra mode and chrominance blocks –DC coefficients are further grouped and transformed  Hadamard transform is used for chrominance block Integer DCT 4x4Integer DCT 8x8Hadamard 4x4 Hadamard 2x2

61. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 61 In loop deblocking filter  Block based operations are responsible for blocking artifacts  In-loop deblock filter –smoothes blocky edges; increases rate- distortion performance.  Applied to all 4x4 blocks except at picture boundaries.  Filtering adaptive at  Slice level  Block level  Pixel level  Vertical edges filtered first (left to right)  Followed by horizontal edges (top to bottom)

62. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 62 Entropy encoding  CAVLC (Context-based Adaptive Variable Length Coding).  CABAC (Context-based Adaptive Binary Arithmetic Coding).  CAVLC makes use of run-length encoding.  CABAC utilizes arithmetic coding; codes both MV and residual transform coefficients.  Typically CABAC provides 10-15 % reduction in bit rate compared to CAVLC, for the same PSNR.  All other syntax elements are encoded by Exp-Golomb codes (Universal Variable Length Codes (UVLC)). CAVLC CABAC

63. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 63 Computational Overhead  Entropy encoding  Multiple block size  Smaller block size  Integer transform  In-loop deblocking

64. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 64 H.264 Extensions Scalable video coding  Application scenario

65. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 65 H.264 Extensions Scalable video coding

66. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 66 Types of Scalability

67. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 67 H.264 Extensions Multi view coding  Applications 3-D Video Stereoscopic TV

68. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 68 H.264 Extensions Multi view coding

69. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 69 VC1 Decoder : Simple and Main Profile

70. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 70 VC-1 Decoder : Advanced Profile

71. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 71 VC1 Profiles ProfileLevelLabel Simple LowSP@LL MediumSP@ML Main LowMP@LL MediumMP@ML HighMP@HL Advanced L0AP@L0 L1AP@L1 L2AP@L2 L3AP@L3 L4AP@L4

72. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 72 Design Feature Highlights  Adaptive variable transform size (16 bit transforms)  Multiple scanning patterns  Quantization with dead zone  Bit plane coding  DC/AC intra prediction  Simple motion estimation  Huffman coding  Intensity compensation  Range reduction  Overlap transform  In loop filtering

73. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 73 VC1 - Intra prediction  No spatial prediction like H.264/AVC  Mandatory DC prediction in the transform domain  Optional AC prediction in the transform domain  Independent luma and chroma intra prediction  Always uses 8x8 transform size  Inter MBs can have up to three 8x8 intra blocks

74. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 74 VC-1 Inter prediction  Two block sizes –16x16, 8x8  Half and quarter pixel accuracy –Bi-linear and bi-cubic interpolation filters  Four ME methods –Mixed block size (16x16 and 8x8), ¼ pixel, bicubic –16x16, ¼ pixel, bicubic –16x16, ½ pixel, bicubic –16x16, ½ pixel, bilinear  Combined block size, MV resolution and filter representation

75. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 75 VC-1 Transform  A variation of the discrete cosine transform  Transform sizes of 8x8, 8x4, 4x8,4x4  Bit accurate transform  Fast algorithm for inverse operation

76. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 76 Intensity Compensation  Reference frame (luma and chroma) data are scaled before using it for motion estimation  Useful in fade in and fade out scenes  Lesser residual energy by using intensity compensation  Defined only for P pictures (not for B pictures)

77. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 77 Overlap Transform  Part of de-blocking process  Removes blocking effect in high-low quality discontinuity  Switches over the edge data of two adjacent blocks  High quality and low quality blocks diffuse each other  Instead of simple switch, filtering operation is performed  Only on intra 8x8 blocks  Not on inter, intra boundaries (inter contains residual)

78. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 78 In loop filtering  I, B – All 8x8 blocks  P – Depending on transform size used  Horizontal edge followed by vertical edge  Only 2 pixels are filtered  Filtering decision per boundary

79. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 79 Innovations in VC-1  Adaptive block size transform –Four different sizes 8x8, 8x4, 4x8,4x4 –Smaller transforms are better in areas with discontinuities –Fewer ringing artifacts  16-bit implementation of the transform –Ease of implementation on chips  Multiple precision modes for motion compensation –16x16 and 8x8, ¼ pixel accuracy, bi-linear/bi-cubic filters  Uniform and non-uniform quantization –Dead zone and uniform quantization  Loop-filtering –Less complex – lesser pixels filtered, lesser filtering decisions  Overlap smoothing –Reduces artifacts in intra coded blocks  Fading compensation –Better prediction in fade in, fade out scenes

80. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 80 H.264 Vs VC-1  H.264 –H.264 is significantly computationally involved CABAC, loop filtering, smaller ME block sizes –More memory requirement Multiple reference picture buffers, max of 16 –Spatial intra prediction 9 (intra 4x4) + 4 (intra 16x16) = 13 modes  VC-1 –VC-1 is comparatively simpler Simpler entropy coding, simpler loop filtering, lesser ME block sizes –Lesser memory requirement Only 2 reference pictures –No spatial intra prediction Only DC/AC prediction in the transform domain

81. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 81 Need for Transcoding Multimedia applications on different devices and platforms Different bitrates, frame rates, spatial resolution & complexity Different video standards; communication & inter-operability ApplicationBitrateVideo standard Digital TV broadcasting 2 to 6 Mbps (10 to 20 Mbps for HD broadcast) MPEG-2, H.264 DVD Video6 to 8 MbpsMPEG-2 Internet video streaming 20 to 200 kbpsFlash – Sorrension spark (based on H.263), VP6 and H.264; Silverlight uses VC- 1; and also MPEG-4 Part 2 Video conferencing and video-telephony 20 to 320 kbpsH.261, H.263, H.263+ Video over 3G wireless 20 to 100 kbpsH.263, MPEG-4. Part 2 High definition – Blu- ray and HD-DVD 36 to 54 MbpsH.264, VC-1 and MPEG-2

82. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 82 Cascaded encoder and decoder  Simplest but most in-efficient as it involves re-encoding  No degradation in visual quality  Full scale motion re-estimation is needed.

83. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 83 Spatial Domain Transcoding

84. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 84 Frequency Domain Transcoding

85. July 14, 2010 Low Complexity H.264 to VC-1 Transcoder 85 Pseudo code #ifdef H264VC1TRANSCODER if((mbType == I4MB) || (mbType == I16MB)) { // update block types for each 8x8 block } else if((mbType == P16x16) || (mbType == P16x8) || (mbType == P8x16) || (mbType == PSKIP)) { if(mbType == P16x16) // update all 4 8x8 block types as 1 MV MB // update the motion vectors from the input file if(mbType == P16x8) // update all 4 8x8 block types as 4 MV MB // compute the median MV from input file // update the motion vectors as the median MV if(mbType == P8x16) // update all 4 8x8 block types as 4 MV MB // compute the median MV from input file // update the motion vectors as the median MV if(mbType == P8x8) // update all 4 8x8 block types as 4 MV MB // compute the median MV from input file depending on the sub macroblock type // update the motion vectors as the median MV if(mbType == PSKIP) // update all 4 8x8 block types as skip // update MV as the predicted MV } #endif