1. 2015/6/25MC 20091 多媒體編碼技術導論 Introduction to Multimedia Coding Text Book Iain E.G. Richardson, “H.264 and MPEG-4 Video Compression,” John Wiley & Sons, 2003. References 1. Yao Wang, Jorn Ostermann, and Ya-Qin Zhang, “Video Processing and Communications,” Prentice Hall Signal Processing Series, 2002. 2. Khalid Sayood, “Data Compression,” Morgan Kaufmann, 2 nd edition, 2000. 3. M.S. Drew and Z.-N. Li, "Fundamentals of multimedia," Prentice-Hall, 2003.

2. 2015/6/25MC 20092 Content Introduction Video Formats and Quality Video Coding Concept H.261 Variable length coding, DCT, Motion estimation The MPEG-4 and H.264 Standards H.264/AVC, Main/High/SVC/Multi-View Wavelets: Image and Video Coding Image Coding Standard: JPEG-2000, HD Photo Voice/Audio Coding Standards: iLBC (Skype) MP3, and AAC Applications and Directions

3. 2015/6/25MC 20093 Bernd Girod, Stanford University http://www.stanford.edu/class/ee398/ Entropy and lossless coding techniques Run-length coding, fax standards Arithmetic coding Rate-distortion limits and quantization Lossless and lossy predictive coding Transform coding, JPEG standard Subband coding, wavelets, JPEG-2000 Motion compensated coding, MPEG standards

4. 2015/6/25MC 20094 Learn and Purpose Provide students with a global overview of state-of-the-art in the area of VIDEO and VOICE COMPRESSION Video: current standards, H.264 and MPEG-4 Voice/Audio: current standards, iLBC (for VoIP), MP3, G.729 Purpose: inspire the students to develop value-added supplementary on H.264 or iLBC

5. 2015/6/25MC 20095 Schedule on Experiments 3 th -4 th weeks: Trace H.261 code 5 th -12 th weeks: Trace H.264 code …

6. 2015/6/25MC 20096 評分標準 Midterm Exam.: 35% Homework: 15% H.264 program trace, Report: 20% Presentation and Discussion:30%

7. 2015/6/25MC 20097 Applications on H.264/AVC standards Broadcast over cable, satellite, cable modem, DSL, terrestrial, DVB-H, etc. Interactive or serial storage on optical and magnetic devices: Blu-ray DVD, HD-DVD, etc. Conversational services over ISDN, Ethernet, LAN, DSL, wireless and mobile networks, modems, etc. or mixtures of these. Video-on-demand or multimedia streaming services over ISDN, cable modem, DSL, LAN, wireless networks, etc. Multimedia messaging services (MMS) over ISDN, DSL, Ethernet, LAN, wireless and mobile networks, 3G, etc.

8. 2015/6/25MC 20098 Chronological Table of Video Coding Standards H.261 (1990) MPEG-1 (1991) H.263 (1995/96) H.263+ (1997/98) H.263++ (2000) H.264 ( MPEG-4 Part 10 ) (2002) MPEG-4 v1 (1998/99) MPEG-4 v2 (1999/00) MPEG-4 v3 (2001) 1990 1992 1994 1996 1998 2000 2002 2003 MPEG-2 (H.262) (1993/94) ISO/IEC MPEG ITU-T VCEG

9. 2015/6/25MC 20099 Position of H.264

10. 2015/6/25MC 200910 H.264/AVC Profiles (1)

11. 2015/6/25MC 200911 H.264/AVC Profiles (2)

12. 2015/6/25MC 200912 H.264/AVC Profiles (3) - Scalable Video Coding (SVC) Functionality of SVC Graceful degradation when “ right ” parts of the bit-stream are lost Bit-rate adaptation to match the channel throughput Format adaptation for backwards compatible extension Power adaptation for trade-off between runtime and quality 2007/8Kai-Chao Yang, NTHU, Taiwan12

13. 2015/6/25MC 200913 Then … MPEG 3D Audio-visual encoding – built on AVC ? Workshops on future directions in video compression Multi-view Video Coding ITU-T VCEG Reduced complexity H.264 ? H.265 with 50% better compression by 2010

14. 2015/6/25MC 200914 Terminology, Chapter 1 Sampling: spatial and temporal Frame and field; progressive and raster scan Color space: RGB, YIQ, YCbCr, CMY, … Video format Analog: NTSC, PAL, SECAM Digital: CCIR601, Sub-QCIF, QCIF, CIF, BT.601(720  480/576), SMPTE295M(1920  1080) 4:2:0, 4:2:2, 4:4:4

15. 2015/6/25MC 200915 Terminology, Chapter 1

16. 2015/6/25MC 200916 Terminology, Chapter 1

17. 2015/6/25MC 200917 Terminology, Chapter 1 Digital Video:BT.601

18. 2015/6/25MC 200918 Terminology, Chapter 1

19. 2015/6/25MC 200919 Terminology, Chapter 1

20. 2015/6/25MC 200920 Subjective Quality Measurement Visual scene: complex interaction between the Human Visual System, the eye, and the brain Double Stimulus Continuous Quality Scale ITU-R BT.500, Double Stimulus Continuous Quality Scale (DSCQS) Mean Opinion Score (MOS) The assessor is presented with a pair of images A and B, one after the other, and is asked to give A and B a ‘quality score’ with five intervals ranging from ‘Excellent’ to ‘Bad’. The selection of which sequence is A (source) and which is B (impaired) is randomized.

21. 2015/6/25MC 200921 Objective Quality Measurement Peak Signal to Noise Ratio (PSNR) Other objective quality metrics, ITU-T Video Quality Experts Group (VQEG) Currently, no objective measurement system is able to replace subjective testing, no one objective model outperforms the others in all cases.

22. 2015/6/25MC 200922 Motion Vector Frame i-1Frame i Current frame (to be encoded) Reference frame (existed) Occlusion matched Motion vector Block matching

23. 2015/6/25MC 200923 Motion Compensation & Residuals motion Residuals occlusion

24. 2015/6/25MC 200924 Encoder Motion Compensation DCT + Q iDCT + iQ MV = ( d x, d y )  ‚ ƒ „ Residuals Previous Frame Buffer Encoder (DCT  Quantization  Binary coding) Residual only

25. 2015/6/25MC 200925 Decoder Coded Bitstream VLD IDCT Motion Compensation Previous Frame memory Reconstructed frame MV Residuals

26. 2015/6/25MC 200926 DCT (1) 8 8 139148150149155164165168 98115130135143146142147 89110125128129121104106 96116128132134132113109 111125127131137 120110 122126 131133131126112 133134136138140144141139 138139 140146148147

27. 2015/6/25MC 200927 DCT IDCT Pixel values in spatial domain DCT coefficients in transform domain DC term: The coefficient of the basis vector (0,0) DCT (2)

28. 2015/6/25MC 200928 Does DCT Transform Really Make Sense ? Energy compaction De-correlation: dependency elimination

29. 2015/6/25MC 200929

30. 2015/6/25MC 200930

31. 2015/6/25MC 200931 Natural Audio Coder Quality Cellular Telephone AM FM CD 248163264 kbit/s Parametric speech (HVXC) High quality speech (CELP) General audio (AAC, TwinVQ) Parametric audio (HILN) From Olivier Dechazal