2015/6/25MC 多媒體編碼技術導論 Introduction to Multimedia Coding Text Book Iain E.G. Richardson, “H.264 and MPEG-4 Video Compression,” John Wiley & Sons, References 1. Yao Wang, Jorn Ostermann, and Ya-Qin Zhang, “Video Processing and Communications,” Prentice Hall Signal Processing Series, Khalid Sayood, “Data Compression,” Morgan Kaufmann, 2 nd edition, M.S. Drew and Z.-N. Li, "Fundamentals of multimedia," Prentice-Hall, 2003.
2015/6/25MC Content Introduction Video Formats and Quality Video Coding Concept H.261 Variable length coding, DCT, Motion estimation The MPEG-4 and H.264 Standards H.264/AVC, Main/High/SVC/Multi-View Wavelets: Image and Video Coding Image Coding Standard: JPEG-2000, HD Photo Voice/Audio Coding Standards: iLBC (Skype) MP3, and AAC Applications and Directions
2015/6/25MC Bernd Girod, Stanford University Entropy and lossless coding techniques Run-length coding, fax standards Arithmetic coding Rate-distortion limits and quantization Lossless and lossy predictive coding Transform coding, JPEG standard Subband coding, wavelets, JPEG-2000 Motion compensated coding, MPEG standards
2015/6/25MC Learn and Purpose Provide students with a global overview of state-of-the-art in the area of VIDEO and VOICE COMPRESSION Video: current standards, H.264 and MPEG-4 Voice/Audio: current standards, iLBC (for VoIP), MP3, G.729 Purpose: inspire the students to develop value-added supplementary on H.264 or iLBC
2015/6/25MC Schedule on Experiments 3 th -4 th weeks: Trace H.261 code 5 th -12 th weeks: Trace H.264 code …
2015/6/25MC 評分標準 Midterm Exam.: 35% Homework: 15% H.264 program trace, Report: 20% Presentation and Discussion:30%
2015/6/25MC Applications on H.264/AVC standards Broadcast over cable, satellite, cable modem, DSL, terrestrial, DVB-H, etc. Interactive or serial storage on optical and magnetic devices: Blu-ray DVD, HD-DVD, etc. Conversational services over ISDN, Ethernet, LAN, DSL, wireless and mobile networks, modems, etc. or mixtures of these. Video-on-demand or multimedia streaming services over ISDN, cable modem, DSL, LAN, wireless networks, etc. Multimedia messaging services (MMS) over ISDN, DSL, Ethernet, LAN, wireless and mobile networks, 3G, etc.
2015/6/25MC Chronological Table of Video Coding Standards H.261 (1990) MPEG-1 (1991) H.263 (1995/96) H.263+ (1997/98) H (2000) H.264 ( MPEG-4 Part 10 ) (2002) MPEG-4 v1 (1998/99) MPEG-4 v2 (1999/00) MPEG-4 v3 (2001) MPEG-2 (H.262) (1993/94) ISO/IEC MPEG ITU-T VCEG
2015/6/25MC Position of H.264
2015/6/25MC H.264/AVC Profiles (1)
2015/6/25MC H.264/AVC Profiles (2)
2015/6/25MC H.264/AVC Profiles (3) - Scalable Video Coding (SVC) Functionality of SVC Graceful degradation when “ right ” parts of the bit-stream are lost Bit-rate adaptation to match the channel throughput Format adaptation for backwards compatible extension Power adaptation for trade-off between runtime and quality 2007/8Kai-Chao Yang, NTHU, Taiwan12
2015/6/25MC Then … MPEG 3D Audio-visual encoding – built on AVC ? Workshops on future directions in video compression Multi-view Video Coding ITU-T VCEG Reduced complexity H.264 ? H.265 with 50% better compression by 2010
2015/6/25MC Terminology, Chapter 1 Sampling: spatial and temporal Frame and field; progressive and raster scan Color space: RGB, YIQ, YCbCr, CMY, … Video format Analog: NTSC, PAL, SECAM Digital: CCIR601, Sub-QCIF, QCIF, CIF, BT.601(720 480/576), SMPTE295M(1920 1080) 4:2:0, 4:2:2, 4:4:4
2015/6/25MC Terminology, Chapter 1
2015/6/25MC Terminology, Chapter 1
2015/6/25MC Terminology, Chapter 1 Digital Video:BT.601
2015/6/25MC Terminology, Chapter 1
2015/6/25MC Terminology, Chapter 1
2015/6/25MC Subjective Quality Measurement Visual scene: complex interaction between the Human Visual System, the eye, and the brain Double Stimulus Continuous Quality Scale ITU-R BT.500, Double Stimulus Continuous Quality Scale (DSCQS) Mean Opinion Score (MOS) The assessor is presented with a pair of images A and B, one after the other, and is asked to give A and B a ‘quality score’ with five intervals ranging from ‘Excellent’ to ‘Bad’. The selection of which sequence is A (source) and which is B (impaired) is randomized.
2015/6/25MC Objective Quality Measurement Peak Signal to Noise Ratio (PSNR) Other objective quality metrics, ITU-T Video Quality Experts Group (VQEG) Currently, no objective measurement system is able to replace subjective testing, no one objective model outperforms the others in all cases.
2015/6/25MC Motion Vector Frame i-1Frame i Current frame (to be encoded) Reference frame (existed) Occlusion matched Motion vector Block matching
2015/6/25MC Motion Compensation & Residuals motion Residuals occlusion
2015/6/25MC Encoder Motion Compensation DCT + Q iDCT + iQ MV = ( d x, d y ) Residuals Previous Frame Buffer Encoder (DCT Quantization Binary coding) Residual only
2015/6/25MC Decoder Coded Bitstream VLD IDCT Motion Compensation Previous Frame memory Reconstructed frame MV Residuals
2015/6/25MC DCT (1)
2015/6/25MC DCT IDCT Pixel values in spatial domain DCT coefficients in transform domain DC term: The coefficient of the basis vector (0,0) DCT (2)
2015/6/25MC Does DCT Transform Really Make Sense ? Energy compaction De-correlation: dependency elimination
2015/6/25MC
2015/6/25MC
2015/6/25MC Natural Audio Coder Quality Cellular Telephone AM FM CD kbit/s Parametric speech (HVXC) High quality speech (CELP) General audio (AAC, TwinVQ) Parametric audio (HILN) From Olivier Dechazal