1 Video Coding Concept Kai-Chao Yang

2 Video Sequence and Picture Video sequence Large amount of temporal redundancy Intra Picture/VOP/Slice (I-Picture) (p.295) Encoded without referencing others All MBs are intra coded Inter Picture/VOP/Slice (P-Picture, B-Picture) (p.295) Encoded by referencing other pictures Some MBs are intra coded, and some are inter coded Intra 0Inter 1Inter 2Inter 3Inter 4Inter 5

3 Group of Pictures Group of Pictures (GOP) (p.295, p ) GOP in H.264/AVC Multiple reference frames B-pictures can be referenced Variable number of B-pictures GOP in Scalable Video Coding (SVC) of H.264/AVC Hierarchical prediction structure IBBPBBP … BBIBBP … Encoding order: Frame order: Video stream GOP …

Picture, Slice, MacroBlock, and Block H.264/AVC Variable MB size 4x4 transform block 4 Picture Slice MB block

5 Flow Chart of Video Coding (p ) TransformQuantization Motion Estimation Entropy Coding Quantization -1 Inverse Transform Motion Compensation Decoder MB (spatial  frequency) (lossy compression)(lossless compression) (frame reconstruction) (estimating similar blocks) (removing temporal redundancy) - + High frequency Low frequency Reference frameCurrent frame

6 Coding of I-Slice Example Original blockTransformed block Quantization matrix … Bit-stream Zig-zag scanEntropy coding DCT

7 Coding of P-Slice Example = = + Motion Vectors Motion Estimation Residual Motion Compensation Original current frame Reconstructed reference frame Frame buffer -

8 Structure of H.264/AVC Encoder Representation of video content Formats of VCL representation

9 H.264/AVC Flow Chart Entropy Coding Scaling & Inv. Transform Motion- Compensation Control Data Quant. Transf. coeffs Motion Data Intra/Inter Coder Control Decoder Motion Estimation Transform/ Scal./Quant. - Input Video Signal Split into Macroblocks 16x16 pixels Intra-frame Prediction De-blocking Filter Output Video Signal

10 New Features of H.264/AVC Multiple reference frames Variable block size 1/4 and 1/8 motion vector precision Weighted prediction 4x4 transform block Intra prediction De-blocking filter CABAC & CAVLC Various block sizes and shapes 8x8 0 4x x4 8x x8 Types 0 16x x16 MB Types 8x x8 1 0

11 FMO and ASO Examples of Flexible MB Order (FMO) Arbitrary Slice Order (ASO) sending and receiving the slices of the picture in any order relative to each other

12 H.264/AVC Profiles

13 H.264/AVC FRExt Amendment Completed in July 2004 For coding of high-fidelity video material Professional film production, video post production, or high- definition TV/DVD Higher quality, higher rates Professional extensions  Fidelity Range extensions (FRExt)

14 Profiles of H.264/AVC FRExt High profiles High profile (HP) Supporting 8-bit with 4:2:0 sampling High 10 profile (Hi10P) Supporting 10-bit with 4:2:0 sampling High 4:2:2 profile (H422P) Supporting 10-bit with 4:2:2 sampling High 4:4:4 profile (H444P) Supporting 12-bit with 4:4:4 sampling, and efficient lossless coding and an integer residual color transform for coding RGB video

References T. Wiegand, G.J. Sullivan, G. Bjntegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard,” in IEEE Transaction on Circuits and Systems for Video Technology, Iain E. G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia, Wiley, G. J. Sullivan, P. Topiwala, and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions,” in SPIE Conference on Applications of Digital Image Processing,