1. 1 Video Coding Concept Kai-Chao Yang

2. 2 Video Sequence and Picture Video sequence Large amount of temporal redundancy Intra Picture/VOP/Slice (I-Picture) (p.295) Encoded without referencing others All MBs are intra coded Inter Picture/VOP/Slice (P-Picture, B-Picture) (p.295) Encoded by referencing other pictures Some MBs are intra coded, and some are inter coded Intra 0Inter 1Inter 2Inter 3Inter 4Inter 5

3. 3 Group of Pictures Group of Pictures (GOP) (p.295, p.455-457) GOP in H.264/AVC Multiple reference frames B-pictures can be referenced Variable number of B-pictures GOP in Scalable Video Coding (SVC) of H.264/AVC Hierarchical prediction structure IBBPBBP … BBIBBP … Encoding order: Frame order: 012345 023156 6 4 Video stream GOP …

4. Picture, Slice, MacroBlock, and Block H.264/AVC Variable MB size 4x4 transform block 4 Picture Slice MB block 8 8 8 8

5. 5 Flow Chart of Video Coding (p.293-295) TransformQuantization Motion Estimation Entropy Coding Quantization -1 Inverse Transform Motion Compensation Decoder MB (spatial  frequency) (lossy compression)(lossless compression) (frame reconstruction) (estimating similar blocks) (removing temporal redundancy) - + High frequency Low frequency Reference frameCurrent frame

6. 6 Coding of I-Slice Example Original blockTransformed block Quantization matrix 15 0 -2 -1 -1 -1 0 … Bit-stream Zig-zag scanEntropy coding DCT

7. 7 Coding of P-Slice Example = = + Motion Vectors Motion Estimation Residual Motion Compensation Original current frame Reconstructed reference frame Frame buffer -

8. 8 Structure of H.264/AVC Encoder Representation of video content Formats of VCL representation

9. 9 H.264/AVC Flow Chart Entropy Coding Scaling & Inv. Transform Motion- Compensation Control Data Quant. Transf. coeffs Motion Data Intra/Inter Coder Control Decoder Motion Estimation Transform/ Scal./Quant. - Input Video Signal Split into Macroblocks 16x16 pixels Intra-frame Prediction De-blocking Filter Output Video Signal

10. 10 New Features of H.264/AVC Multiple reference frames Variable block size 1/4 and 1/8 motion vector precision Weighted prediction 4x4 transform block Intra prediction De-blocking filter CABAC & CAVLC Various block sizes and shapes 8x8 0 4x8 01 01 23 4x4 8x4 1 0 8x8 Types 0 16x16 01 8x16 MB Types 8x8 01 23 16x8 1 0

11. 11 FMO and ASO Examples of Flexible MB Order (FMO) Arbitrary Slice Order (ASO) sending and receiving the slices of the picture in any order relative to each other

12. 12 H.264/AVC Profiles

13. 13 H.264/AVC FRExt Amendment Completed in July 2004 For coding of high-fidelity video material Professional film production, video post production, or high- definition TV/DVD Higher quality, higher rates Professional extensions  Fidelity Range extensions (FRExt) http://plusd.itmedia.co.jp/lifestyle/articles/0407/09/news074.html

14. 14 Profiles of H.264/AVC FRExt High profiles High profile (HP) Supporting 8-bit with 4:2:0 sampling High 10 profile (Hi10P) Supporting 10-bit with 4:2:0 sampling High 4:2:2 profile (H422P) Supporting 10-bit with 4:2:2 sampling High 4:4:4 profile (H444P) Supporting 12-bit with 4:4:4 sampling, and efficient lossless coding and an integer residual color transform for coding RGB video

15. References T. Wiegand, G.J. Sullivan, G. Bjntegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard,” in IEEE Transaction on Circuits and Systems for Video Technology, 2003. Iain E. G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia, Wiley, 2003. G. J. Sullivan, P. Topiwala, and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions,” in SPIE Conference on Applications of Digital Image Processing, 2004. 15