1

1. Problem Statement 2. Overview of H.264/AVC Scalable Extension I. Temporal Scalability II. Spatial Scalability III. Complexity Reduction 3. Previous Parallel Encoding Scheme for Video Coding 1. MB-Level (Wave-front) Parallelism 2. Frame-Level Parallelism 4. Parallel Encoding Based on Hierarchical B-Picture Structure I. Frame-Level Parallel Scheme 5. Conclusions and Future Work 2

 Combined scalability.  H.264 based, layered video coding. 3

 Base Layer (BL) is identical to the standard H.264  Enhancement Layers (EL) have “inter-layer” predictions in additional: H.264: –Inter 16x16 –Inter 8x16 –Inter 16x8 –Inter 8x8 Inter 8x8 Inter 4x8 Inter 8x4 Inter 4x4 –Intra 16x16 (4 modes) –Intra 4x4 (9 modes) SVC additional: –BL Inter 16x16 –BL Inter 8x16 –BL Inter 16x8 –BL Inter 8x8 BL Inter 8x8 BL Inter 4x8 BL Inter 8x4 BL Inter 4x4 –BL Intra 16x16 –BL Intra 4x4 –BL Inter 16x16 w. residue pred. –BL Inter 8x16 w. residue pred. –BL Inter 16x8 w. residue pred. –BL Inter 8x8 w. residue pred.. BL Inter 8x8 w. residue pred. BL Inter 4x8 w. residue pred. BL Inter 8x4 w. residue pred. BL Inter 4x4 w. residue pred. –BL Intra 16x16 w. residue pred. –BL Intra 4x4 w. residue pred. 4

 Three kinds of scalabilities:  Quality (SNR) scalability ▪ Fine-Grain-Scalability (FGS) ▪ Bit-plane coding  Spatial scalability ▪ Decimation ▪ Wavelet transform  Temporal scalability ▪ Hierarchical B-picture 30 fps 15 fps 7.5 fps 4CIF CIF QCIF 5

 Hierarchical B-picture  H.264 allows B pictures may or may not be used as references.  Hierarchical prediction.  Temporal scalability can be achieved by hierarchical truncating B pictures Key Picture Group of Pictures (GOP size = 16) 16 Key Picture 6 Level 1 Level 2 Level 3 Level 4

Higher temporal level, larger distance between current and reference frames. Frames at higher temporal level are the references frames of subsequent lower temporal level frames. Level 1 Level 2 Level 3 Level pictures away 8 pictures away pictures away

Statistical distribution of optimal MVs Obtained from full search. Total 7 test sequences. MVs are scattered sparsely at higher temporal levels. (%) Level 1 Level 2Level 3Level 4 Origin Within 9x Within 15x Level 1 Level 2 Level 3 Level

1.Data-Level Parallelism GOP, Slice, Picture, Macroblock GOP: Extensive memory usage limits its scalability. Picture: Difficult to identify independent pictures. Slice: Coding efficiency degrades due to slice boundaries. MB: Extensive requirement of synchronizations. Applicable to all encoders 2.Function-Level Parallelism Asymmetric workload Depends on encoder implementations 10

MB-Level (Wave-front) Parallelism: Only MB-Level parallelism can be achieved in traditional codecs. Extensive controls and synchronizations required. Frame-Level Parallelism: Using IBBPBBP pattern, set B pictures as non-reference pictures. 11

Proposed Picture Decomposition Based on Hierarchical B-Picture: Utilizing the hierarchical B-Picture structure, picture-level parallelism is allowed in SVC 12 Level 4 Level 3 Level 2 Level 1

Experimental results: execution time of motion estimation 13

 Experimental results: coding efficiency comparison 14

For parallel video encoding, modules like motion compensation and up- sampling are good candidates for data level parallel processing. Along with data level parallelism, the function level one can also be integrated into a hybrid scheme. Platform dependent issues such as power consumption and load balancing on asymmetric architectures are also important research issues 15

16