Efficient multi-frame motion estimation algorithms for MPEG-4 AVC/JVTH.264 Mei-Juan Chen, Yi-Yen Chiang, Hung- Ju Li and Ming-Chieh Chi ISCAS 2004

Outline Introduction Introduction FDVS FDVS –FDVS for multi-frame –FDVS for multi-frame in H.264 VADS VADS – –VADVS in H.264 Refinement Experiment Conclusion

Backgrounds H.264 can refer from previous 1~5 frames, that can get smaller prediction error and obtain better performance. MPEG-4 AVC/JVT/H.264 motion estimation involves considerable complexity, which increases linearly with the number of reference frames.

Introduction In multi-frame motion estimation, we reuse the motion information of each frame to its previous one frame for obtaining the MV.

Introduction Forward Dominant Vector Selection (FDVS) for multi-frame motion estimation in h.264 Variable block size Activity Dominant Vector Selection (VADVS) in H.264 Refinement for the Composed MV

FDVS for multi-frame FDVS ： For frame-rate conversionframe-rate conversion – –new MVs for outgoing bit-stream need to be computed – –generates the outgoing MV from the incoming MVs instead of performing the full-scale motion estimation

Frame-rate conversion

FDVS MV (n-2) (estimated) =MV 1 + MV 2 =I (n) 1 +I (n-1) 2

FDVS for multi-frame Dominant MV ： MV carried by a dominant macroblock Dominant macroblock ： macroblock has the largest overlapping segment with the block pointed by the incoming motion vector

FDVS for multi-frame Current Frame Ref.0 Frame

FDVS for multi-frame motion estimation in h.264 H.264 standard can 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 block size to perform the motion compensation. Therefore, we must store the seven block size MV with for FDVS algorithm for the previous reference frame.

FDVS for multi-frame motion estimation in h.264 Variable block size in h.264 (16x16 8x8 4x4) Variable block size in h.264 (16x16 8x8 4x4) MV 5 =I 1 (n) +I 2 (n-1) +I 1 (n-2) +I 4 (n-3) +I 3 (n-4)

FDVS for multi-frame motion estimation in h.264 Variable block size in h.264 Variable block size in h x8 and 8x4 16x8 and 8x4 8x16 and 4x8 8x16 and 4x8

Variable block size Activity Dominant Vector Selection ( VADVS ) FDVS selects the dominant block with the largest overlapping area from the four neighboring blocks. ADVS algorithm utilizes the activity information of the macroblock. – –Counting the numbers of NZ (nonzero quantized DCT coefficients) of covered 8x8 residual blocks represents the activity information of a macroblock.

VADVS 8x8 dct residual

VADVS in h.264 In H.264 standard, residual coding uses 4X4 block size to perform the transformation. H.264 standard uses integer transform instead of the DCT transform. Dominant MV ： the MV carried by a dominant block. dominant block ： block that has the maximum NZ

VADVS in h.264

Refinement for composed MV A composed MV is usually not optimal because each dominant motion vector is an approximated value of the optimal MV. Horizontal and Vertical Search (HAVS) to refine the composed motion vector.

Refinement for composed MV

Experiment bit-rate variation is less than 2%

Experiment PSNR difference is less than 0.1dB compared with the reference software JM61E_REF5.

Experiment

Conclusion In this paper, we propose the efficient algorithms to improve the coding speed by 4 times in average, and maintain the performance.