1. Ai-Mei Huang And Truong Nguyen Image processing, 2006 IEEE international conference on Motion vector processing based on residual energy information for motion compensated frame interpolation

2. INTRODUCTION Low bandwidth requirements, i.e. video telephony by skipping frames low frame rate video usually results in motion jerkiness Motion-compensated frame interpolation (MCFI) Directly uses the received MVs Suffer from annoying artifacts such as blockiness and ghost effect. Motion re-estimation is not suited for mobile devices Video in encoder decoder Reconstructed frame Bit stream Video coding/compression

3. Motion compensated frame interpolation In MCFI methods, a skipped frame is often bi- directionally interpolated from its two neighboring reconstructed frames. by using the received MVs of the second frame. MV is not always reliable!

4. The correlation between motion vector reliability and residual energy. Macroblock(MB) contain areas of different motion. Encoder favors the MV that can represent most of the region The prediction residual will be generated and encoded. Correct those unreliable MVs for frame interpolation Avoid using those unreliable MVs to correct other MVs.

5. The proposed method Based on residual energy associated with each motion vector iterative process, stops when the MAP is no longer changed, or reaches a pre-defined maximum iteration, max_ite.

6. (1)motion vector classification based on residual energy Classify MVs to 3 Groups. Calculate every 8x8 block’s residual energy E m,n By taking the sum of the absolute value of each reconstructed prediction error for each pixel. E m,n <threshold  MV will be classified as reliable and place into first group G1. E m,n >=threshold  MV will be classified as unreliable and place into first group G3. Unreliable MV’s neighboring MVs within the same MB will be classified as possibly unreliable into the second group G2 (base unit) 16*16 MB=>four 8*8 blocks

7. (2) motion vector correction Works on those unreliable MVs Correcting from neighbor reliable MV A residual-energy constrained median filter (RECMF) is used in this process and defined as S contains the neighboring MVs centered at v* m,n. Select a new MV only from its neighboring reliable MVs. v m,n itself will be excluded from the candidates. we prevent unreliable MVs to be used to correct other unreliable MVs.

8. (3)motion vector similarity check Ensure v* m,n is not identical or similar to v m,n Remain unreliable Use angle variance by taking inner product of the two vectors. two MVs is similar,fail in the check => b m,n still in G3 Pass in the check => b m,n take in G1

9. (4)motion vector re-sampling and smoothing In [7], each 8 x 8 block is further broken into four 4 x4 sub-blocks. The MVs of these four sub-blocks can be obtained simultaneously by minimizing a smoothness measure, which is defined in the following. For example: [7] G. Dane and T. Q. Nguyen, "Smooth motion vector resampling for standard compatible video post-processing," Proc. Asilomar Conf: Signals, Systems and Computers, 2004.

11. SIMULATIONS two video sequences, FOREMAN and SILENT, CIF frame resolution 30 frame per second (fps). both encoded using H.263 but even frames are skipped by the encoder. fix quantization parameter (QP) values. Avg. bitrate FOREMAN is 240.3 Kbps SILENT is184.59 Kbps