1. Tae-Shick Wang; Kang-Sun Choi; Hyung-Seok Jang; Morales, A.W.; Sung-Jea Ko; IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010 ENHANCED FRAME RATE UP- CONVERSION METHOD FOR UHD VIDEO

2. Introduction(1/2) Motion blur caused by inherent characteristic of LCD FRUC MCFI consists of ME and MCI.

3. Introduction(2/2) The motion vector field(MVF) is estimated between two successive frames. MVF should represent the actual object motion Several ME methods for MCFI have been proposed to find the true motion based on BMA Multi-size block matching algorithm Bi-directional Motion Compensate Interpolation(MCI)

4. Motion analysis for UHD video(1/2) Enlarge SR can span regions which belong to another object but are more similar to the current block

5. Motion analysis for UHD video(2/2) Full search, sort all MVs for each block in UHD Increasing order of SAD Search range(SR) should be kept as small as possible if a reliable MV is initially given Large block size gives a more correct motion

6. Proposed ME algorithm The blocks with similar motion are merged to an entity. Introduce a segment-based ME algorithm Divide the frame into several segments Estimate the motion for each segment Large region can be obtained more reliably f n : current frame

7. Block-based segmentation Classify each block into three patterns: edge, plane, and texture Segments are obtained by merging adjacent blocks with an identical pattern Segments are obtained by merging adjacent blocks with an identical pattern

8. Gradient calculation For each block, the gradient is calculated by using the Sobel operator [14]. Gradient gradientimage If large=>significant pixel [14] R. C. Gonzalez and R.E. Woods, Digital Image Processing, 2nd Edition, Prentice Hall, New Jersey, 2002.

9. Gradient Direction Histogram Largest number in the histogram # of significant pixels gx gy = r

10. Segmentation result MVF of f n-1 : k-th segment from f n

11. Segment selection Find segments whose motion can be obtained accurately. Temporal consistency of the segment MV of the segment with high reliability 1. Large segment is more reliably, 2. The pattern of segment is plane or texture 3. has dominant MV pass through  Dominant MV : over 70% MVs are identical

12. Efficient true motion estimation(1/3) Full search with low computation complexity SSAD (subsample SAD) as matching criterion # of blocks in the segment => sub-sample rate σ

13. Efficient true motion estimation(2/3) Determine the search range  Reliable block (RB) : SSAD < 5(B/σ)^2  => the MV’s similarity to the actual motion of the segment => search range  Search range size L SR in the segment: ε =8

14. Efficient true motion estimation(3/3) Three stage ME Refine dominant MV within L SR Refine the received MV (if average SSAD < 5(B/σ)^2) and within L SR No initial MV within max SR

15. Experimental result Three UHD video sequence: Toy and Calendar, Table Setting, and Tractor Compare Ha’s [2] and Huang’s [10] methods [2] T. Ha, S. Lee, and J. Kim, "Motion compensated frame interpolation by new block-based motion estimation algorithm," IEEE Trans. Consumer Electron., vol. 50, no. 2, pp. 752-759, May 2004. [10] A.-M. Huang and T. Nguyen, "A multistage motion vector processing method for motion-compensated frame interpolation," IEEE Trans. Image Process., vol. 17, no. 5, pp. 694-708, May 2008.

16. Subjective comparison

17. Objective comparison Improve quality for the interpolated frame by 2~3 dB Reduce the computation load. RC: relative complexity

18. Conclusion The block-based segmentation method was confirmed to produce meaningful segment information with low complexity. The proposed method can also be successfully employed for various applications including De-interlacing View interpolation for multi-view video.