1. T.-S. Wang, K.-S. Choi, H.-S. Jang and S.-J. Ko Electronics Letters Sponsored by Institution of Engineering and TechnologyInstitution of Engineering and Technology IET Journal

2. Outline Introduction Problem Statement Proposed MCI Method Experimental Results Conclusions

3. Introduction In various MCFI techniques, even though reliable motion is estimated in ME, the interpolated frame can be degraded by annoying artifacts(blocky,flicker). This is mainly because MCFI can utilize wrong motion information around the occlusion region and the motion ambiguity (MA) region.

4. Problem Statement In [4][5]( Wang’s & Haan’s methods ), conventional methods produce the annoying artifacts since they cannot actually discriminate the FG and BG layers. We utilize the occlusion estimation method [6] to identify the occlusion and to remove the inaccurate MVs caused by the occlusion. [4] Wang, D.: ‘Comparison of motion-compensated algorithms for frame interpolation’, Opt. Eng., 2003, 42, pp. 586–590 [5] Haan, G.D., and Ojo, O.A.: ‘Robust motion-compensated video up-conversion’, IEEE Trans. Consume. Electron., 1997, 43, pp. 1045–1056 [6] Ince, S., and Konrad, J.: ‘Geometry-based estimation of occlusion from video frame pairs’. Proc. IEEE Int. Conf. Philadelphia, PA, USA, March 2005, Vol. II, pp. 933–936 In [4], MV having the lowest SAD is selected to solve MA on the assumption that the FG object produces the lowest SAD. In [5], the median of pixel values associated with the multiple MV candidates is exploited for the interpolated pixel value.

5. BG and FG objects can pass through a single location in the interpolated frame. To interpolate the MA region correctly, the motion of the FG object should be correctly chosen. Problem Statement (cont.) Inaccurate MVs caused by the occlusion is removed by [6].

6. Proposed MCI Method MA occurs whenever the magnitude of relative motion between the FG and BG layer, ||v FG − v BG ||, is larger than the size of the FG object. The FG object always moves away from the occlusion belonging to the BG layer(U). O x : Set of pixel positions of uncovered (U) in f n+1 S x : Set of the positions which the multiple MVs causing MA pass through in f n+1 OxOx SxSx

7. Proposed MCI Method (cont.) First find a position within the occlusion, x oc O x, such that ||x oc − x i || is minimal, x i S x. Then, the element of S x, farthest from x oc, is associated with the FG object. For the occlusion regions (no MV passes), the MV of the BG is easily estimated using the neighboring MV. OxOx SxSx

8. Proposed MCI Method (cont.2) However, such a straightforward approach to the distance calculation is computationally expensive. Using a simple morphological operator as follows: 1. The O x area is first do the dilation by repeating with the 3×3 structuring element. At each iteration, the dilated pixels are labeled with the number of the iteration. 2. When the MA occurs, S x is obtained and the distance label is examined. The pixel with the largest label value is utilized to interpolate the frame(FG). 3. Step 2 is repeated until the overlapped area is completely interpolated.

9. Proposed MCI Method (cont.2) OxOx SxSx

10. Experimental Results

12. Conclusions We propose a novel layer inference-based MCI method using the occlusion information. Experimental results confirm that the proposed method achieves much higher visual quality of the interpolated frame without annoying artifacts in both the MA and occlusion regions