1. Serdar Ince and Janusz Konrad Acoustics, Speech, and Signal Processing, 2005. (ICASSP '05). IEEE International Conference

2. Outline Introduction Photometry-based Estimation of Occlusions Geometry-based Estimation of Occlusions – Traditional Approach Geometry-based Estimation of Occlusions – New Approach Experimental Results

3. Introduction Occlusion effects occurring in image sequences are a natural consequence of changing object juxtaposition in a 3-D scene. Object disappears in following frame: occlusion areas (A) Object appears in following frame: newly-exposed areas (B) For regularization, occlusion and newly-exposed areas must be explicitly known.

4. Pixels that disappear(occluded) cannot be accurately matched in the target frame. Therefore induce significant errors: d f : forward motion anchored on sampling grid of F 1 d b : backward motion For target frame F 1 : Occluded: | ε f | > Θ Newly-exposed: |ε b | > Θ Threshold : Θ Photometry-based Estimation of Occlusions F1F1 F2F2 d f [x] I 1 [x] I 2 [x+d f [x]] -d b [x] I 2 [x]

5. Geometry-based Estimation of Occlusions – Traditional Approach Mismatch of forward and backward motion vectors is due to disappearing image areas(occlusion). The vector matching errors indicate the occlusion and newly-exposed areas: For target frame F 1 : Occluded: | ρ f | > Δ Newly-exposed: | ρ b | > Δ Threshold : Δ F1F1 F2F2 d f [x] -d b [ x+d f [x] ]

6. Geometry-based Estimation of Occlusions – New Approach Newly-exposed pixels(in area B) have no relationship with the reference frame and, as such, cannot be pointed to by forward motion vectors. Areas in the target frame that are void of motion-compensated projections can be relatively easily detected.

7. Geometry-based Estimation of Occlusions – New Approach (cont.) Algorithm: Λ: set of 2-D sampling lattice for I 1,I 2 S: {y : y = x + d f [x], x ∈ Λ} r: distance from z i to x For target frame F 1 : Newly-exposed: M[x] < Ψ For r = 2, max{M(x)}= 13, let Ψ = 6 Threshold : Ψ ΛS x r

8. Since it is easier to find regularly-spaced neighbors(Λ) than spaced irregularly (S), the algorithm is implemented differently in practice. For each z i ∈ S, its neighbors x ∈ Λ, such that ||z i − x|| ≤ r, are found, and each neighbor’s counter is incremented. ※ card{Λ} ≥ card{S} After all z i have been scanned, each counter contains the number of projections within distance of r. Geometry-based Estimation of Occlusions – New Approach (cont.2) xixi zizi

9. Experimental Results In all the results shown, motion was computed using 8×8 block matching under spatial regularization. We measure the accuracy of detection using the ground-truth (union of false-positives and misses).

10. Experimental Results