A New Block Based Motion Estimation with True Region Motion Field Jozef Huska & Peter Kulla EUROCON 2007 The International Conference on “Computer as a Tool”

Outline  Introduction  Ambiguity in Motion Estimation Yielding from Optical Flow  Used Models and Estimation Criteria  Region Based Multi-Stage Motion Estimation Framework for the Algorithm  Detection of the Regions, Corner, Edge and Flat Blocks  Motion Vector Field Interpolation and Refinement Stage  Simulation Results  Conclusions

Introduction  True motion vector fields are usually smooth, both spatially and temporally.  To Finding the optimal trade off between minimizing the entropy of vectors and minimizing the displaced frame difference.

Ambiguity in Motion Estimation Yielding from Optical Flow  Assumption : image point has the same intensity along the trajectory of movement:  The consequences:  1) Underdetermined component  2) Aperture problem  3) Indeterminate flow vector

 1) Underdetermined component:  There is only one equation for two unknowns (v x and v y ).  Solution: apply motion measurement to a block of pixels.  2) Aperture problem:  The aperture including only one spatial gradient.  Solution: the estimated aperture contains at least two different gradient directions.  3) Indeterminate flow vector:  In regions with constant brightness so that ∇ I = 0.  The estimation of motion is reliable only in regions with edges or non-flat textures. Ambiguity in Motion Estimation Yielding from Optical Flow (cont.)

Used Models and Estimation Criteria  Considerations of a new motion estimator for the video coding systems:  The goal is to estimate the motion of image points.  Motion is estimated based on the variations of intensity and color in image points.  The motion of neighboring image points within the object’s projection area is very similar.  Linear block trajectory model for temporal motion :

Region Based Multi-Stage Motion Estimation Framework for the Algorithm  Strategies to get around the ambiguities while keeping the prediction error:  Applying motion estimation to block of pixels. (1  Estimation of motion vector fields only for corner blocks(two spatial gradients directions). reliable  For edge blocks is the motion vector value first interpolated from the corner blocks. (2  For flat blocks interpolation are token corner and edge blocks. (3

 The interpolated motion fields based on sparse reliable estimates are more accurate than unreliable dense estimates.  Interpolation concepts:  Interpolation of the motion vectors is done with vectors within the own region.  Motion vector field inside each region would be smooth. Region Based Multi-Stage Motion Estimation Framework for the Algorithm

Detection of the Regions, Corner, Edge and Flat Blocks  Corner detection algorithm:  Covariance matrix A :  The eigenvalues :  Two large eigenvalues of A represent corners.

 An image is segmented into regions with the graph based segmentation [5].  The ability of segmentation is preserving detail in low variability image regions while ignoring detail in high variability regions. Detection of the Regions, Corner, Edge and Flat Blocks (cont.) [5] P. F. Felzenszwalb, D. P. Huttenlocher, "Efficient Graph-Based Image Segmentation," International Journal of Computer Vision, vol. 59, no. 2, Sept Segmentation parameters: Sigma(σ)= 0.5, K = 1000, Min = 100

Detection of the Regions, Corner, Edge and Flat Blocks (cont.2)

Motion Vector Field Interpolation and Refinement Stage  The MVs of corner blocks are estimated with the diamond search (DS).  Interpolation of the MVs for the edge blocks is done with the MVs of the corner blocks.  First, found four nearest corner blocks with MVs {d xi, d yi } from the same region.

 The interpolated MV value is:  |RI i | : the distance between the reference block and currently interpolated block.  Interpolation of the MVs for the flat blocks is done with the same function, but consider corner and edge blocks as the reference. Motion Vector Field Interpolation and Refinement Stage (cont.)

 Predictive diamond search [19] was used for the refinement with small modification.  Like candidates in 3DRS algorithm, there is used interpolated motion vector. Motion Vector Field Interpolation and Refinement Stage (cont.2)

Algorithm description DS : diamond search, MV : motion vector, R : count of regions, C : count of corner locks, E : count of edge blocks, F : count of flat blocks.

Simulation Results  Real sequence:

Simulation Results

 Synthetic sequence: Simulation Results V c : correct velocity V e : estimated velocity

Conclusions  In this paper a new block based true motion estimation algorithm named true region motion field (TRMF) search was presented.  In future work we would like slightly reduce computation demands of our TRMF search and increase output quality.