Fast disparity motion estimation in MVC based on range prediction Xiao Zhong Xu, Yun He ICIP 2008

Outline Disparity estimation in MVC Disparity search range decision Search range reduction Experimental Results Conclusion

Analysis(1/3) Disparity of two views is decided by distance between objects and cameras displacement between two cameras Disparity could be 30~50 pixels for background and up to 100 pixels for foreground 40 pixels 75 pixels

Analysis(2/3) The directions of disparity are identical for all blocks as long as the relative positions between cameras are not changed. Global disparity vector The majority of encoding time is consumed by motion estimation(ME) and disparity estimation(DE). Over 99% encoding time is spent on ME and DE for full search.

Analysis(3/3) Comparison of corresponding bitrate reduction using different search ranges Search range(SR) = ±16 ， QP=22/27/32/37 The larger search range is, the more bitrate can be saved. computation time increases as well

MVC Motion skip mode (1/2) Proposed by LG Co., JVT-W081(April, 2007). Motion information of current MB can be inferred from the corresponding MB in the picture with the same temporal index of the neighboring view. mb_type motion vector (MV) reference indices

MVC Motion skip mode (2/2) Two main stages 1.Search for corresponding MB Global disparity vector (by using SAD) is used. 2.Derivation of motion information Comparing all the MB mode results, and then choose the best.

Disparity map storage Disparities of temporal static objects will be the same of the same view. After encoding inter-view picture, the disparity map is then refreshed by the disparities of the current picture. The disparity map of the current picture is recorded as a disparity predictor. For disparity prediction, the disparity map is stored. Disparity map Disparity estimation Input picture

Local disparity feature decision Three already obtained disparity are used: Co-located MB in the disparity map (dis_col) The disparity of the current 16*16 block (dis_16) The disparity predicted by spatial neighboring blocks (dis_pred) In 16*16 mode, dis_16 is set to 0.

Disparity intensity Disparity intensity: Categorizing the current block into 3 different modes: Low disparity mode – background (DV < 2) High disparity mode – foreground (DV > 3) Intermediate disparity mode (other cases)

Search range reduction(1/2) The directions of disparity are identical for all blocks Symmetrical search window is redundant. Once global disparity direction is detected, the opposite direction could be paid less attention to. Global disparity: summing up the already obtained disparity vectors and check its sign.

Search range reduction(2/2) Using an unsymmetrical window. 4 directional factors, 2 scale factors: Negative vertical scale (NVS) Positive vertical scale (PVS) Negative horizontal scale (NHS) Positive horizontal scale (PHS) Vertical scale (VS) Horizontal scale (HS) directional factors scale factors Initial search window Global disparity direction new search window Low disp.Inter. disp.High disp. VS=4VS=2VS=1 HS=8HS=4HS=1 Global disparity is positive: PVS = VS (PHS=HS) Global disparity is negative: NVS=VS (NHS=HS) ±16pixel search range ： 2 pixel 8 pixel 4 pixel PVS NVS PHSNHS PHS NVS PVS

Simulation results(1/3) Only inter-view prediction is allowed. IIII PPPP Time View Initial search range = ±96 QP=22,27,32,37 50 frames are coded in each view Sequence resolution: 640*480

Simulation results(2/3) ISRP: Inter-view search range prediction JMVM uses extended diamond search DE speedup comparison:

Simulation results(3/3) Encoding time comparison of DE: ◦ The rest parts of encoder are recognized as 1 unit.

Conclusion Combination of ISRP and JMVM performs well. Search range of disparity is properly reduced. retaining certain quality while significantly improving encoding speed of MVC.