3-D Depth Reconstruction from a Single Still Image 何開暘

Visual Cues for Depth Perception Monocular Cues  Texture variations, texture gradients, interposition, occlusion, known object sizes, light and shading, haze, defocus Stereo Cues Motion Parallax and Focus Cues

image → feature → depth Chose features that capture 3 types of cues: texture variations, texture gradients, and color Model conditional distribution of depths given monocular image features p(d|x) Estimate parameters by maximizing conditional log likelihood of training data Given an image, find MAP estimate of depths

Outline Introduction Feature Vector Probabilistic Model Experiments Reference

Feature vectors Two types of features  Absolute depth features―used to estimate absolute depth at a particular patch  Relative features―used to estimate relative depths Capture three types of cues  Texture variation―apply Law ’ s masks to intensity channel  Haze―apply a local averaging filter to color channels  Texture gradient―apply six oriented edge filters to intensity channel

Features for Absolute Depth Compute summary statistics of a patch i in the image I(x,y) as follows  Use the output of each of the 17 (9 Law’s masks, 2 color channels and 6 texture gradients) filters Fn, n=1,…,17 as: (dimension 34) To estimate absolute depth at a patch, local image features centered on the patch are insufficient Use more global properties

More Global Properties Use image features extracted at multiple spatial scales (three scale) Features used to predict depth of a particular patch are computed from that patch as well as 4 neighboring patches (Repeated at each of the three scales) Add to features of a patch additional summary features of the column it lies in (5*3+4)*34=636 dimensional

Features for Relative Depth To learn the dependencies between two neighboring patches Compute a 10-bin histogram of each of the 17 filter outputs, giving a total of 170 features y is for each patch i at scale s Relative depth features y ijs for two neighboring patches i and j at scale s will be the differences between their histogram, i.e., y ijs =y is -y js

Outline Introduction Feature Vector Probabilistic Model Experiments Reference

Gaussian Model

Laplacian Model

Outline Introduction Feature Vector Probabilistic Model Experiments Reference

Result

Improving Performance of Stereovision using Monocular Cues

The average errors as a function of the distance from the camera

Reference A.Y. Ng A. Saxena, S.H. Chung. 3-d depth reconstruction from a single still image. In International Journal of Computer Vision (IJCV), Michels, J., Saxena, A., & Ng, A. Y. (2005). High speed obstacle avoidance using monocular vision and reinforcement learning. In 22 nd international conference on machine learning (ICML).