1. Depth estimation and Plane detection
for Image Analysis and Understanding
Cheng Hanni
Huang Jingjing
Song Mengting
Xiao Ying

2. Project Description
Depth estimation is the set of techniques and algorithms aiming to obtain the distance of each 3D point to the image plane.
The goal of plane detection is to identify pixels in a 2D image which locates on a 2D plane.
Depth estimation and Plane detection are for 3D reconstruction, scene understanding and robotics etc.

3. Overview Literature investigation Website establishment
Depth estimation
Plane Detection
Conclusion

4. Literature investigation
Learning Depth from Single Monocular Images In Proc. of NIPS.2005
Depth Map Prediction from Single Image using Multi Scale Deep Network. in Proc. of NIPS, 2014.
Accurate 3D Ground Plane Estimation from a Single Image, In Proc. of ICRA 2009
Detecting planes and estimating their orientation from a single image, In Proc. of BMVC 2011.

5. Website Establishment
Django is a free and open-source web framework, written in Python, which follows the model-view-template (MVT) architectural pattern.
Our templates include demo.html and show_pic.html
The core code of depth-estimation and plane-detection are under the folder ‘homework (app)’,we call the two functions in views.py.
If I run the django project on my computer, then the url “ can leads to our demo.

6. Depth estimation Methods：Multi-Scale Deep Network
Global Coarse-Scale Network: contains five feature extraction layers of convolution and max-pooling,followed by two fully connected layers.to predict the overall depth map structure using a global view of the scene.

7. Depth estimation Methods：Multi-Scale Deep Network
Local Fine-Scale Network:The fine-scale network stack consists of convolutional layers only, along with one pooling stage for the first layer edge features. To edit the coarse prediction it receives to align with local detais such as object ande wall edges.

8. Depth estimation
Scale-Invariant Error

9. Plane Detection Methods
SLIC Algorithm to over-segment a picture (TPAMI 2012)
For each 2D point in the picture, add the information of depth map to reconstruct a 3D point.
Get the adjacent-matrix according to the segmented clusters.
Go through the adjacent matrix using BFS, judge if two adjacent matrix are coplanar.(dimensionality reduction using SVD)
Compute cosine of each plane

10. Plane Detection Experiment Results a) Source image b) Depth map
c) Plane with avg color(ICRA 2009)
d) Plane with box (demo video)

11. Plane Detection Analysis Solution
The method in ICRA 2009 divide a picture into several irregular parts, draw a bounding box according to the top-left and bottom-right point of a part can’t get the precise result.
The method depends on many parameters,(sample number, cosine threshold), one certain parameter cannot fit all picture.
Solution
Train a number of pictures to choose the right parameters which can make the training error smallest.
Extract other features (texture/gradient) from one picture to get better result.
But time is limited, I… don’t try above methods.()

12. Conclusion
We use Django Framework written in python to establish our website, so that our front-end page can connect with our picture-processing codes.
CNN can be applied to depth estimation. We write a python script to get the output depth map from the pre-trained caffe model.
Plane detection includes three steps.
Reconstruct 3D points using depth information
Over-segment the picture using SLIC algorithm
Treat the coplanar parts of the segmented clusters as a plane.

13. Links http://10.13.49.144:8000/demo/ (Not always available) Demo:
Project website:
Source code:
Course website:

14. Thanks.