1. It’s a 3D World, After All
Alyosha Efros
CMU

2. The sad and miserable life of an object detector…

3. People Detection in the Middle Ages
The Empress Theodora with her court.
Ravenna, St. Vitale 6th c.

4. Multiscale processing
Nuns in Procession. French ms. ca

5. Many Object Categories
San Vitale basilica, Ravenna, Inspired Evangelist

6. Occlusion, Pose
Giotto, The Mourning of Christ, c.1305

7. Perspective North Doors (1424) East Doors (1452) Lorenzo Ghiberti
( )

8. Perspective gone wild
Piero della Francesca,
The Flagellation (c.1469)

9. Real World

10. Objects vs. Scenes

11. Scene Understanding (The Age of Titans)
Guzman (SEE), 1968
Hansen & Riseman (VISIONS), 1978
Barrow & Tenenbaum 1978
Brooks (ACRONYM), 1979
Marr (2 ½ D sketch), 1982
Ohta & Kanade, 1978

12. What Went Wrong? (The Age of Titans)

13. Learning Geometry
Torralba & Oliva, 2001

14. Learning Geometry
Hoiem, Efros, Hebert, 2005
Andrew Ng et al, 2006

15. Automatic Photo Pop-up
Original Image
Geometric Labels
Fit Segments
Cut and Fold
Novel View

16. The World Behind the Image
When people see an image, they see not a plane with patterns of color and texture, but the world behind the image (show video). We want to give computers this same capability – the ability to automatically get a 3D model from a single image, such as this….
Automatic Photo Pop-up, SIGGRAPH’05

17. Parsing Whole Scene

18. Scene

19. Objects and Scenes
Biederman’s violations (1981):

20. Probability, position (2D)
Torralba et al

21. Size
Torralba et al

22. Position (3D)
Saddeth, Torralba, Freeman, Wilsky, 2006

23. Support, Size 2 ? 3 ? 1 ?

24. Improving Object Detection

25. Improving Object Detection

26. Improving Object Detection

27. Improving Object Detection

28. Hoiem, Efros, Hebert, 2006

29. Qualitative Results

30. Top View

31. Interposition, Depth Layers

32. Discussion Questions
1. Can we inject 3D knowledge into appearance-based methods?
2. Should we build datasets supporting 3D shape?
3. Where does segmentation come in?
4. Is 3D scene modeling necessary?
5. Is explicit 3D (e.g. top-down view) necessary?
6. Is depth layer extraction the right problem? How to approach it?