1. Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson

2. What is the 3D Structure of Each Image?

3. Infinite Number of Possible Interpretations

4. Goal 1: 3D Shape Recovery

5. Goal 2: Check Shading Consistency

6. Input Patterns

7. Goal 1: Propose 3D Structure ● Questions: – What distinguishes correct structure? – How to search for it algorithmically?

8. What Distinguishes Correct Structure? WrongRight

9. What Distinguishes Correct Structure? ● Low “Complexity” – Low Angle Variance – Planarity of Faces – Overall Compactness

10. How to Search? ● Minimize Cost Function – How to weight constraints? – Cumbersome ● Incremental Solution – Start with 2-D Line Drawing – “Pull” vertices until regularity is maximized ● i.e. Gradient descent in regularity space

11. Incremental Solution

12. Goal 2: Check Shading Consistency ● Given – 3D structure – 2D gray-level image ● Assume – Structure is uniformly colored ● Find – Single light source to account for shading

13. Quantitative Approach ● Given: – Lambertian Reflectance Model – Surface Normal – Surface Brightness ● Defines: – A cone of valid light directions for each surface

14. Cone of valid light direction N=surface normal E=brightness N

15. Quantitative Approach ● Consider cones for all surfaces ● Intersection is direction of illumination

16. Problems ● Small changes in grey lead to no solution Surface l 3 changes brightness Gradient Space Input Patterns Before After Intersection

17. Qualitative Approach ● Observation – Human vision ● Good at judging relation between brightness ● Bad at judging absolute brightness ● So... – Use binary relations to: ● find light source ● Not commit to particular reflectance function

18. Qualitative Approach ● Each surface now defines a hemisphere of possible light directions – Overall consistency implies finding a non-null intersection of hemispheres

19. Qualitative Approach ● Hemisphere ~ set of vectors t s.t Angle between s and t is less than 90

20. Qualitative Approach ● Hemisphere ~ set of vectors t s.t sij is defined between a surface i and surface j, with normals ni and nj

21. Qualitative Approach ● Hemisphere ~ set of vectors t s.t s is perpindicular to the average of the normals

22. Qualitative Approach ● Hemisphere ~ set of vectors t s.t s is perpindicular to plane defined by the normals

23. Qualitative Approach ● Hemisphere ~ set of vectors t s.t The angle between s and ni is less than 90 & the angle between s and nj is greater

24. Solution ● Solution to constraints lie on a convex polygon on the unit sphere. Direction of agreement

25. Is it consistent? ● If no polygons found that satisfy all constraints, then – Shading is not consistent No maximum

26. Largely Solveable ● For each polygon, count # of constraints matched – A maximum indicates most likely lighting direction

27. Compound Edges ● What do unsatisfied constraints represent? – Compound edge – where surface changes color, not just shading Compound edges

28. Justification ● Why do shape derivation with line drawings and not brightness? – Humans can use edges ● Can humans use grey level? – It seems like it – An experiment

29. An Experiment ● Random Height Tesselation – Can human determine 3d structure? 3D Representation Overhead View

30. Limitations ● Polyhedral objects only ● Later research addresses contoured objects w/ smoothly changing brightness.

31. Conclusions ● Humans mostly use edges to determine 3D structure ● Use shading to verify this determination ● Algorithm effective for polyhedra