Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson

What is the 3D Structure of Each Image?

Infinite Number of Possible Interpretations

Goal 1: 3D Shape Recovery

Goal 2: Check Shading Consistency

Input Patterns

Goal 1: Propose 3D Structure ● Questions: – What distinguishes correct structure? – How to search for it algorithmically?

What Distinguishes Correct Structure? WrongRight

What Distinguishes Correct Structure? ● Low “Complexity” – Low Angle Variance – Planarity of Faces – Overall Compactness

How to Search? ● Minimize Cost Function – How to weight constraints? – Cumbersome ● Incremental Solution – Start with 2-D Line Drawing – “Pull” vertices until regularity is maximized ● i.e. Gradient descent in regularity space

Incremental Solution

Goal 2: Check Shading Consistency ● Given – 3D structure – 2D gray-level image ● Assume – Structure is uniformly colored ● Find – Single light source to account for shading

Quantitative Approach ● Given: – Lambertian Reflectance Model – Surface Normal – Surface Brightness ● Defines: – A cone of valid light directions for each surface

Cone of valid light direction N=surface normal E=brightness N

Quantitative Approach ● Consider cones for all surfaces ● Intersection is direction of illumination

Problems ● Small changes in grey lead to no solution Surface l 3 changes brightness Gradient Space Input Patterns Before After Intersection

Qualitative Approach ● Observation – Human vision ● Good at judging relation between brightness ● Bad at judging absolute brightness ● So... – Use binary relations to: ● find light source ● Not commit to particular reflectance function

Qualitative Approach ● Each surface now defines a hemisphere of possible light directions – Overall consistency implies finding a non-null intersection of hemispheres

Qualitative Approach ● Hemisphere ~ set of vectors t s.t Angle between s and t is less than 90

Qualitative Approach ● Hemisphere ~ set of vectors t s.t sij is defined between a surface i and surface j, with normals ni and nj

Qualitative Approach ● Hemisphere ~ set of vectors t s.t s is perpindicular to the average of the normals

Qualitative Approach ● Hemisphere ~ set of vectors t s.t s is perpindicular to plane defined by the normals

Qualitative Approach ● Hemisphere ~ set of vectors t s.t The angle between s and ni is less than 90 & the angle between s and nj is greater

Solution ● Solution to constraints lie on a convex polygon on the unit sphere. Direction of agreement

Is it consistent? ● If no polygons found that satisfy all constraints, then – Shading is not consistent No maximum

Largely Solveable ● For each polygon, count # of constraints matched – A maximum indicates most likely lighting direction

Compound Edges ● What do unsatisfied constraints represent? – Compound edge – where surface changes color, not just shading Compound edges

Justification ● Why do shape derivation with line drawings and not brightness? – Humans can use edges ● Can humans use grey level? – It seems like it – An experiment

An Experiment ● Random Height Tesselation – Can human determine 3d structure? 3D Representation Overhead View

Limitations ● Polyhedral objects only ● Later research addresses contoured objects w/ smoothly changing brightness.

Conclusions ● Humans mostly use edges to determine 3D structure ● Use shading to verify this determination ● Algorithm effective for polyhedra