3D Shape Inference Computer Vision No.2-1

Pinhole Camera Model the image plane the camera center Principal axis

Perspective Projection the camera center the optical axis Focal length the image plane

Orthographic Projection the camera center the optical axis the image plane

Weak Perspective Projection the reference plane the camera center the optical axis the image plane

Para Perspective Projection the reference plane the camera center the optical axis the image plane

Orthographic Projection the camera center the optical axis the image plane

Obtain a 3D Information form Line Drawing Given Line drawing(2D) Find 3D object that projects to given lines How do you think it’s a cube, not a painted pancake?

Line Labeling Significance Pioneers Provides 3D interpretation(within limits) Illustrates successful(but incomplete)approach Introduces constraints satisfaction Pioneers Roberts(1976) Guzman(1969) Huffman&Clows (1971) Waltz (1972)

Outline Types of lines types of vertices Junction Dictionary Labeling by constraint propagation Discussion

Line Types concave convex occluding occluding

Labeling a Line Drawing Easy to label lines for this solid →Now invert this in order to understand shape

Enumerating Possible Line Labeling without Constraints 9 lines 4 labels each →4ｘ4ｘ4ｘ4ｘ4ｘ4ｘ4ｘ4ｘ4= 250,000 possibilities We want just one reality must reduce surplus possibilities →Need constraints　(by 3D relationship)

Vertex Types Divide junctions into categories Need some constraints to reduce junction types

Restrictions No shadows, no cracks Non-singular views At most three faces meet at vertex

Fewer Vertex Types

Vertex Labeling Three planes divide space into octants Trihedral vertex at intersection of 3 planes Enumerate all possibilities　 (Some full, some empty)

Enumerating Possible Vertex Labeling(1) 0or8octants full--no vertex 2,4,6 octants full singular view 7octants full 1FORK 5octants full 2L,1ARROW

3octants full Enumeration(2) upper behind L right above L left above L straight above ARROW straight below FORK

１octant--Seven viewing octants supply Enumeration(3) １octant--Seven viewing octants supply

Huffman&Clows Junction Dictionary Any other arrangements cannot arise Have reduced configuration from 144 to 12

Constraints on Labeling Without constraints-- 250,000possibilities Consider constraints →3ｘ3ｘ3ｘ6ｘ6ｘ6ｘ5＝ 29,000possibilities We can reduce more by coherency/consistency along line.

Labeling by Constraint Propagation “Waltz filtering” By coherence rule, line label constrains neighbors Propagate constraint through common vertex Usually begin on boundary May need to backtrack

Example of Labeling

Line drawing can have multiple labelings Ambiguity Line drawing can have multiple labelings

Wire-frame cube Necker Reversal(1) Human perception flips from one to the other (After Necker 1832,Swiss naturalist)

Necker Reversal(2)

Impossible Objects No consistent labeling But some do have a consistent labeling What’s wrong here?

Limitations of Line Labeling Only qualitative;only gets topology Something wrong

Preliminary 3D analysis of shape Summary(1) Preliminary 3D analysis of shape 1. Identify 3D constraint 2. Determine how constraint affects images 3. Develop algorithm to exploit constraint --> General method for 3D vision Tool:constraint propagation/satisfaction

Summary(2) Problems 1. Significant ambiguity possible 2. Assumes perfect segmentation 3. Can be fooled without quantitative analysis

Gradient Space Computer Vision No. 2-2

Gradient Space and Line Labeling Last time: line labeling by constraint propagation Use gradient space to represent surface orientation － ＋

Review of Line Labeling Problem Given a line drawing, label all the lines with one of 4 symbols ＋ convex edge － concave edge ←→ occluding edges Approach Narrow down the number of possible labels with a vertex catalog ＋ －

Surface Normal Normal of a plane Rewrite Normal vector (A,B,C)

Surface Gradient Gradient of surface is Gradient of plane

Surface Gradient p1 p3 p2 y q p

Relationship of Normal to Gradient (p,q) 1 p q x y Normal Vector x p1 p4 p5 q p p1 p3 p2 y

Polyhedron in Gradient Space F E D C B I A ＋ － x y A’ D’ C’ B’ I’ H’ G’ F’ E’ p q Top view of polyhedron A ∥ x-y plane Same order as left

Vector on a Surface Suppose vector on surface with gradient Under orthography, vector in scene projects to is surface normal vector, so

Vector on Two Surfaces Suppose vector on boundary between two surfaces Surfaces have gradients and If , then p q G1 G2

Ordering of Points Along Gradient Line Perpendicular to Connect Edge B1’ B2’ B3’ A p q B1 B3 B2 S T A If connect edge ST convex, then points on gradient space maintain same order (left-right) as A and Bi in image If ST concave, then order switches

How does this gradient space stuff help us to label lines? ＋ L is a “connect edge” (vector on two surface) Assume orthography Line in gradient space connecting R1 and R2 must be perpendicular to line L

Line Labeling using Gradient Space 1. Assign arbitrary gradient (0,0) to A 2. Consider B lines 1,2 may be connect edges or may be occluding edges 3. Suppose line 1 a connect edge 4. Suppose line 2 a connect edge, then (line A’B’) (line 2) impossible. So line 2 occluding. B A C 1 2 3 4 5 B’ A’ p q

Line Labeling using Gradient Space 5. Suppose lines 3 and 4 are connect edges 6. and so forth can get multiple interpretations B A C 1 2 3 4 5 B’ A’ p q C’ C ＋ －

Another Payoff: Detect Inconsistencies L2 L1 L1 L2

Summary Can use gradient space to represent surface orientation detect inconsistent line labels constraint labeled line drawings establish line labels without the vertex catalog

References M.B. Clowes, “On seeing things,” Artificial Intelligence, Vol.2, pp.79-116, 1971 D.A. Huffman, “Impossible objects as nonsense sentences,” Machine Intelligence, Vol.6, pp.295-323, 1971 A.K.Mackworth, “On reading sketch maps,” 5th IJCAI, pp.598-606, 1977