1. 1Ellen L. Walker Recognizing Objects in Computer Images Ellen L. Walker Mathematical Sciences Dept Hiram College Hiram, OH 44234 walkerel@hiram.edu http://hirame.hiram.edu/~walkerel

2. 2Ellen L. Walker Why Recognize Objects? n “Put the clothing on the bed” n “Turn left at the apartment building”

3. 3Ellen L. Walker Steps in Object Recognition

4. 4Ellen L. Walker Formation of a 2D Image

5. 5Ellen L. Walker Images are Composed of Pixels Image has 138 x 255 pixels (35190 total) Pixel values range from 0 (black) to 255 (white)

6. 6Ellen L. Walker Edge Detection Use a technique called convolution to detect edges The 3x3 convolution of a pixel is the sum of the products of the pixel and its 8 neighbors with a 3x3 mask The convolution of the shaded cell with the mask is Abs(4x-1 + 3x0 + 2x1+ 4x-2 + 3x0 + 2x2 + 4x-1 + 3x0 + 2x1) = 8

7. 7Ellen L. Walker Edge Detection with Convolution 00000000 00111111 01432222 01432232 01432233 01432232 01432232 00000000 0612129880 05985560 00000130 00000000 00000010 00000000 00000000 06421000 013763100 016884310 016884420 016884410 00000000 00000000 012161410880 01816148660 016884440 0 884420 0 884420 00000000 horiz. conv. vert. conv. both convolutions -101 202 -101 -12-1 000 121

8. 8Ellen L. Walker Edges from Test Image After convolution and thresholding This is still an image … next find endpoints of straight segments

9. 9Ellen L. Walker Fitting Line Segments to Edge Pixels n For each path of connected edge pixels: l draw a segment between the endpoints of the path l find the point farthest from the line l if it’s too far, then make that point an endpoint and recurse

10. 10Ellen L. Walker Segments from Test Image n Circles denote endpoints n Note “broken” segments

11. 11Ellen L. Walker Grouping Collinear Line Segments n Line segments are collinear if: angle between them is small enough (  ) l distance between them is small enough (PD) l “gap” between them is small enough (G)

12. 12Ellen L. Walker Grouped Segments from Test Image n Long vertical and horizontal segments were found

13. 13Ellen L. Walker Higher Levels of Grouping n Junctions n Curves & Polygons (no T-Junctions allowed)

14. 14Ellen L. Walker Groups from the Test Image? n After grouping curves and polygons, and combining polygons that share an edge and no T-junctions n Finally, we’re ready to recognize!

15. 15Ellen L. Walker Types of Models n Exact Models — Describe single objects l e.g. CAD models for factory parts l the object is needed to construct the model! n Generic Models —Describe classes of objects l Exact models with parameters (e.g length, width, radius) l Structural constraints n describe how the object is constructed l Functional constraints n describe how the object is used

16. 16Ellen L. Walker Matching Method Depends on Model Level n Image l when objects are hard to describe (e.g. faces) l match by correlation (like convolution) n Features, Groups l good for geometric objects l match by finding correspondences between image and model features (or groups) n Complete Objects l good for classes of objects l match by applying constraints from model to objects derived from the image

17. 17Ellen L. Walker Sample Object Model (Box) n Constraints l Adjacent faces are perpendicular (12 constraints) l Adjacent edges are perpendicular (8 constraints) l Opposite faces are parallel (3 constraints) l Opposite edges are parallel (6 constraints)

18. 18Ellen L. Walker Finding Correspondences n Search the interpretation tree l each node is a correspondence l each path to a leaf is an interpretation l “prune” the tree based on constraints

19. 19Ellen L. Walker Structural Constraints n Buildings — structural constraints l Buildings have rectangular walls and flat horizontal roofs l Each outer wall of a building intersects exactly two other walls, forming a closed ring l Each edge of the roof is the perpendicular intersection of the roof and a wall

20. 20Ellen L. Walker Functional Constraints n Beds — functional constraints l A bed has a surface to sleep on that is relatively flat. l A bed has physical means of supporting the sleeping surface at a comfortable height. l A bed has clearance above and to at least one side of the sleeping surface.

21. 21Ellen L. Walker Considering Occlusion in the Interpretation n Include names & parameters of objects n Include hypotheses for hidden parts when feasible n Specify only what is guaranteed!

22. 22Ellen L. Walker Toward Real World Solutions n Increasingly complex models n Large number of alternatives to choose from n Uncertainty at all levels n Recognizing partially-visible objects n Recognition must be fast enough n Generating models automatically (learning!)