1. CORNELL UNIVERSITY CS 764 Seminar in Computer Vision Ramin Zabih Fall 1998

2. CORNELL UNIVERSITY 2 Course mechanics, revised n Meeting time will be Tue/Thu 11-12, here No meeting next week; next meeting is Sept 24 RDZ will send email reminder n Home page is now up at: www/CS764 Check out the suggested readings

3. CORNELL UNIVERSITY 3 What is the visual system’s “contract” n Most of the field is low-level vision or model- based recognition Well-defined problems Not what you want for almost any task n Key question: how to avoid brittleness? Can make the visual system compute just what we need for our task (I.e., berries) But how to handle the unexpected (I.e., lions)?

4. CORNELL UNIVERSITY 4 Our path for 764 n No good computational work to read n We will examine papers along these lines: Computational approaches that failed Psychological data that is highly suggestive Neurologically inspired architectures Cognitive scientists and philosophers –Their goal is argument, not algorithm! –They’ve thought the most about these issues

5. CORNELL UNIVERSITY 5 Today: active vision and attention n Faster computers made some tasks attainable Especially, robotics n Basic observations: Robots desperately need vision No one needs what the vision community is trying to provide! One can get by with a lot less –And build robots that work now

6. CORNELL UNIVERSITY 6 The pendulum swings back “In the active vision paradigm, the basic components of the visual system are visual behaviors tightly integrated with the actions they support; these behaviors may not require elaborate categorical representations of the 3-D world… The cost of generating and updating a complete, detailed model of our everyday world is too high” “Active vision encompasses attention, selective sensing in space, resolution and time” -Swain & Stricker, Promising directions in active vision, 1991

7. CORNELL UNIVERSITY 7 Attention n Emphasize the top-down selection of what to compute But is there a cost in brittleness? n Some very nice experimental work Treisman’s spotlight model Extensive psychological studies –But, very simple stimuli! n A natural approach to implement

8. CORNELL UNIVERSITY 8 Visual attention n Psychophysicists study this area a lot Some models seem pretty computational Issues include “pop-outs” n Most famous studies involve response time as a function of number of distractors Linear response implies visual search –Top down driven! Constant response implies pop-outs

9. CORNELL UNIVERSITY 9 Experimental results n A Q will pop out in a field of O’s An O will not pop out in a field of Q’s n A different color will also pop out Red T in a field of green T’s n But: a different color and a different shape will not! Find the red Q among green Q’s, red and green O’s

10. CORNELL UNIVERSITY 10 Spotlight model n Some properties are computed “bottom up” Everywhere, all the time Example: color, motion n A field can signal an outlier, but only one field at once n We use an attentional “spotlight” in order to compute, I.e., conjunctions of prperties Not the same as the fovea!

11. CORNELL UNIVERSITY 11 Computational proposals n Ullman’s influential Visual Routines paper Attentional in flavor, not in details n Pengi and “deictic” representations Moved away from object-based representations n Ullman’s more recent work on sequence seeking Mixture of neural and computational modeling

12. CORNELL UNIVERSITY 12 Some suggested intuitions n Good ideas that lack computational proposals Shortcuts for recognition –Mentioned in Ullman’s recent work –Part of Vera’s thesis Choice of properties to compute –What best distinguishes the thing you want from the other things out there? –How do you know what else will appear?