1. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 1 Computational Architectures in Biological Vision, USC, Fall 2004 Lecture 4: Introduction to Vision. Reading Assignments: Chapters 2 and 3 of textbook.

2. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 2 Today’s lecture - The challenges: -Optics and image formation -Sampling and image representation -Theoretical limits - The biological approach: -Organization of the primate retina -Trading accuracy for coverage: moving eyes - The engineering approach: -Arrays of photosensitive sensors -On-board processing and VLSI sensors -Trading accuracy for coverage: multiple moving cameras

3. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 3 Projection

4. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 4 Projection

5. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 5 Convention: Visual Angle Rather than reporting two numbers (size of object and distance to observer), we will combine both into a single number: visual angle e.g., the moon: about 0.5deg visual angle your thumb nail at arm’s length: about 1.5deg visual angle 1deg visual angle: 0.3mm on retina

6. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 6 Charge-Coupled Devices Uniform array of sensors Very little on-board processing Very inexpensive

7. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 7 Optics limitations: acuity

8. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 8 Sampling

9. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 9 Sampling

10. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 10 Sampling

11. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 11 The Big Question

12. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 12 Convolution & Fourier Transforms

13. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 13 Sampling in the frequency domain

14. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 14 Reconstruction

15. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 15 Aliasing

16. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 16 Sampling Theorem

17. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 17 Aliasing

18. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 18 Eye Anatomy

19. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 19 Visual Pathways

20. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 20 Image Formation Accomodation: ciliary muscles can adjust shape of lens, yielding an effect equivalent to an autofocus.

21. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 21 Phototransduction Cascade Net effect: light (photons) is transformed into electrical (ionic) current.

22. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 22 Rods and Cones Roughly speaking: 3 types of cones, sensitive to red, green and blue.

23. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 23 Processing layers in retina

24. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 24 Retinal Processing

25. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 25 Center-Surround Center-surround organization: neurons with receptive field at given location receive inhibition from neurons with receptive fields at neighboring locations (via inhibitory interneurons).

26. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 26 Early Processing in Retina

27. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 27 Color Processing

28. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 28 Over-representation of the Fovea Fovea: central region of the retina (1-2deg diameter); has much higher density of receptors, and benefits from detailed cortical representation.

29. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 29 Fovea and Optic Nerve

30. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 30 Blind Spot

31. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 31 Retinal Sampling

32. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 32 Retinal Sampling

33. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 33 Seeing the world through a retina

34. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 34 Now with actual eye movements Here is a video clip… we recorded eye movements while observers watched it…

35. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 35

36. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 36 Shifted to eye position

37. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 37

38. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 38 Sampling & Optics Because of blurring by the optics, we cannot see infinitely small objects…

39. Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 4: Introduction to Vision 39 Sampling & optics The sampling grid optimally corresponds to the amount of blurring due to the optics!