1. Physiology of Vision: a swift overview Pixels to Percepts A. Efros, CMU, Spring 2011 Some figures from Steve Palmer

2. © Stephen E. Palmer, 2002 Understanding the Brain Anatomy: The biological study of the physical structure of organisms. Physiology: The biological study of the functional structure of organisms. Anatomy versus Physiology © Stephen E. Palmer, 2002

3. The Brain is tricky business Aristotle thought it’s for cooling the blood Localized or distributed?

4. Phrenology How bad Machine Learning got started…

5. Localized or Distributed? Evidence from patients with partial brain damage Lots of useful data from soldiers in the Russo-Japanese war But also evidence for distributed nature of processing E.g. [Lashley] showed “graceful degradation” of memory performance in rats

6. © Stephen E. Palmer, 2002 The Visual System Both eye and brain are required for functional vision Two kinds of blindness: Normal blindness (eye dysfunction) Cortical blindness (brain dysfunction) © Stephen E. Palmer, 2002

7. The Visual System Eyes register optical information Pathways to occipital cortex “What” pathway to temporal cortex “Where” pathway to parietal cortex Convergence on frontal cortex Two pathways from V1 © Stephen E. Palmer, 2002

8. Pathways to the Brain Anatomy of Pathway to Visual Cortex © Stephen E. Palmer, 2002

9. Pathways to the Brain The Lateral Geniculate Nucleus (LGN) Waystation in Thalamus Projections from both eyes Six layers Projects to cortical area V1 © Stephen E. Palmer, 2002

10. Visual Cortex “Unfolded” view of visual areas in the macaque cortex (sizes not to scale). Map of Visual Areas in Cortex © Stephen E. Palmer, 2002

11. Visual Cortex Evidence from lesions of monkey cortex What/Where Pathways

12. © Stephen E. Palmer, 2002 Visual Cortex Visual agnosia Visual agnosia: Inability to identify objects and/or people Caused by damage to inferior (lower) temporal lobe Disruption of the “what” pathway What/Where Pathways Evidence from Neuropsychology Visual neglect Visual neglect: Inability to see objects in the left visual field Caused by damage to right parietal lobe Disruption of the “where” pathway © Stephen E. Palmer, 2002

13. Visual Cortex Visual Features Color Shape Depth Motion Feature-based Pathways Hypothesis Featural Pathways Separate neural pathways in which different features are processed. © Stephen E. Palmer, 2002

14. The Gross Summary The visual system is composed of many interactive functional parts: Eye (optics of image formation) Retina (light transduction) LGN (waystation?) Area V1 (hypercolumns) Higher cortical areas (features) Cortical pathways (what/where) © Stephen E. Palmer, 2002

15. Image Formation Digital Camera The Eye Film

16. Monocular Visual Field: 160 deg (w) X 135 deg (h) Binocular Visual Field: 200 deg (w) X 135 deg (h)

17. The Eye is a camera The human eye is a camera! Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris What’s the “film”? –photoreceptor cells (rods and cones) in the retina

18. The Retina

19. Retina up-close Light

20. © Stephen E. Palmer, 2002 Cones cone-shaped less sensitive operate in high light color vision Two types of light-sensitive receptors Rods rod-shaped highly sensitive operate at night gray-scale vision

21. Rod / Cone sensitivity The famous sock-matching problem…

22. © Stephen E. Palmer, 2002 Distribution of Rods and Cones Night Sky: why are there more stars off-center?

23. Electromagnetic Spectrum http://www.yorku.ca/eye/photopik.htm Human Luminance Sensitivity Function

24. Why do we see light of these wavelengths? © Stephen E. Palmer, 2002 …because that’s where the Sun radiates EM energy Visible Light

25. The Physics of Light Any patch of light can be completely described physically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm. © Stephen E. Palmer, 2002

26. The Physics of Light Some examples of the spectra of light sources © Stephen E. Palmer, 2002

27. The Physics of Light Some examples of the reflectance spectra of surfaces Wavelength (nm) % Photons Reflected Red 400 700 Yellow 400 700 Blue 400 700 Purple 400 700 © Stephen E. Palmer, 2002

28. The Psychophysical Correspondence There is no simple functional description for the perceived color of all lights under all viewing conditions, but …... A helpful constraint: Consider only physical spectra with normal distributions area mean variance © Stephen E. Palmer, 2002

29. The Psychophysical Correspondence MeanHue # Photons Wavelength © Stephen E. Palmer, 2002

30. The Psychophysical Correspondence VarianceSaturation Wavelength # Photons © Stephen E. Palmer, 2002

31. The Psychophysical Correspondence AreaBrightness # Photons Wavelength © Stephen E. Palmer, 2002

32. Three kinds of cones: Physiology of Color Vision Why are M and L cones so close?

33. Retinal Processing © Stephen E. Palmer, 2002

34. Single Cell Recording Microelectrode Amplifier Electrical response (action potentials) mV © Stephen E. Palmer, 2002

35. Single Cell Recording © Stephen E. Palmer, 2002

36. Retinal Receptive Fields Receptive field structure in ganglion cells: On-center Off-surround Stimulus condition Electrical response © Stephen E. Palmer, 2002

37. Receptive field structure in ganglion cells: On-center Off-surround Stimulus condition Electrical response Retinal Receptive Fields © Stephen E. Palmer, 2002

38. Receptive field structure in ganglion cells: On-center Off-surround Stimulus condition Electrical response Retinal Receptive Fields © Stephen E. Palmer, 2002

39. Receptive field structure in ganglion cells: On-center Off-surround Stimulus condition Electrical response Retinal Receptive Fields © Stephen E. Palmer, 2002

40. Receptive field structure in ganglion cells: On-center Off-surround Stimulus condition Electrical response Retinal Receptive Fields © Stephen E. Palmer, 2002

41. Receptive field structure in ganglion cells: On-center Off-surround Stimulus condition Electrical response Retinal Receptive Fields © Stephen E. Palmer, 2002

42. RF of On-center Off-surround cells Retinal Receptive Fields © Stephen E. Palmer, 2002

43. RF of Off-center On-surround cells Retinal Receptive Fields © Stephen E. Palmer, 2002 Surround Center

44. Retinal Receptive Fields

45. Receptive field structure in bipolar cells Light Retinal Receptive Fields © Stephen E. Palmer, 2002

46. Receptive field structure in bipolar cells Retinal Receptive Fields © Stephen E. Palmer, 2002

47. Visual Cortex aka: Primary visual cortex Striate cortex Brodman’s area 17 Cortical Area V1

48. Cortical Receptive Fields Single-cell recording from visual cortex David Hubel & Thorston Wiesel © Stephen E. Palmer, 2002

49. Cortical Receptive Fields Single-cell recording from visual cortex © Stephen E. Palmer, 2002

50. Cortical Receptive Fields Three classes of cells in V1 Simple cells Complex cells Hypercomplex cells © Stephen E. Palmer, 2002

51. Cortical Receptive Fields Simple Cells: “Line Detectors” © Stephen E. Palmer, 2002

52. Cortical Receptive Fields Simple Cells: “Edge Detectors” © Stephen E. Palmer, 2002

53. Cortical Receptive Fields Constructing a line detector © Stephen E. Palmer, 2002

54. Cortical Receptive Fields Complex Cells 0o0o © Stephen E. Palmer, 2002

55. Cortical Receptive Fields Complex Cells 60 o © Stephen E. Palmer, 2002

56. Cortical Receptive Fields Complex Cells 90 o © Stephen E. Palmer, 2002

57. Cortical Receptive Fields Complex Cells 120 o © Stephen E. Palmer, 2002

58. Cortical Receptive Fields Constructing a Complex Cell © Stephen E. Palmer, 2002

59. Cortical Receptive Fields Hypercomplex Cells © Stephen E. Palmer, 2002

60. Cortical Receptive Fields Hypercomplex Cells © Stephen E. Palmer, 2002

61. Cortical Receptive Fields Hypercomplex Cells © Stephen E. Palmer, 2002

62. Cortical Receptive Fields Hypercomplex Cells “End-stopped” Cells © Stephen E. Palmer, 2002

63. Cortical Receptive Fields “End-stopped” Simple Cells © Stephen E. Palmer, 2002

64. Cortical Receptive Fields Constructing a Hypercomplex Cell © Stephen E. Palmer, 2002

65. Mapping from Retina to V1

66. Why edges?