What is vision Aristotle - vision is knowing what is where by looking

What is vision Aristotle - vision is knowing what is where by looking Helmholtz - vision is an act of unconscious inference Our percepts are inferences about properties of the world from sensory data

What is vision Aristotle - vision is knowing what is where by looking Helmholtz - vision is an act of unconscious inference Our percepts are inferences about properties of the world from sensory data Vision is (neural) computation

Processes of vision

What is vision Aristotle - vision is knowing what is where by looking Helmholtz - vision is an act of unconscious inference Our percepts are inferences about properties of the world from sensory data Vision is (neural) computation Vision controls action

The swinging room

Processes of vision II

Lecture outline Image transduction Neural coding in the retina Neural coding in visual cortex Visual pathways in the brain

The Optic Array: pattern of light intensity arriving at a point as a function of direction (q, W), time (t) and wavelength(l) I = f(q, W, t, l)

Goals of eye design Form high spatial resolution image Accurately represent light intensities coming from different directions. E.g. minimize blur in a camera Maximize sensitivity Trigger neural responses at very low light levels. Particle nature of light places fundamental limit on sensitivity.

Visual angle

Point spread

Point spread function

Resolution (acuity) Optics of eye and physics of light pace fundamental limit on acuity Width of blur circle in fovea = 1’ Blur increases with eccentricity Optical aberrations Depth variation in the environment

Focus - the lens equation

Accommodation - bringing objects into focus Focused on Focused on

Some numbers Refractive power of cornea Refractive power of lens 43 diopters Refractive power of lens 17 (relaxed) - 25 diopters Other eyes Diving ducks - 80 diopter accommodation range Anableps - four eyes with different focusing power

Sampling in the fovea Receptor sampling in fovea matches the width of point spread function (blur circle) Effective width of blur circle ~ 1 minute of arc Spacing of receptors ~ .5 minutes of arc (theoretical requirement for optimal resolution)

Relationship between sampling and blur Two test images 60 cycle / degree grating 120 cycle / degree grating

Relationship between sampling and blur Two test images 60 cycle / degree grating 120 cycle / degree grating Receptor sampling = .5 minutes

Relationship between sampling and blur Two test images 60 cycle / degree grating 120 cycle / degree grating Receptor sampling = .5 minutes 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 .5 Receptor Output

Peripheral sampling More rods than cones in periphery Coarser sampling in periphery

Tricks for maximizing resolution

High resolution coding of intensity information Problem: High resolution coding of intensity information

Example Computer monitors typically use 8 bits to encode the intensity of each pixel. 256 distinct light levels Old monitors only provided 4 bits per pixel. 16 distinct light levels Number of light levels encoded = intensity resolution of the system. Human visual system can only distinguish ~ 200 - 250 light levels.

Code wide range of light intensities Range of light intensities receptors can encode Dynamic range of receptors and of ganglion cells limits # of distinguishable light levels. Problem How does system represent large range of intensities while maintaining high intensity resolution?

Some typical intensity values

Solution Dynamic range of receptors (cones) 10 - 1000 photons absorbed per 10 msec. Range of intensities in a typical scene 10-6 - 10-4 cd / m2 in starlight 102 - 104 cd / m2 in sunlight 100:1 range of light intensities Only need to code 100:1 range of intensities within a scene Solution - Adaptation adjusts dynamic range of receptors to match range of intensities in a scene.

Increase Illumination Adaptation # photons hitting receptor % photons absorbed # photons absorbed Scene 1 10 - 1,000 100% 10 - 1,000 Increase Illumination Adaptation 10% 10 - 1,000 Scene 2 1,000 - 100,000

Sunlight Starlight Moonlight Indoor lighting 10-4 10-2 10 102 104 106 10-6 Window of visibility

(e.g. % photons absorbed = .1) Adapt to the dark (e.g. % photons absorbed = .1) Starlight Moonlight Indoor lighting Sunlight 10-6 10-4 10-2 10 102 104 106 Window of visibility

(e.g. % photons absorbed = .000000001) Adapt to the bright (e.g. % photons absorbed = .000000001) Starlight Moonlight Indoor lighting Sunlight 10-2 10 102 10-4 104 106 10-6 Window of visibility

Hartline Experiment Limulus eye has ommotidia containing one receptor each. Each receptor sends a large axon to the brain. Output of one receptor was inhibited by light shining on a neighboring receptor (lateral inhibition).

Ganglion cell receptive fields Receptive field - region of visual field that cell responds to. Center-surround receptive field On-center, off-surround Off-center, on-surround - - - + + - - - - + + - + - + - + - - - + + + - - + - + - - + + - - + - - - + - - - + - - - + +

Ganglion cells as computational devices Write a mathematical function that calculates firing rate of cell from luminance pattern. 1st guess Increase in firing rate = weighted sum of intensities within receptive field. Problem 1 - Adaptation Problem 2 - dark regions in inhibitory region actually excite cell

Ganglion cells as computational devices Solution Increase in firing rate = weighted sum of local contrast values within receptive field. Local contrast C(x,y) = I(x,y) / M - 1

Ganglion cells as computational devices Solution Increase in firing rate = weighted sum of local contrast values within receptive field. Local contrast C(x,y) = I(x,y) / M - 1 Intensity

Ganglion cells as computational devices Solution Increase in firing rate = weighted sum of local contrast values within receptive field. Local contrast C(x,y) = I(x,y) / M - 1 Mean intensity Intensity

Ganglion cells as computational devices Solution Increase in firing rate = weighted sum of local contrast values within receptive field. Local contrast C(x,y) = I(x,y) / M - 1 Mean intensity Local contrast Intensity

Ganglion Cells Simple Cells

Cells in V1 Simple cells Complex cells orientation selective scale selective (cells have different size receptive fields) some are motion selective some are end-stopped Complex cells same properties as simple cells, BUT ... insensitive to position of stimulus within RF

Cortical pathways V2 MT MST Parietal Lobe V1 V3 V4 Temporal Lobe