1. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as computing a belief distribution Conscious perception better thought as a decision based on both beliefs and the utility of the choice.

2. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Hierarchical Organization of Visual Processing

3. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Visual Areas

4. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Circuit Diagram of Visual Cortex

5. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Motion Perception as Optimal Estimation

6. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Local Translations OpticFlow: (Gibson,1950) Assigns local image velocities v(x,y,t) Time ~100msec Space ~1-10deg

7. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Measuring Local Image Velocity Reasons for Measurement Optic Flow useful: l Heading direction and speed, structure from motion,etc. Efficient: l Efficient code for visual input due to self motion (Eckert & Watson, 1993) How to measure? Look at the characteristics of the signal

8. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 X-T Slice of Translating Camera x t x y

9. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 X-T Slice of Translating Camera x t x y Local translation

10. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Early Visual Neurons (V1) Ringach et al (1997) y x y x t x

11. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 What is Motion? As Visual Input: Change in the spatial distribution of light on the sensors. Minimally, dI(x,y,t)/dt ≠ 0 As Perception: Inference about causes of intensity change, e.g. I(x,y,t) v OBJ (x,y,z,t)

12. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Motion Field: Movement of Projected points

13. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

14. Basic Idea 1) Estimate point motions 2) use point motions to estimate camera/object motion Problem: Motion of projected points not directly measurable. -Movement of projected points creates displacements of image patches -- Infer point motion from image patch motion –Matching across frames –Differential approach –Fourier/filtering methods

15. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

17. Problem: Images contain many edges-- Aperture problem Normal flow: Motion component in the direction of the edge

18. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

20. Aperture Problem (Motion/Form Ambiguity) Result: Early visual measurements are ambiguous w.r.t. motion.

21. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Aperture Problem (Motion/Form Ambiguity) However, both the motion and the form of the pattern are implicitly encoded across the population of V1 neurons. Actual motion

22. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Plaids This pattern was created by super- imposing two drifting gratings, one moving downwards and the other moving leftwards. Here are the two components displayed side-by-side. Rigid motion

23. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Find Least squares solution for multiple patches.

24. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Motion processing as optimal inference Slow & smooth: A Bayesian theory for the combination of local motion signals in human vision, Weiss & Adelson (1998) Figure from: Weiss & Adelson, 1998

25. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

26. Modeling motion estimation From: Weiss & Adelson, 1998 Local likelihood: Global likelihood: Prior: Posterior:

27. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Figures from: Weiss & Adelson, 1998

28. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Figure from: Weiss & Adelson, 1998

29. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Figure from: Weiss & Adelson, 1998

30. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Figure from: Weiss & Adelson, 1998

31. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Lightness perception as optimal inference

32. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Illuminant surface reflectances Surface normal N Light dir. L I(x,y)

33. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

34. Land & McCann’s lightness illusion

35. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Neural network filter explanation

36. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Apparent surface shape affects lightness perception Knill & Kersten (1991)

37. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Inverse graphics solution What model of material reflectances, shape, and lighting fit the image data?

38. PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004