Subject wearing a VR helmet immersed in the virtual environment on the right, with obstacles and targets. Subjects follow the path, avoid the obstacels, and touch the targets.

Top down view of the experiment. A sequence of fixations are shown, directed at the path (green), obstacles (red), and targets. The subject’s path is shown by the thin red line and shows two instances of avoidance of an obstacle (pink). In both cases avoidance occurs while the S is fixating the next target (blue).

General question: How is vision used when walking? 1.How far away is the subject when the obstacle is fixated? (Choose the end of the fixation.) 2. Does this distance very for chairs vs people vs bins? Why? 3. Are some obstacles not fixated at all? Why? 4. What location on the obstacle is fixated? Eg is it the edge closest to the body? 5. Is there any change in gaze behavior as the path becomes more familiar? Why?

A basic visual task: walking

Where do we look when we walk? Why do we look there? Suppose that complex behavior can be broken down into simpler sub-tasks. What sub-tasks can be identified when walking? - Control heading - Avoid obstacles - Update location -Remember surroundings Can we identify the function of the fixations made while walking? Each sub-task can be accomplished either by bottom up capture of attention or by top down search for relevant visual information. The top down factors reflect learning. In this lab we would like to: 1.Describe gaze behavior as a first step to understanding how vision is used. 2.See if we can find evidence for learning where obstacles are. Such learning is critical for top down control.

Looming – a potential bottom up cue that attracts gaze Neurons in Area MT sensitive to looming stimuli.

target selection signals to muscles inhibits SC saccade decision saccade command Neural Circuitry for Saccades Substantia nigra pc (Dopamine) Motion sensitive area MT

Optic Flow – the pattern on the retina when walking

Optic Flow might be used for controlling heading direction. This is commonly thought to be a bottom up mechanism.

How Gaze Patterns are Learned Neuroeconomics

Fixation on Collider

Learning to Adjust Gaze Changes in fixation behavior fairly fast, happen over 4-5 encounters (Fixations on Rogue get longer, on Safe shorter)

Shorter Latencies for Rogue Fixations Rogues are fixated earlier after they appear in the field of view. This change is also rapid.

Neural Substrate for Learning Neurons in substantia nigra pc in basal ganglia release dopamine. These neurons signal expected reward. Neurons at all levels of saccadic eye movement circuitry are sensitive to reward. This provides the neural basis for learning gaze patterns in natural behavior, and for modeling these processes using Reinforcement Learning.

target selection signals to muscles inhibits SC saccade decision saccade command planning movements Neural Circuitry for Saccades Substantia nigra pc Substantia nigra pc modulates caudate

Neurons at all levels of saccadic eye movement circuitry are sensitive to reward. LIP: lateral intra-parietal cortex. Neurons involved in initiating a saccade to a particular location have a bigger response if reward is bigger or more likely SEF: supplementary eye fields FEF: frontal eye fields Caudate nucleus in basal ganglia

This provides the neural basis for learning gaze patterns in natural behavior, and for modeling these processes using Reinforcement Learning. (eg Sprague, Ballard, Robinson, 2007)

Virtual environments allow direct comparison of human behavior and model predictions in the same natural context. Use Reinforcement Learning models with an embodied agent acting in the virtual environment. Modelling Natural Behavior in Virtual Environments.

Assume behavior composed of a set of sub-tasks Sprague, Ballard, Robinson, 2007; Rothkopf,2008 Modelling behaviors using virtual agents

Model agent after learning Pickup litter Follow walkway Avoid obstacles

Choose the task that reduces uncertainty of reward the most Controlling the Sequence of fixations obs can side

Humans walk in the avatar’s environment

Reward weights estimated from human behavior. Human path Avatar path

Time fixating Intersection. “Follow the car.” or “Follow the car and obey traffic rules.” CarRoadsideRoadIntersection Shinoda et al. (2001) Detection of signs at intersection results from frequent looks.

Intersection P = 1.0 Mid-block P = 0.3 Greater probability of detection in probable locations Suggests Ss learn where to attend/look. How well do human subjects detect unexpected events? Shinoda et al. (2001) Detection of briefly presented Stop signs.