1. Chapter 7: Taking Action

2. The Ecological Approach to Perception
Approach developed by J. J. Gibson (began in late 1950s)
Gibson felt that traditional laboratory research on perception was:
too artificial - observers were not allowed to move their heads.
unable to provide an explanation for how pilots used environmental information to land airplanes
The instructor can emphasize that the approach focuses on the moving observer and identifying information in the environment that the moving observer uses for perception. The approach can be stated as “Look for information in the environment that provides information for perception.”

3. The Ecological Approach to Perception - continued
Optic array - structure created by the surfaces, textures, and contours in the environment
Optic flow - appearance of objects as the observer moves past them
Gradient of flow - difference in flow as a function of distance from the observer
Focus of expansion - point in distance where there is no flow

4. Optic Flow
Self-produced information - flow is created by the movement of the observer
Invariant information - properties that remain constant while the observer is moving

5. Figure 7.1 The side and top of the bridge and the road below appear to move toward a car that is moving forward. This movement is called optic flow.
Figure 7-1 p154

6. Figure 7. 2 Optic flow created by an airplane coming in for a landing
Figure 7.2 Optic flow created by an airplane coming in for a landing. The focus of expansion (FOE), indicated by the red dot, is the place where the plane will touch down on the runway.
Figure 7-2 p154

7. Self-produced Information
Somersaulting
Could be performed by learning a predetermined sequence of moves; thus performance would be the same with and without vision
Bardy and Laurent found that expert gymnasts performed worse with their eyes closed.
They use vision to correct their trajectory.
Novice gymnasts do not show this effect.

8. Figure 7.3 The relationship between movement and flow is reciprocal, with movement causing flow and flow guiding movement. This is the basic principle behind much of our interaction with the environment.
Figure 7-3 p155

9. Figure 7.4 “Snapshots” of a somersault, or backflip, starting on the left and finishing on the right.
Figure 7-4 p155

10. The Senses Do Not Work in Isolation
Experiment by Lee and Aronson
13- to 16-month-old children placed in “swinging room”
In the room, the floor was stationary but the walls and ceiling swung backward and forward.
The movement creates optic flow patterns.
Children swayed back and forth in response the flow patterns created in the room.

11. The Senses Do Not Work in Isolation - continued
Adults show the same response as children when placed in the swinging room.
Results show that vision has a powerful effect on balance and even overrides other senses that provide feedback about body placement and posture.

12. Figure 7. 5 Lee and Aronson’s swinging room
Figure 7.5 Lee and Aronson’s swinging room. (a) Moving the wall toward the observer creates an optic flow pattern associated with moving forward, so (b) the observer sways backward to compensate. (c) As the wall moves away from the observer, flow corresponds to moving backward, so the person leans forward to compensate and may even lose his or her balance.
Figure 7-5 p156

13. Navigating Through the Environment
Optic flow neurons - neurons in the medial superior temporal area (MST) of monkeys respond to flow patterns
Experiment by Britten and van Wezel
Monkeys were trained to respond to the flow of dots on a computer screen.
They indicated whether the dots flowed to the right, left, or straight ahead.

14. Figure 7.6 (a) Optic flow generated by a person moving straight ahead toward the vertical line on the horizon. The lengths of the lines indicate the person’s speed. (b) Optic flow generated by a person moving in a curved path that is headed to the right of the vertical line.
Figure 7-6 p157

15. Figure 7.7 The human brain, showing the medial superior temporal area (MST), which responds to optic flow, as discussed here. Other areas, which will be discussed later, are the parietal reach region (PRR) in the parietal lobe, which is involved in reaching and grasping, and the premotor cortex (PM), which is involved in observing other people’s actions.
Figure 7-7 p157

16. Figure 7.8 (a) Response of a neuron in the monkey’s MST that responds to an expanding stimulus, but hardly responds to a stimulus that moves in a circular motion. (b) A neuron that responds to circular movement, but doesn’t respond to expansion.
Figure 7-8 p158

17. Navigating Through the Environment - continued
Experiment by Britten and van Wezel
As the monkeys did the task, microstimulation was used to stimulate MST neurons that respond to specific directions of flow patterns.
Judgments were shifted in the direction of the stimulated neuron.

18. Figure 7.9 (a) A monkey watches a display of moving dots on a computer monitor. The dots indicate the flow pattern for movement slightly to the left of straight ahead. (b) Effect of microstimulation of the monkey’s MST neurons that were tuned to respond to leftward movement. Stimulation (red bar) increases the monkey’s judgment of leftward movement.
Figure 7-9 p158

19. Driving a Car
Experiment by Land and Lee
Car fitted with instruments to measure
Angle of steering wheel
Speed of vehicle
Direction of gaze of driver
When driving straight, driver looks straight ahead but not at focus of expansion

20. Experiment by Land and Lee - continued
When driving around a curve, driver looks at tangent point at side of the road
Results suggest that drivers use other information in addition to optic flow to determine their heading.
They might be noting the position of the car in relation to the center line or side of the road.

21. Figure 7. 10 Results of Land and Lee’s (1994) experiment
Figure 7.10 Results of Land and Lee’s (1994) experiment. The ellipses indicate the place where the drivers were most likely to look while driving down (a) a straight road and (b) a curve to the left.
Figure 7-10 p159

22. Walkers correct when target drifts to left or right
Walking
Visual direction strategy - observers keep their body pointed toward a target
Walkers correct when target drifts to left or right
Blind walking experiments show that people can navigate without any visual stimulation from the environment.

23. Figure 7.11 (a) As long as a person is moving toward the tree, it remains in the center of the person’s field of view. (b) When the person walks off course, the tree drifts to the side. (c) When the person corrects course, the tree moves back to the center of the field of view, until (d) the person arrives at the tree.
Figure 7-11 p159

24. Figure 7.12 The results of a “blind walking” experiment (Philbeck et al., 1997). Participants looked at the target, which was 6 meters from the starting point, then closed their eyes and begin walking to the left. They turned either at point 1 or 2, keeping their eyes closed the whole time, and continued walking until they thought they had reached the target.
Figure 7-12 p160

25. Wayfinding
Landmarks involved taking routes the involves making turns
Landmarks are objects on the route that serve as cues to indicate where to turn.

26. Figure 7. 13 Effect of removing landmarks on maze performance
Figure 7.13 Effect of removing landmarks on maze performance. Red = all landmarks are present; green = half have been removed. (a) Removing half of the least fixated landmarks has no effect on performance. (b) Removing half of the most fixated landmarks causes a decrease in performance.
Figure 7-13 p160

27. Figure 7.14 The human brain, showing three structures important to navigation: the parahippocampal gyrus, the hippocampus, and the retrosplenial cortex.
Figure 7-14 p161

28. Wayfinding - continued
Experiment by Janzen and van Turennout
Observers studied a film that moved through a “virtual museum.”
They were told that they should be able to act as a guide within the museum.
Exhibits appeared both at decision points where turns were necessary and non-decision points.

29. Experiment by Janzen and van Turennout - continued
Observers were given a recognition task while in an fMRI.
They were presented objects they had seen as exhibits, and ones they had not seen.
Results showed the greatest activation for objects at decision points (landmarks) in the parahippocampal gyrus.

30. Figure 7.15 (a & b) Two locations in the “virtual museum” viewed by Janzen and van Turennout’s (2004) observers. (c) Brain activation during the recognition test for objects that had been located at decision points (red bars) and non-decision points (blue bars). Notice that brain activation was greater for decision-point objects even if they weren’t remembered.
Figure 7-15 p161

31. Effects of Brain Damage on Wayfinding
Retrosplenial cortex damage
Maguire (2006)
Patient T. T.
Parahippocampus gyrus
Retrosplenial cortex
Hippocampus

32. Figure 7.16 Responses of a patient with retrosplenial cortex damage when she was asked to identify the viewpoint of a photograph of her garden. The green arrow indicates the correct viewpoint of the photograph. The three red arrows are the patient’s indications of the viewpoints. She was able to identify the garden table, but she could not indicate the direction from which it was seen.
Figure 7-16 p162

33. Figure 7.17 A view similar to the one in the video game The Getaway (© Sony Computer Entertainment Europe), which duplicates the roadways and buildings of downtown London.
Figure 7-17 p162

34. Affordances - What Objects Are Used for
Gibson believed affordances of objects are made up of information that indicates what an object is used for.
They indicate “potential for action” as part of our perception.
People with certain types of brain damage show that even though they may not be able to name objects, they can still describe how they are used or can pick them up and use them.

35. The Physiology of Reaching and Grasping
Neurons in the parietal lobe that are silent when a monkey was not behaving, fire when the monkey reached to press a button to receive food.
This response only happened when the animal was reaching to achieve a goal.
The instructor should use the next slide to explain the experiment by Calton etl al.

36. Figure 7.18 Picking up a cup of coffee: (a) perceiving and recognizing the cup, (b) reaching for it, and (c) grasping and picking it up. This action involves coordination between perceiving and action that is carried out by two separate streams in the brain, as described in the text.
Figure 7-18 p164

37. The Physiology of Reaching and Grasping - continued
Experiment by Connolly - evidence for PRR in humans
Fattori – three different regions are used in grasping
Schindler – obstacle avoidance is controlled by the parietal regions

38. Figure 7.19 (a) The monkey’s task in Fattori and coworkers’ (2010) experiment. The monkey always looks at the small light above the sphere. The monkey sees the object to be grasped when the lights go on, then reaches for and grasps the object once the lights go off and the fixation light changes color. (b) Four of the objects used in the task. Each one involves a different type of grasping movement.
Figure 7-19 p165

39. Figure 7.20 Results of Fattori and coworkers’ (2010) experiment showing how three different neurons respond to reaching and grasping each of the objects. Neuron A responds best to “whole hand prehension” (starred record). Neuron B responds to “advanced precision grip.” Neuron C responds to all of the grips.
Figure 7-20 p165

40. Figure 7.21 (a) Subjects in Schindler and coworkers’ (2004) experiment had to reach between the two cylinders to touch a gray strip located behind the cylinders. (b) The pairs of cylinders in Schindler and coworkers’ (2004) experiment were located in different positions on different trials, as shown in this top view. The red arrows show that control subjects adjusted their reach to compensate for the different locations of the cylinders. The blue arrows, which show the data for one of the ataxia patients, indicate that the patients’ reach stayed the same for all arrangements of the cylinders.
Figure 7-21 p166

41. Mirroring Others’ Actions in the Brain
Mirror neurons in the cortex of monkeys that
Respond when a monkey grasps an object and when an experimenter grasps an object
Response to the observed action “mirrors” the response of actually grasping
There is a diminished response if an object is grasped by a tool (such as pliers).

42. Mirror Neurons in Premotor Cortex - continued
Possible functions of mirror neurons
To help understand another animal’s actions and react to them appropriately
To help imitate the observed action
Audiovisual mirror neurons - respond to action and the accompanying sound
Mirror neurons may help link sensory perceptions and motor actions.

43. Video: Mirror Neurons

44. Figure 7. 22 Response of a mirror neuron
Figure 7.22 Response of a mirror neuron. (a) Response to watching the experimenter grasp food on the tray. (b) Response when the monkey grasps the food. (c) Response to watching the experimenter pick up food with a pair of pliers.
Figure 7-22 p167

45. Figure 7.23 Response of an audiovisual mirror neuron to four different stimuli.
Figure 7-23 p167

46. Predicting People’s Intentions
Iacoboni (2005) – mirror neurons can be influenced by different intentions
The human action observation system consists of neurons in the premotor cortex that contain mirror neurons and some other areas of the brain as well. The instructor should use the next two slides to provide details about the experiment and the results.

47. Figure 7.24 Images from the Context, Action, and Intention film clips viewed by Iacoboni and coworkers’ (2005) subjects. See text for details.
Figure 7-24 p168

48. Figure 7.25 Iacoboni and coworkers’ (2005) results, showing the brain response for the Action, Drinking, and Cleaning conditions.
Figure 7-25 p168

49. Action-Based Accounts of Perception
The traditional approach to perception is focused on how the environment is represented in the nervous system
According to action-based accounts of perception the purpose of perception is to create a representation in the mind of whatever you are looking at