1. Outline: Mental Imagery
When do we use Imagery?
Subcomponents of Imagery .
Relation to perception
Types of Imagery: Spatial vs. Visual
Oral Presentations: Info
Next, we will consider evidence that tests a stronger hypothesis about the pictorial aspects of mental images: Does mental imagery use the same brain structures as visual perception? (alternatively, it could be that mental images just function similarly to visual perception, while being carried out through independent mechanisms)

2. Mental Imagery

3. Which one is bigger, a tennis ball or a 100-watt light bulb?
Do monkey’s ears protrude from their head?
Do robins lay eggs? (no imagery needed)

4. When do we use imagery? when the following three conditions are met:
The information is a subtle visual property
The property has no verbal label (has not been considered before)
The property cannot be inferred from other information
We also use imagery to anticipate trajectories
Would the bilard go into the socket?
These two functions (memory recognition, visual tracking) are also the primary functions of visual perception

5. Sub-components of Visual Imagery
Generating
Scanning (inspection)
Retaining
Mental Rotation (transformation)
Zoom (transformation)

6. Generate Imagine an elephant skating
This occurs in a piecemeal fashion:
First, an ‘image frame’ is generated (global shape)
Next, parts and details are added to the the global shape, based on needs and imager preference
Example: Imagine a letter F

7. Mental Scanning Map is displayed Subjects memorize it
Map is removed (imagery)
Task: “Imagine black speck moving from straw hut to well. Press the key when you arrive”
Independent variable: The distance
Dependent variable: Response Time
Main Finding: Distance scanned in a mental image is directly related to the time it takes to complete the scan
Scan time was directly related to the distance between the objects at the ends of the scan path…suggesting a constant rate of scanning. The image of the subjects really appears to preserve the spatial lay-out of the map itself…
If people were consulting a propositional representation of this image, there is no clear reason why it would take longer to scan “longer” distances in the map
So these data provide some support for the pictoral or analog characteristics of mental images.
One potential alternative explanation is that subjects were just responding with the timing that they thought experimenters would prefer. In other words, subjects might interpret the scanning instructions in such a way that they just waited longer for longer distances even though they could have responded earlier. These kinds of experimenter demand effects have been demonstrated before, and they might emerge even if the experimenter isn’t intentionlly trying to communicate their ideas of what “should” happen. Some work has demonstrated this effect even in Kosslyn’s experiments, BUT
Kosslyn and colleagues tested this possibility by running a similar experiment with a research assistant who was misled about the actual hypothesis. They were told that shorter distance would take longer to scan…but results were the same...

8. Image Retention
Note that in visual perception the very rapid decay is a virtue (but it is a burden for imagery)

9. Mental Rotation Rotation Are they the same shape? 2-D yes
Stimulus: A pair of 3-D objects
Task: Are the two shapes the same but viewed from a different perspective? Or are they different?
Independent variables:
Rotation type (in 2-D, in depth)
Same/different
Angle of rotation (0 --> 180º)
Dependent variable: Response Time
Main Findings:
Imagined movement resembles actual movement (linear relation between RT and angle of rotation)
This is true for both plane and depth rotations
depth
yes
depth no
A is rotated in picture plane (2D transform of same outline)
B&C are depth plane rotated…people have to represent the shapes 3 dimensionally to rotate them...
Shepard’s experiments showed that the spatial characteristics of 3D manipulations of images were similar to 2D manipulations. The amount of rotation was still strongly predictive of response time…
We should bear in mind the question: why would a propositional representation show these timing characteristics?

10. Mental Rotation: What is the nature of mental imagery?
Hypothesis:
- ‘Moving’ mental images is like moving real objects.
Assumptions:
Obviously, real objects move in a continuous fashion
to judge whether the letters are normal or mirror reversed requires rotating image to “upright” positions
Prediction:
Measurements of reaction time are one of the key methods that cognitive psychologists use
draw inferences about mental processes. But you may wonder…what good does it really do
to know how fast someone can carry out a cognitive task? The key here is the step of relating
changes in reaction time with changes in the type of mental processing that are required.
The relationship between reaction time and the particular mental steps that are required
can lead to some compelling conclusions….here’s an example.
You probably all have the strong intuition that you can view “mental images”
(e.g. a horse running across a field, a blue square floating in front of you….).
But what about when you have need to transform the mental image before you?
(to rotate it, or scan across it).
Does that involve the same kinds of spatial rules that govern rotation and scanning of real world objects?
Shepard was interested in whether the manipulation of mental images was in some way
modeled after the movements of objects in the real world.
This need not be the case…it is logically possible that the needed rotations could
take place “instantly”. There is no logical need for intervening angles to be
represented during this task. He manipulated the angle that the letters would have
to be rotated in order to return them to upright position, and measured the subjects’
reaction times to judge whether each letter was mirror reversed or not…
He used the objective measure of reaction time to make inferences
about the unobserved process of mental rotation...
Fast RT
Slower RT
Slowest RT
RT: fast slow

11. Results
Shepard concluded that mental rotation involved the continuous transformation of the image…much like the rotation of objects in the real world. Notice the monotonic relationship between RT and angle...
Here is an example where a compelling hypothesis about an unobserved mental event can be drawn on the basis of experimental data.
The greater the angle the letter had to be rotated to get back to
upright…the longer it took subjects to judge whether they were
mirror reversed or not.

12. Zoom-in Instructions: “Imagine a mouse next to an elephant…”
“Do the mouse’s whiskers point up or down?
Inspection of whiskers requires zooming-in
Such zoom-in process is time consuming (slow RT)
Kosslyn and colleagues devised a clever way to control “image size” in their subjects. People were asked to image objects next to either a large object or a small object. Under the assumption that subjects have a limited amount of imagery space…the latter condition would allow for larger objects
When subjects were asked to consult their image to determine, for instance, that a mouse has whiskers, it took longer with the “smaller” image…
Kosslyn interpreted this as “zoom time”
Here, the “zoom time” phenomenon really seems in contradiction to a propositional representation, where one might argue that all the facts might be available about mice whiskers no matter what the size was…(remember the example from Reisberg text: when people asked to think about image of cat, they verified the presence of a head more quickly than the presence of claws, but when asked to think about the “concept” of a cat (with no reference to visual image), then “claws” was a more readily available feature of the cat.
Another thing you might notice with the larger image is that this picture of a mouse that I dug up is actually suspiciously similar to a rat…

13. Zoom-in (cont’d) Instructions: “Imagine a mouse next to a paper clip…”
“Do the mouse’s whiskers point up or down?
Inspection of whiskers does NOT require zooming-in
Faster RT than when paired to the elephant

14. Overlap of Imagery and Perception
Hypothesis:
Imagery uses the same machinery as perception
Evidence
Behavioral (interference)
Neuroimaging
Brain Lesion

15. Behavioral Evidence (‘inspection’ interference)
Primary Task:
Detect a weak perceptual signal
Secondary Task:
Imagery
Independent variable:
Modality across tasks
Same (high interference):
visual-visual,
auditory-auditory
Different (low interference):
visual-auditory,
auditory-visual
Dependent variable: Perceptual Sensitivity (hit - FA)
Finding: There is interference
Thought question:Predict result when perceptual and imaged stimuli are the same
Remember our discussion of task specific resources? This experiment uses the same logic…if imagery and perception use overlapping brain machinery, then they should also call upon similar “resources”. Thus, we would expect an imagery task to interfere with a visual perception task.
But doing two tasks at once can be harder even if they don’t share task specific resources (remember that there do also seem to be task general resources). So we include conditions in which the specific perceptual systems used DO NOT overlap with the imagery processes.
So the acuity of visual perception is directly affected by the presence of a visual image….and same for the sensitivity of auditory detection…
another clear prediction of the imagery/perception overlap hypothesis is that similar brain regions should be active during imagery and perception…
known effect: perception of larger objects leads to activation in more anterior regions of primary visual cortex than perception of smaller ones…
will imagery show the same characteristics?

16. Neuroimaging Evidence: Similar Brain Areas Activated by Imagery and Perception
These red dots show the overlapping brain regions that were activated by imagery and perception in another study…
other brain evidence: people with damage to primary visual cortex (occipital) have shown associated deficits with formation of visual images
There is both behavioral and neuroanatomical evidence suggesting a strong overlap between the processes of imagery and visual perception.

17. Neuroimaging Evidence: Topographic
correspondence between perception and Imagery
Background:
- Primary visual cortex has a retinotopic map, that is,
- peripheral visual field is represented in anterior part of BA17
- fovea is represented in posterior part of BA17
Task: Letter Imagery
Independent Variable: Size of image
Result: Image of small letters activates more posterior parts of primary visual cortex than image large letters.
But you might notice that there are other differences between these images that might not have been predicted...

18. Neuropsychological Evidence
Subject: Patient with occipital lesion and reduced visual field (vs. Normal controls)
Instructions:
“Imagine an elephant, start walking toward it until it covers all visual field. How far away is it?”
Normal
Visual field
Reduced
Visual field
Kosslyn and colleagues devised a clever way to control “image size” in their subjects. People were asked to image objects next to either a large object or a small object. Under the assumption that subjects have a limited amount of imagery space…the latter condition would allow for larger objects
When subjects were asked to consult their image to determine, for instance, that a mouse has whiskers, it took longer with the “smaller” image…
Kosslyn interpreted this as “zoom time”
Here, the “zoom time” phenomenon really seems in contradiction to a propositional representation, where one might argue that all the facts might be available about mice whiskers no matter what the size was…(remember the example from Reisberg text: when people asked to think about image of cat, they verified the presence of a head more quickly than the presence of claws, but when asked to think about the “concept” of a cat (with no reference to visual image), then “claws” was a more readily available feature of the cat.
Another thing you might notice with the larger image is that this picture of a mouse that I dug up is actually suspiciously similar to a rat…

19. Neuropsychological Evidence
Kosslyn and colleagues devised a clever way to control “image size” in their subjects. People were asked to image objects next to either a large object or a small object. Under the assumption that subjects have a limited amount of imagery space…the latter condition would allow for larger objects
When subjects were asked to consult their image to determine, for instance, that a mouse has whiskers, it took longer with the “smaller” image…
Kosslyn interpreted this as “zoom time”
Here, the “zoom time” phenomenon really seems in contradiction to a propositional representation, where one might argue that all the facts might be available about mice whiskers no matter what the size was…(remember the example from Reisberg text: when people asked to think about image of cat, they verified the presence of a head more quickly than the presence of claws, but when asked to think about the “concept” of a cat (with no reference to visual image), then “claws” was a more readily available feature of the cat.
Another thing you might notice with the larger image is that this picture of a mouse that I dug up is actually suspiciously similar to a rat…

20. Visual vs. Spatial Imagery
Visuo-spatial imagery can be divided into:
Visual Imagery proper
Unavailable to blind people
Possibly related to introspective ‘vividness’ of image
Disrupted by secondary visual task (but not by spatial task)
Spatial Imagery
Evident in blind people
Unrelated to ‘vividness’ of image
Disrupted by non-visual spatial task
Probably more dependent on parietal cortex
Individual differences data argue for independence among subcomponents (also between visual and spatial aspects)

21. Imagery: Conclusions Mental imagery can be studied in the lab
Visual Imagery is not a monolithic concept.
Instead, many sub-components exist (rotate, zoom, scan, generate)
Mental images are a little bit like pictures (analogical)
Imagery seems to rely in perceptual neural substrates
Next, we will consider evidence that tests a stronger hypothesis about the pictorial aspects of mental images: Does mental imagery use the same brain structures as visual perception? (alternatively, it could be that mental images just function similarly to visual perception, while being carried out through independent mechanisms)