1. Visuospatial Representation
Spatial Knowledge, Imagery, Visual Memory

2. Representation What is a representation?
Four aspects of representation
The represented world
The representing world
Set of informational relations on how the two correspond
Set of processes that extract and use information from the representation

3. Meaning Mental representations are carriers of meaning
In order to interact appropriately with the environment we represent info from it and manipulate those representations
Correspondence
Meaning derived from how representation stands in consistent relation to the represented world
Conceptual
Meaning determined by relations to other representations

4. Spatial Knowledge How we represent and use spatial information
Separate from strictly verbal knowledge
Semantic propositions
Dependent on the linear dimension of space.

5. Spatial Cognition
How is the representing world like the represented world?
The represented world is a space
The representing world is a space
What kinds of processes might be involved?

6. Space as a representation
Spatial representation
Representing world is a space. What is a space?
Geometric entity in which locations are specified relative to a set of axes
Dimensionality defined by the number of axes that can point in independent directions
Of interest is the distance between items, which can be measured in different ways
Euclidian
Straight line
Non-independent dimensions
Saturation and brightness
City-block
Distinct dimensions
Color and size

7. Space as a representation
Physical world experienced (at least perceptually) has three dimensions (+ time)
However, the representing world is not confined to any number of dimensions
Represented world does not need to be spatial
Conceptual info can be represented spatially
More on that later

8. Spatial Representation
Analog representation
Representation mimics the structure of the represented world
Multidimensional scaling
Propositional
Abstract assertions regarding the state of the represented world
Not tied to a particular sensory modality

9. Multidimensional Scaling (MDS)
Mathematical technique for taking a set of distances and finding the best-fitting spatial configuration that corresponds to those distances
Input: a distance or proximity matrix that describes how close every object in a set is to every other object
N objects are represented by N(N-1)/2 numbers (distances)
Output: a geometric representation where every object is represented as a point in D-dimensional space
Each object is represented as a point in space
N objects are represented by ND numbers (coordinates)
Purposes of MDS
Give psychological interpretations to the dimensions
Reveal the dimensionality of a data set

10. MDS
Difficult to get a sense of relative distance by means of this information

11. MDS
MDS recovers absolute original locations for the objects from the distances
Flipping on horizontal axis would give us a rough approximation of NSEW
Analog representation

12. Propositional Representation
(A,B) 10 miles east
(E,C) 20 miles south, 10 miles east
(F,D) 10 miles south, 10 miles west

13. Analog vs. Propositional
Good for configural info
Easy incorporation of new info
Propositional
Time-consuming
Lots of info must be represented
E.g. one point added may require many propositions
Allows for communication of spatial knowledge and incorporation of additional information not related to distance
Going south on I35, one must pass through Denton to get to either Fort Worth or Dallas

14. Cognitive Maps Where is Seattle? Where is Terrill Hall?
Large vs. small-scale space
Hierarchical representation

15. Small vs. Large-scale space
Maps of small-scale (navigable space)
Cognitive geography
Maps of large-scale space
What is our sense of the locations of items in the world?

16. Small scale space Survey knowledge Route knowledge
Bird’s eye view (map knowledge)
Good for global spatial relations
Easy acquisition
Not so great for orientation
Route knowledge
Gained from navigating through the environment
Locate landmarks and routes within a general frame of reference
Landmark knowledge
Salient points of reference in the environment
More difficult to acquire but better for navigation in irregular environments
May lead to survey knowledge
Perhaps a different type
Cognitive collage vs. orientation free

17. Large scale space Which is farther north:
Denton, TX or Chicago, IL?
Portland, OR or Portland, ME?
Hierarchical representation of locations

18. Hierarchical representations
Relative locations of smaller regions are determined with respect to larger regions.
States are superodinate to cities, countries superordinate to states
USA is south of Canada
Maine is just south of Canada
Oregon is well south of Canada
Oregon must be south of Maine
Cities in Oregon must be south of cities in Maine
In this case such cognitive economy works against us
Portland OR is north of Portland ME

19. Hierarchical representations
Judge relative position of cities (Stevens and Coupe)
When superordinate info congruent with question, performance better
Is x north of y when one of right side maps presented

20. Hierarchical coding Huttenlocher & Hedges
Category-adjustment model
Combine info across hierarchical levels
If info at subordinate is known with near certainty, there is no appeal to categorical info
If info at subordinate (fine-grained) levels is at all uncertain, people use categorical info in estimation
Bias toward center of category
Bayesian approach utilizing prior knowledge
Gist: errors in estimation are due to categorization rather than nonmetric spatial relations

21. How are maps learned From descriptions From navigation
Taylor & Tversky: People learned maps from survey and route descriptions
From navigation
People can assess distance and direction traveled
Integration of information
Visual information
Vestibular information
Maps formed from video games are less accurate than maps in which people really move
Rotation is particularly important

22. Using spatial cognition
Adaptive context
Locating and way finding
Tool Use
Mental rotation vs. mental movement
Symbolic representations of space
Drawings, maps, models
Language
Thinking

23. Adaptive context Locating and way finding Consider
Hatchling sea turtles finding the sea
Salmon finding way back home
However these are more behavioral instinct and imprinting than pure navigation
Desert ant finding direct route home after meandering paths in featureless environment
Marsh tit stores seeds in holes in a hundred various places for later retrieval

24. Locating and way finding
Ego-centered systems
Environment-centered systems
Hierarchical coding

25. Locating and way finding
Ego-centered system
Location of objects coded relative to self
Updated as we move through the world
Nonconscious
Rieser, Guth, Hill (1986)
Participants asked to point out previously learned locations in unfamiliar room after blindfolded and led along path
Did not matter whether previously told which location they’d be asked about, suggesting attentional focus did not assist in the process
Problem: may not always be accurate over larger distances without detailed environmental information

26. Locating and way finding
Environment-centered system
Object location coded in relation to stable features of the environment
Requires feature-rich environment providing info to dominate sense used by organism
If conditions met, then superior to ego-centered
Allows for rechecking of position (no drift from accumulation of small errors)
Works better for retaining info over long periods of time

27. Cognitive maps
Both humans and animals display errors in judgment that cast doubt in positing a true ‘cognitive map’
Animal studies suggest approximation of distance from a single landmark
Humans make many errors in spatial judgments that suggest no real metric representation
Distance from A to B judged different from B to A
Though again this sort of distortion may be related to categorization (hierarchy)

28. Tool use
Making, using and designing tools for interaction with the environment involves cognitive processes such as mental rotation and imagery for success

29. Shephard & Metzler (1971)

30. Mental rotation vs. Mental Movement
Logically equivalent
However evidence suggests that mental rotation and perspective-taking/mental movement are psychologically distinct
Selection task
Which these arrays/models would be the correct view from over there?
Item question
What object would be nearest to you if you were over there?
Specify frame of reference of relative to the observer

31. Mental rotation vs. Mental Movement
Selection task
Piaget
Kids (< 10) not so hot at such a task
Usually pick egocentric view
Huttenlocher & Presson
They do much better when asked to do mentally rotate
Can also physically move to new location that matches a particular array
Suggests conflict between current physically present perspective and the new (imagined) one they are trying to obtain
MR allows them to stay put in the physically present room
Physical movement physically transforms that perspective

32. Mental rotation vs. Mental Movement
Item questions
If kids do not move item questions help (even as young as 3)
Again, this helps them maintain that egocentric perspective
If asked to mentally rotate, item questions can actually hurt performance compared to selection tasks
It may be that in item questions, whole array must be rotated to determine object relations vs a simple ‘rotation’ of the person or single object in selection task
Gist: mental rotation and mental movement can be differentially affected depending on the nature of the question asked, suggesting there may be different underlying processes involved

33. Drawings maps and models
Spatial learning from maps differs from learning by means of navigation
Map learning may aid configural knowledge and allow for better estimates of distance between points while navigational learning allows for better route distance estimation and location of unseen points
Recall survey vs. route knowledge
Orientation-specific vs. orientation-free learning
Studies show evidence that navigational learning is more a collection of multiple views than orientation-free, though may lead to a sort of orientation-free type of knowledge
Sholl & Friedman

34. Spatial Language Contrasting experience with communication
Experience spatial relations continuously, but language is usually discrete (e.g. near vs. far)
Spatial terms function much like other categories (e.g. fuzzy boundaries, prototypes)
Experience multiple spatial relations simultaneously, but speak of one relation at a time
A frame of reference must be agreed upon in order to communicate spatial relations

35. Spatial Language
Despite the difficulties in communicating spatial knowledge, ambiguities are generally overcome and information encoded (survey, route knowledge)
However it does seem that spatial language may bias or constrain spatial representation, and may even affect the development of spatial concepts and categories
Even so, the actual link between spatial language and spatial representation is not entirely clear
Impaired sight individuals may have difficulties with a variety of spatial tasks but have intact spatial language

36. Thinking
Spatial cognition also contributes to logical reasoning, metaphor, and creativity
Transitive reasoning
A > B, B > C
A ? C
Metaphor
The future stretched in front of them
My heart is a flame turned upside down
Structural alignment of spatial and temporal concepts
Diagrams as aids to understanding
Show conceptual similarity of items, connections amongst various concepts etc.
Creativity
E.g. visualization for problem solving
Taking someone else’s point of view?

37. Imagery Some information in memory is purely verbal
Who wrote the Gettysburg address?
Other memories seem to involve mental images
Trying to recall a procedure
Making novel comparisons of visual items
What is a mental image?
How are mental images represented and processed?
Are mental images like visual images?

38. Evidence for use of visual imagery
Selective interference
Segal & Fusella
Imagery interferes with detection of stimuli (sensitivity decreased)
Auditory imagery interfered with auditory detection, visual imagery with visual stimuli
Manipulation of images
Mental rotation studies

39. Evidence for use of visual imagery
Kosslyn
Learn a map
Mentally travel from one point to another
Measure time to make this mental trip
Results
Time to make trip increases with distance
Times increase with imagined size of the map.

40. Evidence for use of visual imagery
Moyer 1973
Subjects were given the names of two common animals and asked to judge which was larger
Which is larger, a moose or a roach?
Wolf or Lion?
The time delays as a function of size difference were similar to those usually found for perceptual judgments.

41. Kosslyn RT True False Kosslyn 1975
Scenario I: Imagine an elephant standing next to a rabbit. Does a rabbit have a beak?
Scenario II: Imagine a fly standing next to a rabbit. Does a rabbit have an eyebrow?
People made faster judgments when relying on a larger mental image (such as the rabbit next to the fly) than when using a smaller mental image (such as the rabbit next to an elephant)
Kosslyn suggested that the size of an image is an important factor in determining how fast we can make judgments about it. RT
Elephant
Fly
True
False
Inconsistent
Consistent

42. Paivio's Dual-Coding Theory
Information is mentally represented either in a verbal system (propositional) or a nonverbal (analogical) system (or both).
Each system contains different kinds of information.
Each concept is connected to other related concepts in the same system and the other system.
Activating any one concept also leads to activation of closely related concepts.

43. Paivio
The hypothesis of multiple codes (verbal and spatial) is based on the demonstration of independence of effects.
Pictures of objects
Words of objects

44. Paivio (1975) compared reaction times for consistent and inconsistent visual stimuli
If the stimuli are processed semantically, there should be no difference between consistent and inconsistent presentations.
If stimuli are processed spatially, inconsistent stimuli should require a mental conversion to appropriate size.
Which takes time

45. Consistent

46. Inconsistent

47. Results
“Which is larger?” RT
Inconsistent
Consistent

48. Paivio
Consistent
RABBIT
FLY

49. Paivio
Inconsistent
RABBIT
ELEPHANT

50. Paivio RT Congruity Effect only for Pictures (not words)
Picture
Words
Inconsistent
Consistent
Congruity Effect only for Pictures (not words)
Imagery relies on perceptual detail and semantic does not
Such findings as this and picture superiority effect (pictures are better recognized than words), and that verbal + imagery encoding leads to best recall, suggest a Dual Code Theory

51. Santa 1977 More evidence of dual coding
Ss presented array of objects or words
On test presentation asked whether the elements were same as studied
E.g. In geometric condition first two would be yes responses

52. Santa 1977 Results of positive responses
Spatial configuration is preserved in geometric encoding
Compared to verbal presentation, which was encoding in typical English reading style and benefited from the linear configuration

53. Representation of images
What is the relationship between imagery and perception?
Can imagining interfere or facilitate detection of stimuli?
Similar processes involved?

54. Spatial Knowledge Symbolic Distance Effect (Moyer 1973)
Process of imagery = process of perception
As perceptual distance increases so does psychological distance (RT).
Items “farther apart” are more quickly distinguished
Which is larger?
Rabbit-Elephant
Rabbit-Dog

55. Representation of images
Contrary evidence
Chambers and Reisberg
Images are (committed to) a particular interpretation
E.g. The rabbit comes once drawn but was only a duck as imaged
Contrast with perception which requires interpretation, images are already interpretations

56. Are visual images visual?
Plenty of evidence to suggest a spatial component to visual imagery, but perhaps the visual part is represented propositionally
Kerr
Congenitally blind also take longer to imagine longer map routes like the one in Kosslyn

57. Are visual images visual?
Images are also not as sharp as real pictures
Form a mental image of a tiger
Does it have stripes?
How many?
It is hard to examine details of mental images that would require eye movements

58. Making new pictures Finke, Pinker, Farah Example
Imagine a capital letter H and a triangle
Rotate the H 90 degrees
Place the triangle on top of it
What is it?
Suggests images can take on new interpretations

59. Are visual images visual?
Facilitation and interference (Farah)
Have people imagine a letter (H or T)
Present one of the letters to the screen briefly (20 ms), or present nothing, followed by mask
Asked if they saw a letter
People are more likely to detect the stimulus if it was the same as what they were imaging, suggesting that visual and imaginal representations joined or fed into the same process

60. Are visual images visual?
Evidence from neuroscience
Patients with lesions of visual cortex that lead to perceptual problems also have problems with mental imagery
ERP evidence PET evidence: Visual imagery leads to activation of visual cortex. Auditory imagery does not
In general, results of studies from mental rotation to brain imaging support the idea of both visual and spatial representation of images

61. Translating Words to images
Franklin and Tversky
Create a mental image based on the description
Asked to identify location of items in that imagined environment
Results are what one might expect given an imagined spatial environment
Up-down, front-back more relevant in navigating real world
Left-right confusion in real world and imagined world

62. Visual memory
Although our visual memory seems to be excellent, it turns out not to be that great in many respects
In general, our memory for details is lost, much like with other types of memory

63. Visual memory Memory for pictures is quite good generally
Again, don’t get too detailed
Standing
Presented photos over several days
Old-New memory over 80%
Picture superiority effect
Better memory for pictures than words