Perception-Based Knowledge Representations

Representing Information Once information enters sensory memory and pattern recognition occurs, it must be encoded in a more permanent manner in the cognitive system so further processing can occur. We will focus on perception-based representations at this time and examine how visual and verbal information are represented and acted upon in the absence of external perceptual stimuli. The manner in which the processing occurs depends on the way information is represented (e.g., visual detail, spatial location, meaning, etc.).

Dual-Code Theory Evidence to support the distinction: verbal information is remembered better if encoded verbally and visually (e.g., visual mnemonic) cortical activity corresponds differentially to verbal and visual tasks Paivio argues there are separate representations for verbal and visual information; a dual-code. J = mentally rehearse a “jingle.” R = rehearse finding way around neighborhood.

Santa’s Face Experiment triangle circle square triangle circle square triangle circle arrow triangle circle square triangle circle arrow Some information, like the geometric objects, are encoded according to spatial position while other information (e.g., words) are encoded linearly (serially).

Visual and Spatial Imagery Are “mental images” processed in the same way as viewing a real object or scene? Several lines of research have shed light on this question: – mental rotation – rotation of “R”s – paper folding – imagine scanning (“island map”) In each case, RT was a linear function of distance.

Visual and Spatial Imagery (con’t) The question still remains: are those images tied to the visual modality? A study in which subjects had to image a block letter “F” provides some insight:

Two Types of Imagery? The above studies suggest mental images are spatial analogs which are not tied to the visual modality. However, consider the following research in which subjects are required to process visual details of images: Moyer had subjects judge the relative size of pairs of animals (e.g., Which is larger, moose or roach?”). He found RT was a linear function of the difference between sizes… the larger the difference, the faster the RT. Thus, subjects are able to judge visual details which have no spatial information connected to them.

Are Visual Images Like Visual Perception? Given the foregoing evidence, psychologists ask to what degree are visual images similar to actually seeing an object? we can create images, transform images, and then recognize the newly formed image we can form and experience visual illusions through our images similar neural patterns are observed in the visual cortex when actually viewing patterns or imaging those patterns (e.g., in the temporal lobe, the fusiform area [for faces] and parahippocampal place area [for pictures of locations])

Are Visual Images Like Visual Perception? However, we can “reverse” and see the second figure in reversible figures (but it does appear to be more difficult, requiring some hints) relationships easily perceived in pictures are more difficult to recognize in images Example: Imagine a 2x1 inch rectangle. Draw a vertical line bisecting the rectangle to form two equal squares. Draw a diagonal line from the upper-left most corner to the lower-right most corner of the rectangle. Draw a diagonal line in the right square from the upper-right corner to the lower-left corner. What is the relative length of the line segment below this diagonal line to that of the line segment above it?

Are Visual Images Like Visual Perception? complex images tend to be organized into pieces or “chunks” preventing identification of hidden objects while visual perceptions would not, yet the objects do have a similar hierarchical structure as seen in visual perceptions (meaningful segments as seen in stabilized images) 1 in. 2 in. In general, it seems harder to process an image than an actual stimulus.

Cognitive Maps Our visual and spatial imagery are often tied together, such as when we consider our environment (e.g., the layout of the campus, a city, etc.). We begin to form cognitive maps by first learning specific routes between destinations. As we gain experience, we identify landmarks and develop relative positions of those landmarks until we acquire a reasonable “survey map” of our environment.

Cognitive Maps (con’t) Which is further east, San Diego, CA or Reno, NV? Which is further north, Seattle, WA or Montreal, Canada? Which is further west, the Atlantic or the Pacific entrance to the Panama Canal? Cognitive maps have a hierarchical organization as well (i.e., we form chunks of areas and sub-areas within those, etc.), which sometimes lead to distortions where a visual perception would not. For example:

Representation of Verbal Information – The Second of the “Dual” Codes Just as there is evidence for different representations of visual information (i.e., spatial and visual imagery), there is also similar evidence for representing different types of verbal information (e.g., representations of sounds and those of the printed word). There is also “serial order” (i.e., words, objects, etc. which appear in a sequence) which is amodal, similar to amodal spatial information. Thus, there is an analogy between the representation of verbal information and visual imagery: we have separate representations of amodal information and sensory property information.

Representation of Verbal Information Front anchoring - Ss have better access to the beginning of lists. Hierarchical nature of serial -order information -- break sequences into chucks (e.g., the alphabet)… recall starts at the beginning of a chunk and proceeds in a serial search until item is found. Evidence for amodal representation: likely to confuse similar sounds (e.g., mistake “D” as “T,” but not “L”)

Baddeley’s Theory of Verbal Working Memory When we “imagine” something, we are attempting to create a perceptual experience in the absence of any external stimuli (e.g., rotating a cube or “saying” the words in a list to be recalled). Baddeley has proposed that “visuospatial sketchpad” and “phonological loop” slave systems maintain visual and verbal information in working memory. A “central executive” directs the use of those systems (e.g., inputs and retrieves information from either, translates between system, etc). Phonological Loop Central Executive Visuospatial Sketchpad Phonological store our “inner ear” Articulatory Loop our “inner voice”