PSY 369: Psycholinguistics Language Comprehension: The role of memory

Memory and comprehension Many sources of information may be used in processing sentences Syntactic structure Word frequency Plausibility Discourse context Intonational information The use of these information sources may be constrained by the amount of working memory resources that are available

Center embedded structures The house burned down. The house the handyman painted burned down. The house the handyman the teacher hired painted burned down.

Center embedded structures The house burned down.

Center embedded structures The house burned down. The house the handyman painted burned down.

Center embedded structures The house burned down. The house the handyman painted burned down. This one may be legal, but that doesn’t mean that it is (easily) comprehensible ( the handyman that the teacher hired painted the house that burned down ) The house the handyman the teacher hired painted burned down.

Memory and comprehension The man that the woman that the child hugged kissed laughed. Most readers having trouble figuring out who did what to whom (called thematic role assignment). Easier to assign thematic roles in the two sentences that form it: The man that the woman kissed laughed. The woman that the child hugged kissed the man. The trouble: Insufficient working memory resources to retain the intermediate products of computation made building the complex syntactic structure

Daneman and Carpenter (1980) Technique: This technique involves presenting sequences of 2 to 6 sentences, each of 12 to 17 words. The participant has to read the sentences out loud, and attempt to remember the last word of each. Then asked to recall as many last words as possible (in any order). Measuring memory span

When at last his eyes opened, there was no gleam of triumph, no shade of anger. Measuring memory span

The taxi turned up Michigan Avenue where they had a clear view of the lake. Measuring memory span

Recall the last words Measuring memory span When at last his eyes opened, there was no gleam of triumph, no shade of anger. The taxi turned up Michigan Avenue where they had a clear view of the lake.

I turned my memories over at random like pictures in a photograph album. Measuring memory span

I will not shock my readers by describing the cold-blooded butchery that followed. Measuring memory span

He had an odd elongated skull which sat on his shoulder like a pear on a dish. Measuring memory span

You can check out the books that you need for this course at the local library. Measuring memory span

The radio station was promoting the concert with free tickets and back stage passes. Measuring memory span

The professor could be seen on weekends in the backyard garden pulling out weeds. Measuring memory span

Recall the last words Measuring memory span I turned my memories over at random like pictures in a photograph album. I will not shock my readers by describing the cold-blooded butchery that followed. He had an odd elongated skull which sat on his shoulder like a pear on a dish. You can check out the books that you need for this course at the local library. The radio station was promoting the concert with free tickets and back stage passes. The professor could be seen on weekends in the backyard garden pulling out weeds. Ok for two sentences; Hard at 3 sentences; Very hard for 4 or more. Used to classify readers as high and low span

Memory and online comprehension The Capacity Theory of Comprehension (Just & Carpenter, 1992) Proposed that individual differences in working memory capacity should influence how readers comprehend sentences

Memory and online comprehension The Capacity Theory of Comprehension (Just & Carpenter, 1992) Studied garden path sentences The animacy of the first noun may constrain the possible interpretation of the sentence Semantically Unconstrained: The defendant examined by the lawyer shocked the jury. The defendant that was examined by the lawyer shocked the jury. Semantically Constrained (so should be faster if animacy can be used) The evidence examined by the lawyer shocked the jury. The evidence that was examined by the lawyer shocked the jury. “that was” disambiguates these sentences

Memory and online comprehension Just the ambiguous sentences The defendant examined by the lawyer shocked the jury. The evidence examined by the lawyer shocked the jury. High span readers could use the semantic information to resolve the ambiguity

Memory and online comprehension King and Just (1991) Verbs which could provide strong pragmatic cues as to which of the two potential actors in the sentence was the agent: The robber that the fireman rescued stole the jewelry. Embedded clause Two possible agents: the robber the fireman Two verbs, which is the main verb of the sentence?: rescued stole

Memory and online comprehension King and Just (1991) Verbs which could provide strong pragmatic cues as to which of the two potential actors in the sentence was the agent: The robber that the fireman rescued stole the jewelry.. The robber that the fireman rescued watched the program. The robber that the fireman detested stole the jewelry. The robber that the fireman detested watched the program. Can bias which Noun goes with which Verb pragmatically (or not)

Memory and online comprehension King and Just (1991) Verbs which could provide strong pragmatic cues as to which of the two potential actors in the sentence was the agent: The robber that the fireman rescued stole the jewelry. The robber that the fireman rescued watched the program. The robber that the fireman detested stole the jewelry. The robber that the fireman detested watched the program. Results High-capacity subjects did not improve Low-capacity subjects did Main verbembedded verb

Memory and online comprehension Garnsey, Pearlmutter, Pirog (2003) The professor (who was) confronted by the student was not ready for an argument. The professor (had) confronted the student but was not ready for an argument. Question: Do readers differ specifically in how quickly they can use disambiguating words to rule out incorrect alternatives?

Memory and online comprehension - By whether preview of “by” while still fixating on verb likely Eye fixations were analyzed separately The professor confronted by the student was not ready to … Garnsey, Pearlmutter, Pirog (2003) If last fix was here, trial not used If last fix was here, trial coded as Preview Unlikely If last fix was here, trial coded as Preview Likely

Memory and online comprehension Readers who score high on the Reading Span test - Make better use of a peripherally visible disambiguating word to quickly rule out a preferred but incorrect interpretation

Memory and online comprehension What information is used to resolve syntactic ambiguities depends on individuals working memory capacity (but see Walters and Caplan (1996) for alternative view) Just & Carpenter (1992) - high span readers used semantic information early, but low span readers didn’t King & Just (1991) - high span readers did not use pragmatic information to resolve ambiguity, but low span readers did Garnsey, Pearlmutter, Pirog (2003) - span differences may also depend on where the eye lands (which determines what kind of preview readers get)

Memory and comprehension What about memory for language over the longer term? What do we remember about sentences?

Memory for sentences Fillenbaum (1966) Given: The window is not closed Tested: The window is not closed The window is closed The window is not open The window is open <-- surface similar, meaning different <-- surface different, surface different Most common error Meaning gets preserved, surface structure (and syntax) forgotten

Memory for sentences Sachs (1967, 1974) Heard (read): He sent a letter about it to Galileo, the great Italian scientist.” Tested: Same: He sent a letter about it to Galileo, the great Italian scientist. Act/Pass: A letter about it was sent to Galileo, the great Italian scientist. Formal: He sent Galileo, the great Italian scientist, a letter about it. Meaning: Galileo, the great Italian scientist, sent him a letter about it. Measured accuracy of detecting changes

Memory for sentences Meaning gets preserved, surface structure (and syntax) forgotten

Just good enough representations Ferreira and colleagues (Christianson et al 2001) Garden-path sentence While Anna dressed the baby played in the crib While Anna dressed, the baby played in the crib 100% correct 40% correct Comprehenders don’t always get all of the meaning right, but get enough to get by Did the baby play in the crib? Did Anna dress the baby?

Propositions A mouse bit a cat bit (mouse, cat) Good memory for meaning but not for form How do we represent sentence meaning? Propositions Two or more concepts with a relationship between them

Propositions A mouse bit a cat bit (mouse, cat) Good memory for meaning but not for form How do we represent sentence meaning? Propositions Two or more concepts with a relationship between them Can represent this within a network framework

Meaning as Propositions Propositions A set of conceptual nodes connected by labeled pathways that expresses the meaning of a sentence A mouse bit a cat or A cat was bitten by a mouse mouse bit cat agent patient relation

Deriving Propositions More complex example: Children who are slow eat bread that is cold Slow children Children eat bread Bread is cold relation subject time relation subject SlowChildren PastEat Cold Bread

Evidence for Propositions Memory better for sentences with fewer propositions “The horse stumbled and broke a leg” horse stumbled horse broke leg Three propositions Two propositions “The crowded passengers squirmed uncomfortably” passengers crowded passengers squirmed passengers uncomfortable

Evidence for Propositions Bransford & Franks, 1971 Constructed four-fact sentences, and broke them down into smaller sentences: 4 - The ants in the kitchen ate the sweet jelly that was on the table. 3 - The ants in the kitchen ate the sweet jelly 2 - The ants in the kitchen ate the jelly. 1 - The jelly was sweet.

Evidence for Propositions Bransford & Franks, 1971 Study: Heard 1-, 2-, and 3-fact sentences only Test: Heard 1-, 2-, 3-, 4-fact sentences (most of which were never presented)

Evidence for Propositions Bransford & Franks, 1971 Results: the more facts in the sentences, the more likely Ss would judge them as “old” and with higher confidence Even if they hadn’t actually seen the sentence Constructive Model: we integrate info from individual sentences in order to construct larger ideas emphasizes the active nature of our cognitive processes

Priming Propositions Ratcliff and McKoon (1978) Involves two propositions: P1 [OVERLOOK, MAUSOLEUM, SQUARE] P2 [ENSHRINE, MAUSOLEUM, TSAR]. “ The mausoleum that enshrined the tsar overlooked the square.”

Priming Propositions ConditionExamplesRT to TargetPriming Effects Across sentences Between two propositions in the same sentence Within a single proposition square- clutch square-Tsar square- mausoleum 671 msec 571 msec 551 msec None; baseline 100 msec facilitation 120 msed facilitation Ratcliff and McKoon (1978) Results in a cued memory task (how long does it take to verify “square” was in the sentence:

Alternative Representations Propositions are symbolic Problems: The referential problem The implementation problem The lack of scientific productivity The lack of a biological foundation Alternative Embodied representations (e.g., Barsalou; 1999; Glenberg, 1999)

Embodiment in language Embodied representations Perceptual and motor systems play a central role in language production and comprehension Theoretical proposals Linguistics: Lakoff, Langacker, Talmy Neuroscience: Damasio, Edelman Cognitive psychology: Barsalou, Gibbs, Glenberg, MacWhinney Computer science: Steels, Feldman

Embodiment in language Embodied representations Perceptual and motor systems play a central role in language production and comprehension Words and sentences are usually grounded to perceptual, motoric, and emotional experiences. In absence of inmediate sensory-motor referents, words and sentences refer to mental models or simulations of experience.

Embodiment in language Embodied representations Brain activity Comprehension and images Concrete words Action words activate motor representations

We understand utterances by mentally simulating their content. Simulation exploits some of the same neural structures activated during performance, perception, imagining, memory… Language gives us enough information to simulate Simulation hypothesis

Inference in comprehension Not all propositions come from the bottom-up Elaboration - integration of new information with information from long term memory Memory for the new information improves as it is integrated Inferences - a proposition (or other representation) drawn by the comprehender From LTM, not directly from the input

We draw inferences in the course of understanding new events. The inferences get encoded into our memory of the events. e.g., drawing inferences of instruments Bransford, and colleagues (1972, 73) Inference in comprehension

John was trying to fix the birdhouse. He was looking for the nail when his father came out to watch him and to help him do the work. Bransford, and colleagues (1972, 73) John was using the hammer to fix the birdhouse when his father came out to watch him and to help him do the work. Inference in comprehension

John was trying to fix the birdhouse. He was looking for the nail when his father came out to watch him and to help him do the work. Bransford, and colleagues (1972, 73) John was using the hammer to fix the birdhouse when his father came out to watch him and to help him do the work. was not mentioned in the text, but was inferred Inference in comprehension

What does language do? “ Harry walked to the cafe.” “Harry walked into the cafe.” A sentence can evoke an imagined scene and resulting inferences : CAFE –Goal of action = inside cafe –Source = outside cafe –cafe = containing location –Goal of action = at cafe –Source = away from cafe –cafe = point-like location

Embodied inferences WALL Bonk!! The scientist walked into the wall. The hobo drifted into the house. The smoke drifted into the house.

Summing up The results of sentence comprehension are meaning representations Some debate over what these representations are Whatever they are, they get integrated with existing knowledge from LTM

Discourse Psycholinguistics Traditional Psycholinguistics Determining what happens when we understand sentences Broader View How we resolve/understand sentences against the current discourse representation Sentence comprehension is a process that anchors the interpretation of the sentence to the representation of the prior text

Processing Discourse What is discourse? Units of analysis larger than a sentence Applies to both spoken and written forms Ways we process (i.e., comprehend and remember) units of language larger than a sentence lectures personal narratives expository discourse

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

To whom does “him” refer to? Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

To whom does “him” refer?Bach Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

To whom does this “him” refer? Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

To whom does this “him” refer?Bach again Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

To whom does this “him” refer?Bach again Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” Why not Abe?

Huh!? Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

Huh!? Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” Oh yeah, they’re time travelers.

Characteristics of Discourse Local Structure (microstructure): The relationship between individual sentences Global Structure (macrostructure): The relationship between the sentences and our knowledge of the world

Characteristics of Discourse Local Structure: Coherence: Does the passage make sense Logical consistency and semantic continuity Cohesion: Does the discourse “stick together” Interpretation of one sentence depends on other sentences

Characteristics of Discourse Coherence: does it make sense? Incoherence When the meanings of individual sentences do not hang together Given/new distinction Readers expect speakers to cues as to what information is old (already known by the listener) and what is new (not known)

Developing coherence Haviland and Clark (1974) Process of understanding a sentence in discourse context involves 3 stages: identifying the given and new info in the current sentence finding an antecedent in memory for the given information attaching the new information to this spot in memory

Developing coherence Haviland and Clark (1974) Herb unpacked some beer. The beer was warm.

Developing coherence Haviland and Clark (1974) Herb unpacked some beer. The beer was warm. Definite article “the” signals that “the beer” is given information

Developing coherence Haviland and Clark (1974) Herb unpacked some beer. The beer was warm. Definite article “the” signals that “the beer” is given information Connect the new information “was warm” to the appropriate discourse concept

Developing coherence Haviland and Clark (1974) Herb unpacked some beer. The beer was warm. Definite article “the” signals that “the beer” is given information Connect the new information “was warm” to the appropriate discourse concept “some beer” This process is called Direct Matching

Developing coherence Herb unpacked some picnic supplies. The beer was warm. Definite article “the” signals that “the beer” is given information So connect the new information “was warm” to the appropriate discourse concept Need a bridging inference to connect “the warm beer” to “some picnic supplies” Haviland and Clark (1974)

“some beer” Direct Matching Bridging Inference “the beer” “some picnic supplies”“the beer” World knowledge Developing coherence

“Murray poured water on the fire.” “The fire went out.” Singer, Halldorson, Lear, & Andrusiak (1992) “Murray drank a glass of water.” “He watched the fire go out.” T/F “water extinguishes fire” Faster Requires inference No Required inference Suggests that the inference was made Developing coherence

Characteristics of Discourse Cohesion: Interpretation of one sentence depends on other sentences Referential Cohesion “Dude, you should hear him play…” Substitution Cohesion “We’ve got to get these dudes back to …” And many more See pg 155 of textbook for table of other categories of cohesion The relationship between the referring expression and the antecedent create referential cohesion of discourse

Types of Referential Cohesion Anaphoric Reference Using an expression to refer back to something previously mentioned in discourse “…Bach was in the music store …” “Dude, you should hear him play, he rocks.” Cataphoric Reference Using an expression to refer forward to something that is coming up in discourse Dude, did you find him?” “Yeah, Bach was in the music store...”

Reading Span Test Smaller reading spans = smaller working memory capacity Comprehending Anaphoric References Daneman and Carpenter (1980) Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest “Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the music. I was especially perturbed by the expression on my best friend’s face. Wayne looked intense and was pounding the table furiously to the beat. Now, I like most of the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim jeans, fancy T-shirs and sneakers. It is not that I dislike rock music but I think it is supposed to be fun and not taken too seriously. And here he was, “all shook up” and serious over the crazy music. Comprehension task Reading a passage and answer questions about the referents of pronouns Question: Who was “all shook up” and serious over the music?

Reading Span Test Smaller reading spans = smaller working memory capacity Comprehending Anaphoric References Daneman and Carpenter (1980) Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest “Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the music. I was especially perturbed by the expression on my best friend’s face. Wayne looked intense and was pounding the table furiously to the beat. Now, I like most of the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim jeans, fancy T-shirs and sneakers. It is not that I dislike rock music but I think it is supposed to be fun and not taken too seriously. And here he was, “all shook up” and serious over the crazy music. Comprehension task Reading a passage and answer questions about the referents of pronouns Question: Who was “all shook up” and serious over the music?

Reading Span Test Smaller reading spans = smaller working memory capacity Comprehending Anaphoric References Daneman and Carpenter (1980) Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest “Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the music. I was especially perturbed by the expression on my best friend’s face. Wayne looked intense and was pounding the table furiously to the beat. Now, I like most of the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim jeans, fancy T-shirs and sneakers. It is not that I dislike rock music but I think it is supposed to be fun and not taken too seriously. And here he was, “all shook up” and serious over the crazy music. Comprehension task Reading a passage and answer questions about the referents of pronouns Question: Who was “all shook up” and serious over the music?

Manipulated how many sentences intervened between the pronoun and the antecedent Comprehending Anaphoric References Daneman and Carpenter (1980)