2. Language Use and Understanding BCS 261 LIN 241 PSY 261 CLASS 6: EFFECTS OF DISFLUENCY ON REFERENCE COMPREHENSION

3. What are the cognitive processes of reference comprehension? …the apple……thee uh apple…

4. Everyone is disfluent About 6% of speech is disfluent Um, uh Um, uh Repeats, repairs Repeats, repairs “thee” for “the” “thee” for “the” Pauses Pauses Elongations Elongations Pitch contours Pitch contours ok, give the gray squirrel a blue umbrella ok, take, ok, give, give the bl- the gray squirrel a blue umbrella

5. Most comprehension research ignores disfluency Laboratory speech Laboratory speech Sanitized Sanitized Scripted Scripted Fluent Fluent ok, give the gray squirrel a blue umbrella Disfluency not “core” language Disfluency not “core” language Discussion Question: Why not? (MR) Discussion Question: Why not? (MR) At best: irrelevant At best: irrelevant At worst: a source of processing difficulty At worst: a source of processing difficulty

6. Why study disfluency? Disfluency is ubiquitous Disfluency is ubiquitous Disfluency is systematic Disfluency is systematic It therefore provides a unique window onto language processing It therefore provides a unique window onto language processing Does expectancy affect reference comprehension? Does expectancy affect reference comprehension?

7. How does disfluency affect language comprehension?

8. Me: … the battery’s really low. Can you hand me thee um - Me: … the battery’s really low. Can you hand me thee um - Dana: yeah Dana: yeah

9. Language is ambiguous Point to the camel… with a red bow. Point to the camel… with a red bow. Point to the camel. Point to the camel. Point to that animal. Point to that animal. Point to it. Point to it.

10. Can listeners use expectancy to facilitate language comprehension, given temporary indeterminacies? Can listeners use expectancy to facilitate language comprehension, given temporary indeterminacies? How do listeners integrate information from various sources? How do listeners integrate information from various sources? Lexical information Lexical information Discourse context Discourse context

11. Many information sources can facilitate reference resolution Each piece of information can be Each piece of information can be PARTIAL PARTIAL PROBABILISTIC PROBABILISTIC How might this make the comprehension system more efficient? How might this make the comprehension system more efficient?

12. Disfluency is systematic Disfluency indicates production difficulty Disfluency indicates production difficulty “thee” likely to be followed by: “thee” likely to be followed by: Pause Pause Repeat Repeat Um / Uh Um / Uh Theee, um,... (Clark & Wasow, 1998, Fox Tree & Clark, 1997)

13. Given/New fundamental for language GIVEN: “…. the ball. Give me the ball.” GIVEN: “…. the ball. Give me the ball.” NEW: “…. the ball. Give me the bat.” NEW: “…. the ball. Give me the bat.” Language structure codifies information structure Language structure codifies information structure Given before new in production (e.g., Arnold, Wasow, Losongco, & Ginstrom, 2000) Given before new in production (e.g., Arnold, Wasow, Losongco, & Ginstrom, 2000) Given more accessible in comprehension Given more accessible in comprehension Pronouns (e.g., he, she, it) Pronouns (e.g., he, she, it) Definite noun phrases (e.g., the ball, the bat) Definite noun phrases (e.g., the ball, the bat) (e.g., Chafe, 1976; Clark & Sengul, 1979; Dahan et al., 2002)

14. Does disfluent speech make new objects more expected? GIVEN: “…. the ball. Give me the ball.” GIVEN: “…. the ball. Give me the ball.” NEW: “…. the ball. Give me thee uh bat.” NEW: “…. the ball. Give me thee uh bat.” (Arnold & Tanenhaus, to appear; data from Arnold, Wasow, Losongco & Ginstrom, 2000) Disfluency more likely for NPs with new referents

15. Discussion Q Is there a possibility that, as adults, we are naturally predisposed toward disfluent speech in order to allow our children the opportunity to realize that we are providing new information (prime them, such as was mentioned in the article), and to give them a small amount of extra time to construct a memory for the newly acquired information? (Jessee Blake)

16. EXPERIMENT 1 Does disfluency lead comprehenders to expect reference to a new object? “Put the grapes below the camel. Now put the-” Off-line task: What is the speaker likely to mention next?

17. Context: Put the grapes below the camel. Short instructionLong instruction Fluent Now put --Now put the-- Disfluent Now put --Now put thee uh --

18. Context: Put the grapes below the camel. Short instructionLong instruction Fluent Now put --Now put the-- Disfluent Now put --Now put thee uh --

19. Exp. 1 Results Disfluency makes reference to new objects more expected Disfluency makes reference to new objects more expected Does disfluency also affect the on-line processes of reference comprehension? Does disfluency also affect the on-line processes of reference comprehension?

20. EXPERIMENT 2 Does disfluency create an on-line bias toward new objects? Put the grapes below the camel. Now put theee, uh candle above the salt shaker.

21. Target = camel Competitor = candle salt shaker grapes Unrelated = salt shaker grapes Percentage of Looks over Time Time Percentage of looks Click on the camel. 200 ms 1234512345 Trials Time

22. Given vs. New cohorts (Dahan, et al. 2002) Put the grapes above the camel. Put the grapes above the camel. Now put the CA…. Now put the CA…. NEW target GIVEN target

23. Fluent- given bias Put the grapes above the camel. Put the grapes above the camel. Now put the CA…. Now put the CA…. (Dahan, et al., 2002) Disfluent- new bias? Put the grapes above the camel. Put the grapes above the camel. Now put thee, uh CA…. Now put thee, uh CA….

24. Comprehension Study Methods Instructions recorded by experimenter Instructions recorded by experimenter Subjects told they were recorded by another subject Subjects told they were recorded by another subject Disfluencies sounded natural Disfluencies sounded natural Lexical retrieval was a plausible source of disfluency Lexical retrieval was a plausible source of disfluency

25. Experiment 2 design & predictions FLUENT / GIVEN Put the grapes above the camel. Now put the CAMEL... FLUENT / NEW Put the grapes above the candle. Now put the CAMEL... DISFLUENT / GIVEN Put the grapes above the camel. Now put thee uh CAMEL... DISFLUENT / NEW Put the grapes above the candle. Now put thee uh CAMEL...

26. FLUENT / GIVEN Put the grapes above the camel. Now put the CAMEL... FLUENT / NEW Put the grapes above the candle. Now put the CAMEL... DISFLUENT / GIVEN Put the grapes above the camel. Now put thee uh CAMEL... DISFLUENT / NEW Put the grapes above the candle. Now put thee uh CAMEL... Experiment 2 design & predictions

27. FLUENT / GIVEN Put the grapes above the camel. Now put the CAMEL... FLUENT / NEW Put the grapes above the candle. Now put the CAMEL... DISFLUENT / GIVEN Put the grapes above the camel. Now put thee uh CAMEL... DISFLUENT / NEW Put the grapes above the candle. Now put thee uh CAMEL... Experiment 2 design & predictions

28. FLUENT / GIVEN Put the grapes above the camel. Now put the CAMEL... FLUENT / NEW Put the grapes above the candle. Now put the CAMEL... DISFLUENT / GIVEN Put the grapes above the camel. Now put thee uh CAMEL... DISFLUENT / NEW Put the grapes above the candle. Now put thee uh CAMEL... Experiment 2 design & predictions

29. camel… candle… camel… candle… Context: “Put the grapes above the camel.” Given cohort New cohort unrelated

30. Context: “Put the grapes above the camel.” Given cohort New cohort unrelated

31. Discussion Question Is it not possible that the disfluencies gave the participants more time to think and therefore let their eyes wander to the new objects whereas in the fluent case they had no time for such meandering? (Nicole Dobrowolski) Is the difference in difference between discourse-old and discourse-new stimuli in this experiment basically related to the idea of frequency in lexical items? (i.e. candle 'appears' more in common English than camel) (Anthony Shook)

32. Exp. 2 conclusions Disfluency affects reference comprehension Disfluency affects reference comprehension 200 msec after onset of target 200 msec after onset of target Combines with lexical information Combines with lexical information Disfluency eliminates/reverses given bias Disfluency eliminates/reverses given bias

33. Why does disfluency create a bias toward new objects? New objects are harder to refer to New objects are harder to refer to Theee, um,... Aha! This speaker is having trouble!

34. Does disfluency create biases to other types of referents? Known objects Novel objects

35. EXPERIMENT 3 Does disfluency cause a novel bias on-line? Click on the red … Click on thee uh red…

36. EXPERIMENT 3 DESIGN FLUENT: Click on the red … FLUENT: Click on the red … KNOWN… ice cream cone. KNOWN… ice cream cone. NOVEL… funny squiggly shape… NOVEL… funny squiggly shape… DISFLUENT: Click on thee uh red … DISFLUENT: Click on thee uh red … KNOWN… ice cream cone. KNOWN… ice cream cone. NOVEL… funny squiggly shape… NOVEL… funny squiggly shape…

37. known looks novel looks unrelated looks

38. Exp. 3: Competitor looks 200-600 msec after onset of color word

39. Exps. 2-3 summary Fluency information affects on-line reference comprehension Fluency information affects on-line reference comprehension Disfluency introduces biases Disfluency introduces biases NEW objects NEW objects NOVEL objects NOVEL objects

40. Discussion Questions Do these findings possibly happen because of our ability to use top-down processing and our previous experience with everyday speech through speaking and listening? (Jessica DeSisto)

41. What would happen if the disfluency occurred within a different part of speech, such as the noun or a verb in a sentence? Are these effects of disfluency on language comprehension and the expectancy hypothesis only observed when a definite article is disfluent? (Beth Riiina) What would be the effect on resolution of longer disfluencies, according the the results of this paper? (e.g. my friend uses "whats its face" whenever searching for a word). Would the paper even have anything to say about it, or does the sheer length of certain phrases influence the resolution too much? (Anthony Shook)

42. I realize that the article summarized the finding that disfluent speech leads us to new information, rather than given-is it that the disfluent speech allows for a moment to search your memory for the information that is being presented, and if you do not have a memory for it, you then build one? Or does the subject, when presented with disfluent speech, automatically look to the new information without hesitation? (Jessee Blake)

43. Does anyone feel that these findings could be useful in programming a computer to comprehend everyday speech? Would these findings aide in our own understanding of language comprehension, increasing our ability to program a computer to understand human speech errors? (Jessica DeSisto)