Language Perception Eva M. Fernández Queens College & Graduate Center CUNY ABRALIN22-FEB-05

2 Language Is… SIGNALMEANING grammar & lexicon PERCEPTION PRODUCTION logic knowledge about the real world

3 Production SIGNALMEANING Lexical Retrieval Structural Assignment Phonological Encoding I’ll give you my undevoted attention! You’ll earn her eternal grapefruit. This restaurant hasn’t been awake very long. Put the oven on at a very low speed. We have a lot of churches in our minister. They roasted a cook. If you give the nipple an infant… You ordered up ending. phonological fool a glear plue sky spattergrain

4 Perception SIGNALMEANING Structure Building Lexical Access Phonological Decoding Lexical Retrieval Structural Assignment Phonological Encoding

5 Not Present in (Speech) Signal: phonemes word boundaries clause boundaries location of empty categories intended attachments for locally or globally ambiguous strings hidden intents of the speaker! SO HOW COME WE’RE SO GOOD AT DECODING?

Visual Illusions when the experiences people report don’t correspond to physical properties of the stimulus very cool… but also very informative about the way the visual / perception system works (which is: modularly)

7 The Hermann Grid Illusion How many grey dots do you see at the “cross-roads”? Source: A great page to visit for many more visual illusions.

8 A Face Can’t Be Hollow! A face is always perceived as convex… not concave. Max Planck Institut für Biologische Kybernetik

Perceptual Illusions also very cool… and also very informative about the way the language perception system works  its modularity ensures its speed and accuracy, which are both in turn compromised when the signal is AMBIGUOUS

10 McGurk Effect by Arnt Maasø, of the University of Oslo:

11 Perceptual Displacement and Phonemic Restoration “The state governors met with their respective legislatures convening in the capital city.”

12 “legislatures”, with “cough!” (~ 145 msec) spliced in

13 “legislatures”, intact --- [s] ~ 145 msec

14 Bottom  ~ Top  study with Broca’s patients (Pollack & Picket, 1964) The apple the boy is eating is red. The girl the boy is chasing is tall. bait, date, gate study (Garnes & Bond, 1976) Here’s the fishing gear and the ___. Check the time and the ___. Paint the fence and the ___.

15 Structure Building: The Parser its input is a string of lexical items its job is to build syntactic structure its output is sent to a mechanism that decodes meaning it probably has limited access to information that’s not in the grammar or in the lexicon it probably operates following a very small set of strategies, grounded on limitations imposed by working memory

RSVP Paradigm Center-screen, word-by-word display Timing: N ms per word (here: N = 500 ms) Sentence-recall task

17  Thebeautifulblackcatchasedthecolorfulball. The beautiful black cat chased the colorful ball.

18  Blackcolorfultheballchasedcatbeautifulthe. Black colorful the ball chased cat beautiful the.

19 The Garden Path Sentence The soldiers marched into the desert surprised the Persian forces. Since Joel always jogs a mile seems like a short distance to him. Carmela put the candy on the table in her mouth. Konstantin understood the problem had no solution. Everybody at the party knew Ann’s date was a total fool. Local ambiguity Disambiguation downstream, which goes against parser’s preferences Reanalysis… or meltdown!

20 The Garden Path Theory Lyn Frazier & Janet Fodor, late 1970s Minimal Attachment: build the simplest tree Late Closure: attach locally Minimal Chain Principle / Active Filler Strategy: posit shortest possible chain / posit gaps for fillers ASAP (the parser is lazy)

21 Minimal Attachment The soldiers marched into the desert surprised the Persian forces. Since Joel always jogs a mile seems like a short distance to him. Carmela put the candy on the table in her mouth. Konstantin understood the problem had no solution. Everybody at the party knew Ann’s date was a total fool. building complex structure = processing cost

22 Late Closure John said Mary will arrive last night. Physicists are thrilled to explain what they are doing to people. Under the glistening tree there was a gift for a boy in a box. Professor Humperdinck artfully avoided looking at the exams of the students that were sitting in his office ungraded. Two sisters reunited after 18 years in check-out counter! attaching non-locally = processing cost

23 Mary saw a gift for a boy… NP PP P for NP a boy NP a gift Mary saw a gift for a boy in a box. PP in a box attaching non-locally = processing cost Late Closure

24 Late Closure John said Mary will arrive last night. Physicists are thrilled to explain what they are doing to people. Under the glistening tree there was a gift for a boy in a box. Professor Humperdinck artfully avoided looking at the exams of the students that were sitting in his office ungraded. Two sisters reunited after 18 years in check-out counter! attaching non-locally = processing cost

25 The RC Attachment Ambiguity N1 The plot concerns the guardian of the prince who was exiled from the country for decades RC La trama es sobre el guardián del príncipe que fue exiliado del país por décadas N2

26 Cross-Linguistic Differences N1 attachment rates (%), in studies using questionnaire instruments where: RC was long N1/N2 were equal in animacy Complex NP was in canonical object position for the language SP EN (US) 40 (UK) 4748 Cuetos &Mitchell,1988 Bradley etal., 2003 Carreiras,1992 Ehrlich et al.,1999 Fernández,2000/2003 Hemforth etal., submittd

27 Cross-Linguistic Differences As in previous table, for languages other than English & Spanish, listed (for lack of a better strategy!) in alphabetical order: SP-like !! (Fra) 60 (Can) EN-like (FS) AFRIKAANS ARABIC BULGARIAN CROATIAN DUTCH FRENCH GERMANHEBREW ITALIAN JAPANESE NORWEGIAN PORT. (BRAZIL) PORT. (EUROPE) ROMANIAN RUSSIAN SWEDISH

28 Cross-Linguistic Differences … could be driven by … genetic relationship? syntactic properties? existence of unambiguous alternatives? distribution of unambiguous strings in input? prosody? PARSER PROSODY (phonology) PRAGMATICS

29 Pragmatics? Grice’s Cooperative Principle “Make your conversational contribution such as is required, at the stage at which it occurs, by the accepted purpose or direction of the talk exchange in which you are engaged” The plot concerns the guardian of the prince who was exiled the prince’s guardian who was exiled * the prince’s guardians who was exiled

30 Long RCs are Informationally Heavy The plot concerns the guardian of the prince … who was exiled. … who was exiled from the country for decades. Long RC has more lexical content, so it’s more informative. Does informativeness influence attachment? RC length effect, confirmed in: English, Spanish; Arabic, Bulgarian, Croatian, French, German, Hebrew, Hindi, Portuguese… N1 INTERP MORE LIKELY

31 Long RCs are Prosodically Heavy The plot concerns the guardian of the prince who was exiled. The novel’s plot concerns the guardian of the prince who was exiled. The plot concerns the guardian of the prince who was exiled from the country for decades. The novel’s plot concerns the guardian of the prince who was exiled from the country for decades.

32 Elicited Production N = 8 native US English speakers — 5F, 3M N = 6  4 sentences, RC Length  Matrix-Subject Weight RC = 1 versus 3 prosodic words …who was exiled ( from the country for decades ) MX = 1 versus 2 prosodic words The ( unusual ) plot… Fernández, Bradley & Taylor, in prep

33 The unusual plot concerns the guardian of the prince. The prince was exiled from the country for decades. 1

34 Acoustic Analysis: Regions The ( unusual ) plotconcernsthe guardian of the prince who was exiled ( from the country for decades ) WtSVN1N2RC1 RC3 Duration: Uniform acoustic signature of phrasal break Fernández, Bradley & Taylor, in prep

35 Acoustic Analysis: Regions S ] [ V V ] [ N1N1 ] [ N2N2 ] [ RC Fernández, Bradley & Taylor, in prep

36 N2] [RC Fernández, Bradley & Taylor, in prep

37 Region = N2 Fernández, Bradley & Taylor, in prep

38 Elicited Production: Summary MX:F 1 (1,7) = 2.80, p=.138 RC:F 1 (1,7) = 11.46, p<.02 F 2 (1,5) = 2.07, p=.209 F 2 (1,5) = 9.96, p<.05 Interaction MX x RC: F 1.25 … N2 ] [ RC — and nowhere else Likelihood of break grades with RC length and matrix weight, additively, i.e., with sentence length Fernández, Bradley & Taylor, in prep

39 Questionnaire Procedure “Reading comprehension test” 36 targets, 108 fillers (1:3 ratio) Comprehension question after each sentence Example of target The plot concerns the guardian of the prince who was exiled from the country for decades. Who was exiled?the guardian the prince Example of filler The sneaky burglars took all the stereo equipment but overlooked the computer system. What was stolen? the stereo the computer Fernández, Bradley & Taylor, in prep

40 Questionnaire Participants N = 44, Queens College students US English speakers Language-history questionnaire, non-native speakers excluded/replaced Rejected/replaced for errors > 15% in fillers Fernández, Bradley & Taylor, in prep

41 Questionnaire Results Relative Clause Length F 1 (1,40) = 24.95, p<.001 F 2 (1,32) = 30.12, p<.001 Matrix Subject Weight F 1 (1,40) = 5.51, p<.05 F 2 (1,32) = 9.43, p<.01 Interaction F 1 < 1 F 2 < 1 Fernández, Bradley & Taylor, in prep

42 The Implicit Prosody Hypothesis (IPH) “In silent reading,a default prosodic contour is projected onto the stimulus,and it may influence syntactic ambiguity resolution” (Fodor 1998, 2002) the brother of the bridegroom who snores the brother of the bridegroom ][ who snores

43 Prosody and Syntax Align the brother of the bridegroom ][ who often unknowingly snores the brother of the bridegroom who snores NP N1PP NPP RCN2 NP N1PP NPP RC N2 el hermano del novio ][ que a menudo inconscientemente roncaba el hermano del novio ][ que roncaba prosodic discontinuity syntactic discontinuity Selkirk, 1986

44 Empirical Support for the IPH Behavioral evidence on how RCs are interpreted during silent reading existing dataset: Hemforth et al. (submitted) Evidence on how the N-of-N-RC construction is produced in discourse-neutral speech elicited production experiment  Do the patterns in the two datasets match up?

45 Materials in English and Spanish: with short and long RCs N1-N2-RC placed post- and pre-verbally Behavioral Evidence The guest impressed X.X impressed the guest. El invitado impresionó a X.X impresionó al invitado. X =the brother of the bridegroom who (often unknowingly) snores el hermano del novio que (a menudo inconscientemente) roncaba Hemforth et al. (submitted)

46 Behavioral Evidence Post-Verbal Objects Pre-Verbal Subjects Who snores? The brother (N1) Post-Verbal Objects: Cross-linguistic difference RC length effect Pre-Verbal Subjects: RC length effect reduced Cross-linguistic difference reduced Hemforth et al. (submitted)

47 N2][RC RC.] N2][RC RC][V ENGLISH SPANISH The guest impressed the brother of the bridegroom who often unknowingly snores. El invitado impresionó al hermano del novio que a menudo inconscientemente roncaba. The brother of the bridegroom who often unknowingly snores impressed the guest. El hermano del novio que a menudo inconscientemente roncaba impresionó al invitado.

48 Experiment: Elicited Production Participants, N = 8 per language English  New York Spanish  Madrid Materials, N = 8  4 per language (selected from Hemforth et al.’s 32  4) Post- and pre-verbal of identical length RC’s right boundary with same lexical content, whether short or long The guest impressed X.X impressed the guest. X =the brother of the bridegroom who (often unknowingly) snores Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004

49 Analyses: N2 & RC’s Verb Duration: Presence of Boundary Pitch movement: Type of Boundary The guest impressed the brother of the bridegroom ][ who … snores.] N2][RCRC.] The brother of the bridegroom ][ who … snores ][ impressed N2][RCRC. ][V the guest. Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004

50 ENGLISHSPANISH Pre-Verbal Subjects Long RC Short RC 100 ms Monolinguals: N2 Durations Placement × Length Interaction F 1 (1,14) = 5.77, p <.05, F 2 (1,14) = 12.37, p < msPost-Verbal 68 msPre-Verbal Post-Verbal Objects RC-Length  =

51 Monolinguals: RC Vb Durations Placement × Length Interaction F 1 (1,14) = 6.38, p <.025; F 2 (1,14) = 5.90, p <.05 –10 msPost-Verbal 35 msPre-Verbal Long RC Short RC 100 ms Pre-Verbal Subjects Post-Verbal Objects RC-Length  = ENGLISHSPANISH

52 Monolinguals: N2 Pitch Placement × Language Interaction F 1 (1,14) = 16.56, p <.002, F 2 (1,14) = 14.43, p <.002  0.4 Hz/200 msEnglish 23.6 Hz/200 msSpanish Placement  = ENGLISHSPANISH Long RC Short RC Post Pre

53 Monolinguals: RCVb Pitch ENGLISHSPANISH Interaction: Placement × Language F 1 (1,14) = 6.05, <.05, F 2 (1,14) = 14.72, <.002  8.7 Hz/200 msEnglish  38.6 Hz/200 msSpanish Placement  = Long RC Short RC Post Pre

54 N2 ][ RC RC.] N2][RC RC ][ V ENGLISHSPANISH Pre-Verbal Subjects Post-Verbal Objects Duration & Pitch: Monolinguals Post-Verbal, Short Pre-Verbal, Short Pre-Verbal, Long Post-Verbal, Long Post-Verbal, Short Pre-Verbal, Short Pre-Verbal, Long Post-Verbal, Long

55 Summary of Data Outcomes Pitch Movements: Type of Boundary and Cross-Linguistic Differences Spanish: N2 falls pre-verbally, rises post-verbally English: N2 uniformly falls, pre- and post-verbally Duration: Presence of Boundary and Cross-Linguistic Similarities In both languages: Likelihood of breaks before RC is modulated by position Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004

56 Conclusions and Speculations Behavioral similarities and differences are indexed in the prosodic patterns of Spanish and English But what is the source for the contrasting sentence-medial tunes in Spanish? Are such patterns projected entirely within the syntax-prosody interface? Or are such patterns the result of an interplay of syntax, prosody, and information structure?