1. Word Recognition (Sereno, 4/04) How long does it take to recognise a visual word? –What is meant by “recognition” or “lexical access”? –Can lexical access be accurately measured? –What factors affect lexical access and when? The “magic moment” (Balota, 1990) of lexical access: “At this moment, presumably there is recognition that the stimulus is a word, and access of other information (such as the meaning of the word, its syntactic class, its sound, and its spelling) would be rapid if not immediate.” (Pollatsek & Rayner, 1990)

2. Background: Basic Units of Language A. ~5,000 languages phonemes  morphemes  sentences  conversations (sounds) & words B. Phonemes = elementary sounds of speech phonemes are not letters... to, too, two, through, threw, shoe, clue, view vowel & consonant phonemes 11-144 phonemes in any given language English has ~ 40; Hawaiian has ~16 combining phonemes is rule-governed

3. Wordness: For each row of 3 possible new words, which one will probably never make it : ( blicksplungerlight sbarmwumpleturl mancernserhtcrelurious intheriwhucrneen shacefringngout

4. Basic Units of Language C. Morphemes = smallest meaningful unit of lang. can be a word, word stem, or affix (prefix, suffix) word:help, love word stem:spir, ceive, duce prefix/suffix:re-, dis-, un- / -less, -ful, -er derivational & inflectional morphemes derivational – change the grammatical class V + -able = Adj (adorable, believable) V + -er= N (singer, runner) inflectional – grammatical markers V + -ed= past tense (walked) N + -s= plural (cows) “free” { “bound” {

5. Basic Units of Language C. Words Content vs. function (open- vs. closed-class) words Content words = carry the main meaning nouns, verbs, adjectives, adverbs Function words = grammatical words articles (a, the, this), conjunctions (and, but), prepositions (in, above) Psychological reality of the content-function word distinction in aphasia  selective impairment of content (Wernicke’s) or function words (Broca’s aphasia) Cattell (1886) & Stroop (1925)

6. Word superiority effect (Cattell, 1886) –Reicher (1969); Wheeler (1970) –tachistoscopic presentation –more accurate identification of the letter when stimulus is a word –pseudoword superiorty effect --- dkdk word  dkdk d 

7. RED BLUE BLACK GREEN RED GREEN BLACK BLUE RED BLACK BLUE BLACK BLUE GREEN BLUE GREEN RED GREEN NAME THE COLOUR OF THE INK

8. Basic Units of Language C. Words Ambiguity 1 word form, but 2 (or more) word meanings Ex: bank (N-N, “money” vs. “river”) watch(N-V, “clock” vs. “look”) bass (N-N, “guitar” vs. “fish”) 2 word forms, but 1 pronunciation Ex: sail/sale, right/write Generally unaware of ambiguity... even though it is quite pervasive even though it affects behaviour (RT, etc) homographs homophones

9. Basic Units of Language D. Sentences Syntax = the rule-governed system for grouping words together into phrases and sentences Sentences introduce a concept that they are about, the subject (or noun phrase), and then propose something about that concept, the predicate (or verb phrase). Ex:“The boy hit the ball.” doer act done-to (thematic roles) subject predicate

10. Basic Units of Language D. Sentences Same deep structure, different surface structure “The boy hit the ball.”(active) “The ball was hit by the ball.” (passive) Same surface structure, different deep structure [The French bottle] NP [smells.] VP [The French] NP [bottle smells.] VP THEY are boring. VISITING THEM is boring. cf. ambig. figures in perception: 1 form, 2 interpretations “The French bottle smells.” “Visiting relatives can be boring.”

11. Necker cube

13. New obesity study looks for larger test group Reagan wins on budget, but more lies ahead Man struck by lightening faces battery charge Enraged Cow Injures Farmer with Axe Milk Drinkers Are Turning to Powder Local High School Dropouts Cut in Half British Left Waffles on Falklands Dealers Will Hear Car Talk at Noon Miners Refuse to Work after Death Beating Witness Provides Names Squad Helps Dog Bite Victim Kids Make Nutritious Snacks Headlines

14. Stolen Painting Found by Tree Prostitutes Appeal to Pope Red Tape Holds up Bridge Deer Kill 17,000 Teenage Prostitution Problem is Mounting Child Stool Great for Use in Garden Shouting Match Ends Teacher’s Hearing Man Robs then Kills Himself Lung Cancer in Women Mushrooms Mondale’s Offensive Looks Hard to Beat Tuna Biting off Washington Coast Chinese Apeman Dated Headlines

15. Basic Units of Language D. Sentences Syntactic ambiguities “She hit the boy with the big stick.” “She hit the boy with the runny nose.” Interpretation depends on structural preferences (certain constructions used more often, favoured), as well as the prior discourse context.

16. Word Recognition (Sereno, 4/04) Measures Components Models Eye movements (EMs) Event-related potentials (ERPs) Word frequency & lexical ambiguity

17. Measures Standard behavioural techniques –lexical decision, naming, categorisation; also RSVP, self-paced reading –priming, masking, lateralised presentation –Donders (1868): subtractive method assumes strictly serial stages of processing additive vs. interactive effects –automaticvs. strategic (Posner & Snyder, 1975) unconscious exogenous bottom-up benefit controlled endogenous top-down cost & benefit

18. Measures Eye movements Neuroimaging –“Electrical”: EEG, MEG –“Blood flow”: PET, fMRI

19. MEASURE Normal reading TASK fixation duration (as well as location and sequence of EMs) TIME RES. GOOD POOR “blood flow” imaging: fMRI, PET “electrical” imaging: EEG, MEG various word tasks ms-by-ms seconds various word tasks naming categorisation lexical decision Standard word recognition paradigms (± priming, ± masking): RT ~500 ms ~600 ms ~800 ms ~250 ms

20. Components Orthography of language –English vs. Hebrew or Japanese Language skill –beginning (novice) vs. skilled (expert) reader –easy vs. difficult text

21. Components Intraword variables –word-initial bi/tri-gramsclown vs. dwarf –spelling-to-sound regularityhint vs. pint –neighborhood consistencymade vs. gave –morphemes prefix vs. pseudoprefixremind vs. relish compound vs. pseudocompoundcowboy vs. carpet

22. Components Word variables –word lengthduke vs. fisherman –word frequencystudent vs. steward –AoAdinosaur vs. university –ambiguitybank vs. edge, brim –syntactic classopen vs. closed; A,N,V –concretenesstree vs. idea –affective tonelove vs. farm vs. fire –etc.

23. Components Extraword variables –contextual predictability The person saw the... moustache. The barber trimmed the... –syntactic complexity Mary took the book.*Mary took the book was good. Mary knew the book. Mary knew the book was good. *Mary hoped the book. Mary hoped the book was good. –discourse factors (anaphora, elaborative inferences) He assaulted her with his weapon.... knife... stabbed

24. Models Dual-route account (Coltheart, 1978) Direct route (addressed) phonologysemanticsorthography Indirect route (assembled)

25. Models Interactive (Morton, 1969; Seidenberg & McClelland, 1989) /m A k/ phonology meaningorthography M A K E context

26. Models Modular (Forster, 1979; Fodor, 1983) decision output Lexical processor Syntactic processor Message processor General Problem Solver input features

27. Models Hybrid –2-stage: generate candidate set  selection –(Becker & Killion; Norris; Potter)

31. Word Recognition (Sereno, 4/04) Measures Components Models Eye movements (EMs) Event-related potentials (ERPs) Word frequency & lexical ambiguity

32. MEASURE Normal reading TASK fixation duration (as well as location and sequence of EMs) TIME RES. GOOD POOR “blood flow” imaging: fMRI, PET “electrical” imaging: EEG, MEG various word tasks ms-by-ms seconds various word tasks naming categorisation lexical decision Standard word recognition paradigms (± priming, ± masking): RT ~500 ms ~600 ms ~800 ms ~250 ms

33. Tools of choice: Recording eye movements in reading Recording ERPs in language tasks

34. Eye Movements (EMs) Best on-line measure of visual word recognition in the context of normal reading: Fast (avg fixation time ≈ 250 ms) Ecologically valid task Eye-mind span is tight

36. P1 N1 P300 N400 Number of trials 1 2 4 8 16 EEG ERP

37. ERPs Best real-time measure of brain activity associated with the perceptual and cognitive processing of words: Continuous ms-by-ms record of events Early, exogenous components (before 200 ms) should reflect lexical processing

38. (Sereno & Rayner, Trends in Cognitive Sciences, 2003)

39. DIVERSION High-density ERP Analysis: A case of “too many notes”?

40. High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component

42. High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component Hunt down where/when the effect is strongest and gather data from those electrodes/time window

44. High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component Hunt down where/when the effect is strongest and gather data from those electrodes/time window Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms)

47. High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component Hunt down where/when the effect is strongest and gather data from those electrodes/time window Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms) Spatial and/or temporal principal component analysis (PCA)

48. Scalp topography of the N1 @ 132-192 ms SF1 loadingsVoltages (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

49. Word frequency Word frequency effect represents the differential response to commonly used high- frequency (HF) words versus low-frequency (LF) words that occur much less often. Presence of word frequency effects is used as a marker of successful word recognition or lexical access.

50. 553 ms 490 ms 259 ms 275 ms 280 ms 293 ms (Sereno & Rayner, Trends in Cognitive Sciences, 2003)

51. Comparative EM & ERP studies Word frequency Word frequency X Context The sore on Tam-Tam’s was swollen. (LF) rump (HF) back (Neutral context) To our surprise we saw a (Biasing context) Flying to its nest was a hawk. (LF) (Neutral context) She looked over the (Biasing context) She read the new book. (HF) (Sereno, Rayner, & Posner, NeuroReport, 1998) (Sereno & Rayner, Perception & Psychophysics, 2000) (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

52. More comparative EM & ERP studies Word frequency X Orthographic regularity Lexical ambiguity X Context (Neutral context) James peered over at the (Biasing context) The mud was deep along the bank. Mike wasted the whole and then regretted it. ( LF-Reg) cask (LF-Exc) pint (HF-Reg) week (HF-Exc) hour (Sereno, Pacht, & Rayner, Psychological Science, 1992) (Sereno, JEP: Learning, Memory, and Cognition, 1995) (Sereno, Rayner, & Posner, NeuroReport, 1998) (Sereno & Rayner, Perception & Psychophysics, 2000) (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

53. (Sereno & Rayner, Trends in Cognitive Sciences, 2003)

54. Lexical Ambiguity Resolution Interactive position –Access is selective: Context guides access towards the appropriate sense of an ambiguous word; while both senses may be initially activated, only the contextually appropriate sense is fully accessed. Modular position –Access is exhaustive: All meanings of ambiguous words are automatically accessed; context cannot directly affect lexical processing, but instead operates on the output of the lexical processor to select the appropriate sense.

55. Ambiguity: Cross-modal priming Paradigm: Aud amb Visual Auditory context prime target “The building was infested with BUGS” ANT “and it…” SPY SEW Results: –In general, support the modularity of lexical processing. (Swinney, 1979)

56. Ambiguity: ERP unimodal priming Paradigm: –The only ERP ambiguity study employed a unimodal (visual) version of the cross-modal paradigm (Van Petten & Kutas, 1987). –Measured ERPs to targets that followed presentation of the ambiguous word prime. Results: –Support an interactive account of lexical processing.

57. Ambiguity: EM “fast priming” Paradigm: –Measured fixation duration on targets that followed presentation of the “fast” ambiguous word prime across various context conditions...

58. Ambiguity: EM “fast priming” Priming Effect ~30 ms n.s. (Sereno, JEP: LMC, 1995)

59. Ambiguity: EM “fast priming” Results: Support a modified interactive account of lexical processing - “reordered access” - in which both (1) meaning frequency (Dom vs. Sub) (2) prior context affect access speed. Specifically, (1) Alternative meanings become activated in order of their meaning frequency (2) Context can “boost” the activation of one of the meanings, possibly reordering access procedures

60. AmbiguousThe mud was deep along the bank... (subordinate) LFThe mud was deep along the brim... HFThe mud was deep along the edge... (Sereno, Pacht, & Rayner, Psych Sci, 1992) Fixation time:Amb = LF > HF “Spillover” time:Amb > LF > HF Support modified interactive account of access: “reordered access”. Paradigm: Results: Ambiguity: EM normal reading

61. Critique of methods: Cross-modal priming RT (lexical decision, naming) –Slow (500-900 ms) compared to speed of lexical access (~100-200 ms); susceptible to response bias. Secondary (indirect) measure –Effects of context on ambiguous word gauged by priming effects on target downstream.

62. Critique of methods: ERP unimodal priming Time-locked averages of the EEG –Ms-by-ms voltage fluctuations reflect processing in real time. Secondary (indirect) measure –Effects of context on ambiguous word gauged by priming effects on target downstream.

63. Critique of methods: EM “fast priming” Fixation time –Relatively fast (~375 ms) and on-line, but can reflect lexical and post-lexical integration effects. Secondary (indirect) measure –Although much quicker time course than cross- modal or ERP unimodal, effects of context on ambiguous word still gauged by priming effects on target downstream

64. Critique of methods: EM normal reading Fixation time –Fast (~250 ms) and on-line, but can reflect lexical and post-lexical integration effects. Primary (direct) measure –Effects of context on ambiguous word gauged by comparing its fixation time to control word.

65. ERP Ambiguity Experiment Biased ambiguous words were presented in neutral and biasing contexts in a word-by-word sentence reading paradigm. Biasing contexts always instantiated the subordinate sense. ERPs on the ambiguous words, themselves, were measured. ERPs to ambiguous words were then directly compared to ERPs to unambiguous control words. Control words - matched either to the dominant (HF) sense of the ambiguous word or to the contextually instantiated subordinate (LF) sense of the ambiguous word - were presented in neutral and biasing contexts. Comparisons across conditions were made at an early, lexical stage of processing (N1, 132-192 ms).

66. Example Stimuli

67. Scalp topography of the N1 @ 132-192 ms SF1 loadingsVoltages

68. Scalp topography of the N1 @ 132-192 ms SF1 loadingsVoltages ± 0.7 factor loading contours

69. Factor scores for SF1 Voltages (electrodes with SF1 loading > +0.7 or SF1 loading < -0.7) N1 @ 132-192 ms

71. (Sereno & Rayner, Trends in Cognitive Sciences, 2003)

72. Summary From the EM record, we can infer when lexical processing should occur (~100-200 ms). From the ERP record, we can see when certain differences first appear in real time. Effects of word frequency as well as context initially appear very early in the ERP record (N1 @ 132 ms post-stimulus). We can begin to establish a realistic time-line of word recognition in reading.

74. ERP Word Recognition Words Reg Exc LF cask pint HF time hour PseudoWords welf Consonant Strings fhvr LexicalityW vs PW vs CSP1 @ 100-132 ms FrequencyLF vs HFN1 @ 132-164 ms RegularityLF Reg vs LF ExcP2 @ 164-196 ms Stimuli Results (Sereno, Rayner, & Posner, NeuroReport, 1998)

75. CS-WPW-WCS-PWp < Lexicality Effects: P1 (100-132 ms) (Sereno, Rayner, & Posner, NeuroReport, 1998)

76. Frequency Effects: N1 (132-164 ms) LF-HF (Sereno, Rayner, & Posner, NeuroReport, 1998)

77. Regularity Effects: P2 (164-196 ms) LF Exc -LF Reg Ss with RT effectSs with no RT effect (Sereno, Rayner, & Posner, NeuroReport, 1998)

79. Scalp topography of the N1 @ 132-192 ms SF1 loadingsVoltages ± 0.7 factor loading contours (Sereno, Brewer, & O’Donnell, Psychologcial Science, 2003)

80. Emotion words Valence Arousal + ve – ve LoHi peacelove boredfire Neutral controls:hotel, farm

81. Questions of interest: How fast are words recognised? What factors affect lexical access? How early do these factors operate?

82. Models Interactive vs. Modular –Logogen (Morton, 1969) –PDP (Seidenberg & McClelland, 1989) Read-out –Search (Forster & Bendall, 19xx; ) Hybrid –2-stage: generate candidate set  selection –Becker & Killion, 19xx; Norris, 1984; Potter

83. Q: Why are camels called “ships of the desert?” A: Because they’re always filled with Arab sea-men. Q: Would you prefer roses on your piano or tulips on your organ? Q: What’s the difference between a rolling stone and a Scotsman? A: A Rolling Stone says “Hey you get off of my cloud!” and a Scotsman says “Hey McLeod get off my ewe!” Q: Why is men’s ‘sea-men’ white and their urine yellow? A: So they can tell if they’re coming or going. Jokes

84. Current Directions Emotion word processing Contextual constraint

85. ERP Ambiguity Experiment Design/Stimuli 6 experimental conditions: Word Type X Context LF Neutral HF Biasing Amb Word specifications: LF = 6 per million HF = 60 per million Amb = 63 per million (Dominant sense = 89 % Subordinate sense = 9%)

86. Spatial Principal Components Analysis Sample-by-sample voltage data at all 129 electrodes for all 14 Ss in all 6 conditions were submitted to a spatial PCA with a Quartimax rotation (cf. Spencer, Dien, & Donchin, 1998). The first Spatial Factor (SF1) accounted for a high degree of the variance (44%). 3(Word Type) x 2(Context) ANOVA was performed on the factor scores.