MEG, the Mental Lexicon and Morphology LP, Aug 03, Tateshina MEG, the Mental Lexicon and Morphology Liina Pylkkänen Department of Linguistics/ Center for Neuromagnetism New York University

MEG, the Mental Lexicon and Morphology LP, Aug 03, Tateshina MEG, the Mental Lexicon and Morphology Day 1 Lexical access 1: The M350 as an MEG index of lexical activation Day 2 Lexical access 2: The M350 and mechanisms of recognition Day 3 Morphology 1: The M350 as a tool for investigating similarity and identity Day 4 Morphology 2: Electrophysiological and behavioral evidence for early effects of morphology

LP, Aug 03, Tateshina Day 3 Morphology 1: The M350 as a tool for investigating similarity and identity Do effects of morphology reduce to combined effects of phonological and semantic similarity? Fiorentino & Poeppel: M350 evidence for decomposition in compounding. Stockall et al.: M350 evidence for decomposition in irregular inflectional morphology.

LP, Aug 03, Tateshina Day 3 Morphology 1: The M350 as a tool for investigating similarity and identity Do effects of morphology reduce to combined effects of phonological and semantic similarity? Fiorentino & Poeppel: M350 evidence for decomposition in compounding. Stockall et al.: M350 evidence for decomposition in irregular inflectional morphology.

Same vs. similar TEACHER vs. TEACH BROTHEL vs. BROTH SORCERY vs. MAGIC

Morphological decomposition TEACH ER vs. TEACH BROTHEL vs. BROTH SORCERY vs. MAGIC

Alternative(?): “emergent morphology” Morphology is similarity at the extreme. Effects of morphology should reduce to combined effects of semantic and phonological/formal similarity. (Seidenberg and Gonnerman, 2000) Gonnerman and Plaut (2000)

To test the theories: What are the effects of phonological and semantic similarity? Do effects of morphology reduce to combined effects of semantic and phonological similarity?

Dependent measures Behavioral lexical decision times to visually presented words Measurements of brain activity using MagnetoEncephaloGraphy (MEG) In particular the M350 -- an MEG index of automatic lexical activation

Magnetoencephalography (MEG) EEG http://www.ctf.com/Pages/page33.html

Magnetoencephalography (MEG) EEG http://www.ctf.com/Pages/page33.html

Magnetoencephalography (MEG) Distribution of magnetic field at 93 ms (auditory M100) Averaged epoch of activity in all sensors overlapping on each other. Outgoing Ingoing

Magnetoencephalography (MEG)

What happens in the brain when we read words? Letter string processing (Tarkiainen et al. 1999) Lexical activation (Pylkkänen et al. 2002) -100 0 100 200 300 400 500 600 700 [msec] 200 [fT] 150-200ms (M170) 200-300ms (M250) 300-400ms (M350) 400-500ms Pylkkänen and Marantz, Trends in Cognitive Sciences, in press.

Activation Selection Competition M350 (i) 1st component sensitive to lexical factors (such as lexical frequency) (ii) not affected by competition time level of activation resting level Stimulus: TURN TURN TURNIP TURF TURTLE Activation Selection Competition

M350 (i) 1st component sensitive to lexical factors (such as lexical frequency) (ii) not affected by competition (Pylkkänen, Stringfellow, Flagg, Marantz, Biomag2000 Proceedings, 2000) Repetition Frequency (Embick, Hackl, Shaeffer, Kelepir, Marantz, Cognitive Brain Research, 2001) (i) (ii) Stimuli that speed up early lexical processing but induce intense competition, delaying RT elicit faster, not slower M350’s. Methods: RMS

Effect of probability/density (n=10) * n.s. * (Pylkkänen, Stringfellow, Marantz, Brain and Language, 2002)

Activation Selection Competition M350 (i) 1st component sensitive to lexical factors (such as lexical frequency) (ii) not affected by competition time level of activation resting level Stimulus: TURN TURN TURNIP TURF TURTLE Activation Selection Competition

To test the theories: What are the effects of phonological and semantic similarity? Behaviorally? On the M350? Do effects of morphology reduce to combined effects of semantic and phonological similarity?

(non-onset-matching) Crossmodal priming (materials adapted from Gonnerman (1999)) SOA: Duration of prime Task Lexical decision 21 subjects Auditory prime Visual target Phonological (onset-matching) spinach spin (non-onset-matching) teacher reach Semantic (synonyms) idea notion Morphological teach (pseudo-affixed) corner corn (non-transparent) dresser dress

Phonological similarity single subject (Pylkkänen, Stringfellow & Marantz, submitted)

Semantic similarity Behaviorally facilitory NURSE primes DOCTOR Would the M350 show semantic priming?

Results M350 = First component affected by semantic relatedness n=21 single subject (RMS) (Pylkkänen, Stringfellow, Gonnerman, Marantz, in prep.)

Phonological and semantic relatedness affect the same component, the M350 Consistent with recent ERP results showing that phonological and semantic relatedness affect the same ERP component, the N400 (Radeau et al. 1998)

So far: Onset matching phonological similarity and semantic similarity have opposite effects: IDEA – NOTION Priming in M350 and RT. SPINACH – SPIN Inhibition in M350 and RT.

What about TEACHER-TEACH? Decomposition view: Relationship is one of identity. TEACHER contains TEACH Morphemes are emergent (e.g. Seidenberg and Gonnerman 2000): Relationship is one of similarity. TEACHER and TEACH are only semantically and phonological similar

What about TEACHER-TEACH? Decomposition view: Relationship is one of identity. Morphemes are emergent (e.g. Seidenberg and Gonnerman 2000): Relationship is one of similarity. M350 & RT should show repetition priming M350 & RT should show added effects of phonological and semantic similarity

Materials (crossmodal) (part of previous experiment) AUDITORY PRIME VISUAL TARGET RELATED teacher teach UNRELATED ocean teach

Results Repetition priming (Pylkkänen et al 2000)

Results: like repetition priming, not additive similarity effects

What about TEACHER-TEACH? Decomposition view: Relationship is one of identity. Morphemes are emergent (e.g. Seidenberg and Gonnerman 2000): Relationship is one of similarity. M350 & RT should show repetition priming M350 & RT should show added effects of phonological and semantic similarity

Possible objection: Phonological similarity is only inhibitory in the absence of semantic similarity. Prediction: ritzy – glitzy should prime very much like morphologically related pairs.

Behavioral data from Gonnerman (1999) EXPERIMENT 1 Prime-target example Priming Low sem, no morph: spinach-spin -19 Low sem: corner-corn -24 Mid sem: dresser-dress 19* High sem: teacher-teach 40* Hi sem, no phon: idea-notion 13* Morphological priming exceeds semantic priming EXPERIMENT 4 Prime-target example Priming Psychology undergrads High sem & phon: ritzy - glitzy -16 mid sem & phon: dismal-dismay -12 low sem & phon: rankle-rank 12 High sem, no phon: idea - notion 21* Hi sem, no phon: pumpkin-pump -19 semantic priming exceeds ritzy – glitzy priming (Gonnerman, 1999, PhD thesis, USC)

Behavioral data from Gonnerman (1999) EXPERIMENT 1 Prime-target example Priming Low sem, no morph: spinach-spin -19 Low sem: corner-corn -24 Mid sem: dresser-dress 19* High sem: teacher-teach 40* Hi sem, no phon: idea-notion 13* Morphological priming exceeds semantic priming EXPERIMENT 4 Prime-target example Priming Honors students High sem & phon: ritzy - glitzy 24* mid sem & phon: dismal-dismay -5 low sem & phon: rankle-rank 19 High sem, no phon: sorcery-magic 54* Hi sem, no phon: pumpkin-pump -39* semantic priming exceeds ritzy – glitzy priming (Gonnerman, 1999, PhD thesis, USC)

Possible objection: Phonological similarity is only inhibitory in the absence of semantic similarity. Makes the wrong predictions

What about TEACHER-TEACH? Decomposition view: Relationship is one of identity. Morphemes are emergent (e.g. Seidenberg and Gonnerman 2000): Relationship is one of similarity. M350 & RT should show repetition priming M350 & RT should show added effects of phonological and semantic similarity

(non-onset-matching) Crossmodal priming (materials adapted from Gonnerman (1999)) SOA: Duration of prime Task Lexical decision 21 subjects Auditory prime Visual target Phonological (onset-matching) spinach spin (non-onset-matching) teacher reach Semantic (synonyms) idea notion Morphological teach (pseudo-affixed) corner corn (non-transparent) dresser dress

Transparent morphological: teacher-teach * ** **

Opaque morphological: dresser-dress *

Pseudoaffixed: corner-corn

Opaque morphological: dresser-dress ** ** *

Pseudoaffixed: corner-corn ** * * ** **

LP, Aug 03, Tateshina Day 3 Morphology 1: The M350 as a tool for investigating similarity and identity Do effects of morphology reduce to combined effects of phonological and semantic similarity? Fiorentino & Poeppel: M350 evidence for decomposition in compounding. Stockall et al.: M350 evidence for decomposition in irregular inflectional morphology.

Decomposition of compound words: an MEG measure of early access to constituents Robert Fiorentino1 and David Poeppel1,2 1Department of Linguistics University of Maryland, College Park 2Department of Biology, Neuroscience and Cognitive Sciences Program

Does lexical decomposition occur online? The psychological reality of the morphological complexity difference between compounds (‘flagship’) and single words (‘crescent’) is controversial Does lexical decomposition occur online? If so, then under what conditions, and on what time course? Experiment: test pairwise-matched compounds and single words in a classic visual lexical decision paradigm, using both RT and brain-level magnetoencephalography (MEG) measures to characterize the deployment and time course of lexical decomposition in English noun-noun compounds

Measuring decomposition online: Response time (RT) may reflect differential processing of compounds and single words, if decomposition is present either early or late--possibly subject to lexicalization, length or other constraints (e.g. Andrews, 1986; Van Jaarsveld & Rattink, 1988; Bertram & Hyönä, 2003, among others) MEG offers a millisecond/millimeter spatiotemporal resolution measure of activation along the way to decision An MEG component reflecting lexical activation should track RT when stimulus properties affect lexical access, but contrast with RT in cases where stimuli elicit early access but delayed post-access processing (e.g. Embick et al., 2001; Pylkkänen et al., 2002, among others) In this way, a combined RT and MEG experiment can test for decomposition effects and place them early or late in time course

Example: ‘flagship’ versus ‘crescent’ Items in the compound (CW) and single word (SW) conditions are pairwise matched for overall log frequency, letter-length, and syllabicity. In whole-word properties: In internal structure: flagship = crescent flagship  crescent Log Freq. .68 .69 1.49 1.95 .69 # Letters 8 8 4 4 8 # Syllables 2 2 1 1 2

Stimuli 60 CW and 60 pairwise matched SW, all nouns from Cobuild Corpus (320 million words); 120 disyllabic pronounceable nonwords, including 18 W-NW foils (1:1 word:nonword ratio) Condition Mean Log. Freq. Mean # Letters Example Compound (CW) .455 7.82 flagship  CW 1st/2nd constituents 1.96/1.96 3.82/4.0 flag/ship Single Wd (SW) .455 7.78 crescent Nonword (NW) 7.81 nishpern W-NW Foil (WNW) 7.94 crowskep Measures RT (Visual Lexical Decision); Peak Latency from RMS analysis on 10 left-hemisphere sensors from source distribution (MEG)

CW SW W-NW N=7 analyzed subjects. CW SW W-NW CW SW W-NW M350 RT

M350 and RT: Analysis on Compounds, Single Words, and Word-Nonword Foils W-NW is a control condition which rules out the possibility that subjects launched an early ‘yes’ response based on occurrence of a word morpheme (in that case, RT should be as fast as CW but accuracy lower, but here we find longer RT with extremely high accuracy). CW were faster in both RT and M350 (first MEG component diverging across conditions) In W-NW foils, the M350 shows early activation, but RT flips in the other direction

CW SW Hi-Freq Low-Freq Hi-Freq Low-Freq M350 RT Matched subsets pooled from among the highest- and lowest 20% from the CW and SW bins Hi-Freq Low-Freq Hi-Freq Low-Freq M350 RT

M350 and RT: Analysis on higher- and lower-frequency subsets of CW and SW M350 frequency effect for SW but not CW. RT matches M350 in highest-frequency, but not for the lowest frequency, although M350 tracks the constituent properties even in this case. CW were faster in both RT and M350 for higher-freq. subset of SW and CW (12 items) Early M350 activation preserved in low frequency items, although RT shows no effect; expected for treating some lower-frequency items as novel combinations/nonwords

M350: First Divergent Peak (RMS of 10 sensors, 1 subject) CW SW Recording and analytical method: 160-channel KIT-UMD MEG system (Kanazawa Institute of Technology, Japan); 1000Hz sampling rate; noise reduction; averaging by condition; baseline correct (–100 to 0 ms); band-pass filter (1-30Hz); peak latency analysis from RMS of 10 sensors from M350 source distribution M350: Source Distribution for CW and SW

Conclusions An MEG component around 350ms after stimulus onset varies with the properties of the CW predicted by early access to constituents, which RT also reflects in CW vs. SW comparison. The RT effect reverses, however, for the word-nonword foils and some lower-frequency items. The MEG component tracks early activation of the constituents even in these cases where post-access effects contribute to delayed RT. These effects were found in lexicalized compound word items with short constituents. The RT and MEG results together argue for early access to noun-noun compound constituents, regardless of lexicalization or length constraints.

References Andrews, S. (1986). Morphological Influences on Lexical Access: Lexical or Nonlexical Effects? Journal of Memory and Language, 25, 726-740. Bertram, R., & Hyönä, J. (2003). The length of a complex word modifies the role of morphological structure: Evidence from eye movements when reading short and long Finnish compounds. Journal of Memory and Language, 48, 615-634. Embick, D. et al. (2001). A magnetoencephalographic component whose latency reflects lexical frequency. Cognitive Brain Research, 10, 345-348. McQueen, J.M.,& Cutler, A. (1998). Morphology in word recognition. In A. Zwicky & A. Spencer (Eds.), The handbook of morphology. Oxford: Blackwell. Pylkkänen, L. et al. (2002). Neuromagnetic evidence for the timing of lexical activation: an MEG component sensitive to phonotactic probability but not to neighborhood density. Brain and Language, 81, 666-678. Van Jaarsveld, H., & Rattink, G. (1988). Frequency effects in the processing of lexicalized and novel nominal compounds. Journal of Psycholinguistic Research, 17, 447-473.

LP, Aug 03, Tateshina Day 3 Morphology 1: The M350 as a tool for investigating similarity and identity Do effects of morphology reduce to combined effects of phonological and semantic similarity? Fiorentino & Poeppel: M350 evidence for decomposition in compounding. Stockall et al.: M350 evidence for decomposition in irregular inflectional morphology.

MEG Evidence for the morphological complexity of the English Irregular Past Tense Linnaea Stockall, Priya Singh, Pranav Anand, Justin Fitzpatrick and Alec Marantz Dept. of Linguistics and Philosophy, MIT & KIT/MIT MEG Lab

Irregular Past Tense Priming Marslen-Wilson et al (1993): Cross-Modal Priming Compared: WALKED-WALK (regular past tense) with GAVE-GIVE (irregular past tense) Result: Priming for regulars, but not for irregulars

Irregular Past Tense Priming Allen and Badecker (2002): Cross-Modal Priming Experiment 1: Compared STIFF-STAFF (form overlap) vs. PLUM-STAFF (no form overlap) Result: Inhibition for form overlap

Irregular Past Tense Priming Allen and Badecker (2002): Cross-Modal Priming Experiment 2: Compared GAVE-GIVE (high form overlap) vs. TAUGHT-TEACH (low form overlap) Result: no priming for GAVE-GIVE BUT priming for TAUGHT-TEACH

Irregular Past Tense Priming Allen and Badecker (2002): Interpretation: RT priming for GAVE-GIVE is cancelled out by form overlap induced competition

Priming in the Brain Pylkkänen et al (2002): Cross-modal priming TEACHER-teach vs. OCEAN-teach Results: RT and M350 priming for morphologically related pairs

Priming in the Brain Pylkkänen et al (2002): Amount of priming for TEACHER-teach vs. OCEAN-teach ** *

Irregular morphology & M350 If irregular past tense forms are morphologically complex: They should prime their stems, just as regular past tense forms do If that priming is not apparent in RT effects: It’s disappearance should be due to competition induced inhibition

Irregular morphology & M350 SO The priming should be apparent at an earlier stage in processing that precedes competition – namely the M350 PREDICTIONS: GAVE-GIVE and TAUGHT-TEACH should both show M350 priming GAVE-GIVE should show reduced or non-existent RT priming TAUGHT-TEACH priming should persist at RT

Method Stimuli: 4 comparisons Identity: ghost-ghost vs. trick-ghost Irregulars high form overlap: gave-give vs. plum-give low form overlap: taught-teach vs. warp-teach Orthographic Overlap: stiff-staff vs. clap-staff 20 test and 20 control items per condition = 320 prime-target pairs Plus 320 fillers (NW-NW, W-NW & NW-W)

Method Subjects: 14 right handed native English speakers with normal vision gave informed consent to participate in this experiment. RT and MEG data was collected from 8 subjects, RT data only from 6 subjects.

Method Design: Visual-visual immediate priming (see Pastizzo and Feldman 2002 ) prime + target 450 50 200 0 …2500ms Duration of trial (ms)

Method MEG Data Analysis Epochs corresponding to correct trials averaged together for each condition M350 component identified on basis of time window and magnetic field distribution Best two sensors (one sink, one source) selected Amplitude and latency of peak RMS wave form recorded

Results MEG data (n=8) * * * n.s.

Results MEG data (single subject) M170 M350

Results MEG Data: Significant priming for : Identity condition (*p=0.01) GAVE-GIVE vs. PLUM-GIVE(*p=0.05) TAUGHT-TEACH vs. WARP-TEACH (*p=0.04) No reliable effect for: STIFF-STAFF vs. CLAP-STAFF (p=0.13) But trend towards priming

Results Behavioural Data (n=14) ** * * n.s.

Results Behavioural Data: Significant priming for Identity condition (**p=0.0009) GAVE-GIVE vs. PLUM-GIVE (p=0.03) Significant inhibition for STIFF-STAFF vs. CLAP-STAFF (p=0.01) No reliable effect for TAUGHT-TEACH vs. WARP-TEACH (p=0.21) (but trend towards inhibition)

Discussion Dissociation between M350 and RT w.r.t inhibition Support for claim that M350 indexes early (pre-competition) lexical activation Marlsen-Wilson and Tyler (2003):   Irregular past tense priming survives in long distance priming paradigm (semantic priming and form inhibition disappear); thus M350 priming is true morphological priming Both categories of irregular past tense prime the M350 Morphological priming = support for claim that irregular past tenses are morphologically complex, ie: GAVE = GIVE + PAST Opposite RT results from Allen and Badecker w.r.t. irregulars GAVE primes GIVE despite form overlap, while TAUGHT does not prime TEACH - difference between cross- and intra- modal priming? - GAVE primes GIVE less at RT than at M350