Presentation is loading. Please wait.

Presentation is loading. Please wait.

Looking for a gold standard to measure language complexity: What psycholinguistics and neurolinguistics can (and cant) offer to formal linguistics Lise.

Similar presentations


Presentation on theme: "Looking for a gold standard to measure language complexity: What psycholinguistics and neurolinguistics can (and cant) offer to formal linguistics Lise."— Presentation transcript:

1 Looking for a gold standard to measure language complexity: What psycholinguistics and neurolinguistics can (and cant) offer to formal linguistics Lise Menn & Jill Duffield Linguistics Dept/Institute of Cognitive Science University of Colorado, Boulder

2 Background for our stance Cross-linguistic work on basic morphosyntax in aphasia, and on the earliest stages of child phonology shows that these areas are loaded with individual and with language-specific differences. Markedness keeps vanishing into the mist of unverifiability. Its no guide to complexity. So, the issue of whats simple and why, especially in those domains, has been a constant undercurrent and a frequent topic of discussion. 3/23/12 Grammatical Complexity Workshop 2

3 What were going to say – 1 to 4 1. What our brains find difficult is not always what grammars consider complex, partly because whats hard for our brains is not constant; it depends on many factors. (Its complicated.) 2. Proposed metrics for language or grammar complexity should correspond in some way to the gold standard of whats hard for our brains to process. 3. Language complexity measures will have to go beyond a single measure of grammar complexity, because complexity for speakers/writers is not the same as for hearers/readers. 4. Psycho- and neurolinguistics cant provide a royal road to measuring the complexity of a grammar or a language, but they do provide tools to measure processing complexity of individual sentences/utterances for speakers vs. listeners or readers, and learners vs. skilled language users. 3/23/12 Grammatical Complexity Workshop 3

4 What were going to say – 5 to 8 5, Psycho- and neurolinguistic studies indicate that a valid measure of complexity will have to integrate across many linguistic levels (including semantics) and take frequency into account. 6. This implies that construction-based and usage-based approaches to grammar can provide insights into how grammars can come closer to reflecting what our brains do. 7. But: Complexity measures must handle competition and how it gets resolved in both comprehension and production: the paradigmatic axis also plays a role in complexity. 8. Pragmatics/real-world knowledge are involved in resolving this competition. (Implications for practical applications are clear; for comparison of languages, much less so.) 3/23/12 Grammatical Complexity Workshop 4

5 Problematic examples to start from Pourquoi laphasique peut-il dire: "Je ne peux pas le dire" et pas "Elle ne peut pas la chanter''? (Nespoulous & Lecours 1989) [Why can the aphasic person say I cant say it but not She cant sing it ?] –Possible culprits: lexical frequency, collocation frequency (formula status?), emotional weight… Dresslers (1991) work on Breton: A speaker with fluent aphasia tends to name pictures or examples of a single object using the plural form if the object itself is most frequently found in quantity (leaves, potatoes); using the dual if the object is usually found in pairs (eyes, hands). –Goes against all notions of markedness. Relative frequency of particular inflected form of particular word must be the explanation. What brains find difficult is not always what grammar and linguists consider complex. 3/23/12 Grammatical Complexity Workshop 5

6 Why is there a problem? 3/23/12 Grammatical Complexity Workshop 6 1. What our brains find difficult is not always what grammars consider complex, partly because whats hard for our brains is not constant; it depends on many factors. We cannot equate simpler with what is learned earlier, and the reason we cannot do it is that the neural networks change over the course of development.

7 Another example from aphasia Aphasic verb tense production errors are often, as one might expect, substitutions of present tense for past tense. But the reverse seems to be true for at least some agrammatic aphasic speakers of Arabic (Mimouni & Jarema, 1997), Polish (Jarema & Ka ̨ dzieława, 1990), and Korean (Halliwell, 2000) 3/23/12 Grammatical Complexity Workshop 7

8 What brains find difficult is not always what grammars consider complex This is not just about speakers with brain damage. Difficulty is tied to particular circumstances. Production/comprehension asymmetry: –most obvious case: ambiguity. Speakers, who know what they intend to say, often produce utterances that are difficult for hearers because of ambiguity in their referring expressions (He did it!) and elsewhere –Long history of studies of ambiguity resolution in psycholinguistics that demonstrates & relies on the processing difficulty caused by hearers or readers need to resolve ambiguity on-line. –Other studies showing speakers have to put effort into being simple for their hearers (any teacher knows this!) 3/23/12 Grammatical Complexity Workshop 8

9 What brains find difficult is not always what grammars consider complex Learning changes the brain (how could it fail to?), creating a learner/skilled user asymmetry So, relative simple-to-complex rankings must shift as we learn our first languages (OT calls this constraint (re-)ranking). –Phonotactics provides many uncontroversial examples Blevins (1995) illustrations of syllable types: Spanish and Sedang permit CCVC but not CVCC, while the reverse is true for Klamath and Finnish. Japanese speakers struggle with English /tr/ but routinely produce [tst ʃ ], e.g. in place name Tsuchiura) 3/23/12 Grammatical Complexity Workshop 9

10 Before we go on: An essential distinction The grammar of a language as an abstraction across speakers – the patterns out there to be learned (E-language) –The grammar of a language as an abstraction across speakers isnt directly testable by psycholinguistics/neurolinguistics. If that grammar is your main concern, what we have to say has to be mediated by your idea of the relationship between the grammar of a language and the grammar of each speaker. the grammar internal to a given speaker, which should be that speakers internal approximation to the patterns out there ( I -language). –This is what were concerned with in this presentation. 3/23/12 Grammatical Complexity Workshop 10

11 But focusing on speaker-internal grammar is only a start Making this distinction can handle (some) differences between learners (who have a cruder approximation to that abstract grammar) and skilled users (who have a better one). But there are more problems to deal with. One that well keep coming back to: If theres only one internal grammar, a single measure of its complexity cant handle the the fact that whats difficult for comprehension (e.g. ambiguity, unclear reference) is not necessarily difficult for production. 3/23/12 Grammatical Complexity Workshop 11

12 its Complicated 3/23/12 Grammatical Complexity Workshop 12 Back to our first three points, slightly elaborated: 1.What our brains find difficult is not always what grammars consider complex, partly because whats hard for our brains is not constant and depends on many factors. 2.Proposed metrics for language or grammar complexity – at least for speaker-internal language or grammar – should correspond in some way to the empirical gold standard of whats hard for our brains to process. 3.So language complexity measures that claim to be valid metrics for whats in human minds will also have to go beyond a single measure of grammar or language complexity.

13 And we still have the problem cases we started with: Some aphasic people can say I cant say it. but not She cant sing it. A Breton speaker with fluent aphasia tends to name pictures or examples of a single object using the plural form if the object itself is most frequently found in quantity (leaves, potatoes) using the dual form if the object is usually found in pairs (eyes, hands). 3/23/12 Grammatical Complexity Workshop 13

14 What can (or cant) psycholinguistics & neurolinguistics offer? 3/23/12 Grammatical Complexity Workshop 14

15 Whats a gold standard? Why do linguists need one for complexity? A rigorous standpoint, outside of particular formalisms and levels of language, to inform proposed measures of complexity Needed in order to test whether a proposed metric corresponds to measures of what our brains find effortful to process …just as a proposed metric of color must correspond to some psychophysical measure of human responses to color if its going to be useful in accounting for perception. A metric that is useful for calibrating printers may not do well at accounting for what colors people find similar. 3/23/1215 Grammatical Complexity Workshop

16 Computational measures of utterance complexity need to be validated against processing measures – i.e., measures of performance Validating a particular formal analysis of processing (e.g., an analysis that can take the number of competing antecedents for a relative pronoun into account, an analysis that can take various aspects of frequency into account) puts us into the domains of psycholinguistics and neurolinguistics, as other speakers have already made clear. 3/23/1216 Grammatical Complexity Workshop

17 Quotes on taking a psycholinguistic approach to grammatical complexity …the emerging correlation between performance and grammars exists because grammars have conventionalized the preferences of performance, in proportion to their strength and in proportion to their number, as they apply to the relevant structures in the relevant language types. (Hawkins 2004) In order to test the hypothesis that typological distributions reflect processing complexity, an independently motivated, well-defined, and empirically assessable notion of processing difficulty is essential. (Jaeger & Tily 2010) …not only [should] grammatical theorists …be interested in performance modeling, but also …empirical facts about various aspects of performance can and should inform the development of the theory of linguistic competence. –Sag & Wasow, 2011 …the competence-performance distinction acknowledges the value of the sort of work linguists do in their day-to-day research, while recognizing that this work eventually must be placed in a broader psychological context. (Jackendoff 2002) 3/23/12 Grammatical Complexity Workshop 17

18 Complexity measures must predict processing effort for different levels, and their interactions Psycholinguistic experiments with normal speakers Study of cognitive constraints and island effects (Hofmeister & Sag, 2010) Results: Island constraints interact with other features to affect processing effort, correlating with grammaticality judgments –WH-island violations are processed more easily when the extracted element is complex (a WH-phrase) Which employee did Albert learn whether they dismissed after the annual performance review? processed more easily than Who did Albert learn whether they dismissed after the annual performance review? 3/23/12 Grammatical Complexity Workshop 18

19 Measuring complexity for speakers: Error rate as an example Studies of formation of grammatical dependencies: even subject-verb agreement can be complex! Producing subject-verb agreement: a local noun embedded in a subject noun phrase may interfere with the production of agreement - but structure constrains that interference. (Bock & Cutting 1992) The key [ PP to the wooden cabinet]… IS / ARE … The key [ PP to the wooden cabinets]… IS / ARE … The key [ RC that ___ opened the cabinet]… IS / ARE … The key [ RC that ___ opened the cabinets]… IS / ARE … 3/23/12 Grammatical Complexity Workshop 19 attraction effect no attraction effect

20 And theres a glitch: Complexity measures must be compatible with different performance metrics Sp eakers and listeners show different sensitivities to certain structures in processing tasks –While error data Bock & Cutting (1992) showed that relative clauses isolate information interfering with agreement for speakers while prepositional phrase modifiers do not (i.e., a clause boundary effect), Tanner (2012) uses ERP and reading times to show no interaction between structure x local noun number x grammaticality (no clause bounding effect) –in production The key [ PP to the wooden cabinet(s)]… The key [ RC that ___ opened the cabinet(s)]… –in comprehension The key [ PP to the wooden cabinet(s)]… The key [ RC that ___ opened the cabinet(s)]… 3/23/12 Grammatical Complexity Workshop 20 effect of number of embedded noun differs across structure here, but not here

21 Glitch: Complexity measures must be compatible with different performance metrics Speakers and listeners show different sensitivities to certain structures in processing tasks Speaker-hearer asymmetries arent just matters of discourse ambiguity…. –which means that there cant be just one measure of complexity of a sentence – comprehension and production may give different complexity rankings 3/23/12 Grammatical Complexity Workshop 21

22 Glitch: Different sensitivities to structures in different processing tasks Study of discourse and weight-based factors on relative clause extraposition: Francis & Michaelis (2012) used two different tasks to measure the combined effects of: Definiteness: e.g., (Some/The) research… VP length: e.g. (…was conducted/…has been conducted fairly recently) RC length: e.g., (….that refutes the existing theories/…that refutes the existing theories with very clear and convincing evidence) 1.Judgment task: readers saw two versions of a relative clause sentence (e.g., "Further research that indicates...." vs. "Further research has been conducted...") 2.Elicited production task: speakers were given three constituents - a subject NP, a relative clause, and a verb phrase, and asked to order those constituents in a full sentence. 3/23/12 Grammatical Complexity Workshop 22

23 Glitch: Speakers and listeners may show different sensitivities to structures in particular processing tasks In both experiments, indefinite subjects (e.g. "Some research" vs. "The research"). as opposed to definite ones, were more likely to be used with extraposed relative clauses. BUT: In the judgment (comprehension) task, readers preferred the extraposed version for longer relative clauses VP length, however, didn't matter In the production task, VP length did matter: shorter VPs were more likely to predict extraposed relative clauses. Relative clause length, however, didn't matter in production. No explanation for this particular pair of comprehension/production discrepancies yet… 3/23/12 Grammatical Complexity Workshop 23

24 So: We dont have a royal road to offer; how about a set of road-building tools? What tools do we have to measure utterance complexity? Complexity for the listener (comprehension) –Neural correlates of relative effort: one current measure is Event-Related Potential (ERP); –Errors, reaction time and eye-tracking can also be used. (Imaging tools. though glamorous, dont have good enough spatiotemporal resolution yet) Complexity for the speaker (production) –Error-based measures of relative effort comprise the majority of production research. –Reaction time measures and eye-tracking measures also are useful. 3/23/12 Grammatical Complexity Workshop 24

25 Things you probably know about measuring performance Available performance measures for both comprehension and production can only look at one word, utterance, or short passage at a time. Performance always has a large random element – minds differ, and they are simultaneously busy with many things besides the task set by the experimenter (wishing for coffee, worrying about politics or the weather…) So most measures have to be averaged over fairly large sets of similar items, and sometimes over speakers as well. 3/23/12 Grammatical Complexity Workshop 25

26 Measuring complexity for listeners: Event-Related Potential as an example An ERP study of metaphor comprehension (Lai, Curran, & Menn 2009) Comprehending conventional and novel metaphors: Lexical semantics affects comprehension effort when structure is held constant. Sense/nonsense judgment task, comparing listeners ERPs for these four semantic groups: –Literal: Every soldier in the frontline was ATTACKED. –Conventional: Every point in my argument was ATTACKED. –Novel: Every second of our time was ATTACKED. –Anomalous: Every drop of rain was ATTACKED. (assignment of sentences to groups checked by naïve subject ratings) 3/23/12 Grammatical Complexity Workshop 26

27 Measuring complexity for listeners: Event-Related Potential as an example Result: Conventional metaphors required a short burst of additional processing effort when compared with literal sentences. Novel metaphors required a more sustained effort, similar to the effort observed in anomalous sentences. –Literal: Every soldier in the frontline was ATTACKED. –Conventional: Every point in my argument was ATTACKED. –Novel: Every second of our time was ATTACKED. –Anomalous: Every drop of rain was ATTACKED. Comprehension of metaphors involves an initial stage of mapping from one concept to another; such mappings are cognitively taxing, implying that complexity (as processing effort) involves more than structure. ERP matches our intuitions about complexity – and elaborates them. 3/23/12 Grammatical Complexity Workshop 27

28 Measuring complexity for speakers: more about agreement errors Studies of formation of grammatical dependencies: even subject-verb agreement can be complex! Producing subject-verb agreement: a local noun embedded in a subject noun phrase may interfere with the production of agreement - but structure constrains that interference. (Bock & Cutting 1992) The key [ PP to the cabinet]… IS / ARE … The key [ PP to the cabinets]… IS / ARE … The key [ RC that ___ locks the cabinets]… IS / ARE … 3/23/12 Grammatical Complexity Workshop 28

29 Measuring complexity for speakers: more about agreement errors Producing subject-verb agreement: Local nouns that are embedded more deeply are less likely to interfere with agreement production (Franck, Vigliocco & Nicol, 2002) The threat [ PP to the president [ PP of the companies]] … IS / ARE … The threat [ PP to the presidents [ PP of the company]] … IS / ARE … –so syntactic structure can directly affect production complexity – and here, the more complex structure has made one component of producing this sentence easier! 3/23/12 Grammatical Complexity Workshop 29

30 Measuring complexity for speakers: Other measures: Onset latencies – how long before speaker starts to respond –Show difficulty in processing (e.g., competition between verb forms in subject-verb agreement: Haskell & MacDonald, 2003; Staub, 2009) Eye tracking –Shows the interface between high-level message formulation and sentence planning (Brown-Schmidt & Tanenhaus, 2006) Directed elicitation of alternate forms –Provides a measure of accessibility (e.g., production of optional complementizers allows speakers time to access upcoming constituents: Ferreira & Firato, 2002) 3/23/12 Grammatical Complexity Workshop 30

31 So that was the fourth point: Psycho- and neurolinguistics cant provide a royal road to measuring the complexity of a grammar or a language, but they do provide tools to measure processing complexity of individual sentences/utterances for speakers vs. listeners or readers, and learners vs. skilled language users. 3/23/12 Grammatical Complexity Workshop 31

32 5. A valid measure of complexity will have to integrate across many linguistic levels (including semantics), and take frequency into account. What does that mean, and what kinds of data support it? Lets break it up into sub-claims that we can examine one at a time. 3/23/12 Grammatical Complexity Workshop 32

33 5. Complexity measures must predict processing effort for multiple levels, and for interactions among levels, taking frequency into account 3/23/12 Grammatical Complexity Workshop 33 OGrady: … the interaction of simple elements and phenomena can yield systems and effects of a qualitatively different and more complex nature. What valid complexity measures would have to do:

34 First, lets talk about processing effort for different levels and their interactions A testable general complexity measure would have to be able to make predictions about the effort for processing short individual utterances or passages, and correctly predict the relative effort needed. Weve already seen that this effort depends partly on the choice of individual lexical items within those utterances or passages –She cant sing it; Every minute of our time was attacked. 3/23/12 Grammatical Complexity Workshop 34

35 5. Complexity measures must predict processing effort for different levels and their interactions While linguists may analyze a particular utterance into several components, the mind may store it and process it as a whole, or as both whole and analyzed. –Work on idiom blends suggests that idioms are both analyzable and stored as lexical items (Cutting & Bock, 1997) Observed speech error: Help all you want, blended from idioms/formulas Help yourself! and Take all you want! - idiom blends like this respect the internal structure of each component, because their surface syntax remains well-formed, so the speaker must have access to those internal structures. As many theorists now argue, The whole vs. analyzed opposition is much too crude, as many linguists have argued on theoretical grounds (Culicover 1999, among many others.) 3/23/12 Grammatical Complexity Workshop 35

36 5. Complexity measures must predict processing effort for different levels, and their interactions Start with some intuitive evidence for constructions Particular verbs may be specified or preferred by a particular construction, such as mind if in the politeness formula Do/will [person A] mind if [event X]? (Do you mind if I sit here?). Easy to process, and not completely fixed lexically. (This construction is nested in more general patterns, cf. Would your mother have a fit if I…) Conversely, when an unexpected word is used in a familiar construction – especially if it evokes a different construction - it can make the construction relatively harder to process (Would you care if I sit here?). –so effort for different levels needs to include mixed levels, e.g. structures with some lexical items specified 3/23/12 Grammatical Complexity Workshop 36

37 Psycho- and neurolinguistics also tell us that the whole vs. analyzed opposition is too crude. We even have to go beyond constructions and talk about collocations. Collocations with high transitional probabilities, even when the collocations dont form constructions (e.g., subject + aux collocations, he has or I am), are easier for speakers to produce. (Thats why we can have contractions across the NP-VP boundary!) Smooth flow across this boundary is well exemplified in fluent aphasias, e.g. French jargon aphasia (Lecours et al. 1981) Learning probabilities is a subconscious (procedural) and gradual process. Expectations that A will be followed by B, or that A will occur in structure α, become stronger over time, rather than clicking from nothing to all.. 3/23/12 Grammatical Complexity Workshop Complexity measures must predict processing effort for different levels and their interactions

38 Agrammatic aphasic speakers show the effect of high sequential probabilities … forgot the wash the dishes forgot that she was washing the dishes … I like the go home. Id like to go home. Once they have chosen the definite article to follow forget or like, these speakers are in trouble; both plug in familiar phrases (wash the dishes, go home) with appropriate semantic content, but in forms that cannot follow the. These utterances are difficult to explain in grammatical terms, because they show the article being substituted for the infinitive marker, and, even more strikingly, because the collocation V+the goes across the major syntactic boundary between the verb and what should be the start of its NP object. 3/23/12 Grammatical Complexity Workshop 38

39 5. Complexity measures must predict processing effort...taking frequency into account We cant escape dealing with usage and frequency. In the limit, there may be no empirically testable difference between being stored as a unit and having very strong, predictable links from one sub-unit to another. So if your theory doesnt permit multi-level or other kinds of complex units and/or doesnt recognize collocations that arent constructions, thats not necessarily a problem. What you do need is – at least - a way to incorporate item frequencies and transition probabilities into the representation of a structure after the structure has its words filled in (or during the process of getting them filled in). 3/23/12 Grammatical Complexity Workshop 39

40 5. Complexity measures must predict processing effort for different levels, and their interactions Well shortly consider some more evidence for constructions, as structures lying between the extremes of lexical items and full clauses. But: the complexity of deploying a construction is not determined by construction frequency alone (e.g., Dutch word order in aphasia, Bastiaanse, Bouma & Post 2009) For example, the interpretation or deployment of a construction, such as the English subject or object cleft, may be made better or worse by the existence of similar constructions (Dick et al. 2001:772): the paradigmatic axis' is relevant to processing complexity – more on that soon. 3/23/12 Grammatical Complexity Workshop 40

41 Purely structural complexity does affect normal and aphasic language processing (e.g., Thompson & Shapiro 2007 showed that practice on structurally more complex clause generalizes to improvement on less complex clauses, but practice on simple clauses doesnt generalize to more complex ones ). But in general: processing effort is a function of the interaction of structure and frequency at multiple levels, Lets look at a psycholinguistic study of normal speakers and a related one of aphasic speakers which demonstrates this. 3/23/12 Grammatical Complexity Workshop Complexity measures must predict processing effort for different levels, and their interactions, taking frequency into account

42 An example of frequency/structure interaction: Relative verb-(subcategorization) frame frequencies create a bias (readers expectation) that affects readers processing patterns and comprehension. Shrink, for example, has a syntactic bias towards the undergoer- subject argument structure – it is more frequently used in the unccusative frame than in any other. The sweater shrank two sizes They shrank the sweater two sizes 5. Complexity measures must predict processing effort for different levels, and their interactions – taking frequency into account

43 The sweater shrank two sizes They shrank the sweater two sizes Eye movements during reading: verbs with similar syntactic biases pattern together, whereas verbs that are similar only in meaning do not, Garnsey et al Comprehension: Clauses that conform to a verbs bias, - They proposed X, They suggested that Y - comprehended faster and more accurately than bias-violating sentences with the same structure - They proposed that X, They suggested Y (ibid.) Production studies supporting this: Gahl & Garnsey 2004, Complexity measures must predict processing effort for different levels, and their interactions – taking frequency into account

44 Gahl et al. 2003: People with aphasia comprehend sentences better when the verb is in its preferred frame. Clauses with unaccusatives (undergoer-subjects) can be hard or easy, depending on how typical it is for the verb to be used in the unaccusative construction. This supports a processing model that has relatively direct semantic construal of the verb frames of simple clauses, rather than indirect construal involving, e.g., traces. 5. …taking both structure and frequency into account

45 The interaction of levels and the effect of frequency on complexity also support our sixth point: 6. Construction-based and usage-based approaches to grammar can provide insights into how grammars can come closer to reflecting what our brains do. 3/23/12 Grammatical Complexity Workshop 45 Finally, lets look at evidence for our last two points

46 Complexity measures must be compatible with different performance metrics 3/23/12 Grammatical Complexity Workshop 46 a valid measure of complexity will have to integrate across many linguistic level, and furthermore… There can be no single performance metric.

47 Complexity measures must be compatible with different performance metrics There can be no single performance metric. Why not? What is effortful for speakers does not always match what is effortful for hearers Speakers and hearers have been shown to be sensitive to different structural features the same utterance types Therefore: The goal of an overall complexity measure needs to be split into sub-goals; several roughly commensurable but sometimes incompatible measures are required. Lets look at some experimental evidence 3/23/12 Grammatical Complexity Workshop 47

48 Complexity measures must be compatible with different performance metrics Identifying referents--Production Cognitive demands can swamp information that would improve referential success (Wardlow Lane & Ferreira, 2008) 3/23/12 Grammatical Complexity Workshop 48 Speakers are asked to name target objects (in the common ground) so that listeners can identify them, while being faced with privileged objects of varying saliency. Speakers could name the target object by only using information that is common ground (e.g., the heart) or by also using privileged information (e.g., the small heart) that is useless and possibly confusing to the hearer.

49 Complexity measures must be compatible with different performance metrics Identifying referentsProduction experiment Cognitive demands swamp information that would improve referential success (Wardlow Lane & Ferreira, 2008) 3/23/12 Grammatical Complexity Workshop 49 Results: When the saliency of privileged information was increased, speakers made more reference to it, (e.g., identifying the target object as the small heart) than when such information was less salient, despite the risk of confusing the listener. In this task, cognitive demands result in a lower processing load for the speaker when producing more (complex) descriptions.

50 Complexity measures must be compatible with different performance metrics Identifying referentsComprehension experiment using eye-tracking (following the listeners successive visual fixations) Visual context affects ambiguity resolution (Spivey, Tanenhaus, Eberhard & Sedivy, 2002) Four conditions: Listeners heard either Put the apple on the towel in the boxor Put the apple thats on the towel in the box Sometimes what they saw was this: 3/23/12 Grammatical Complexity Workshop 50 A. on the towel is redundant: theres only one apple

51 Complexity measures must be compatible with different performance metrics Identifying referentsComprehension experiment using eye-tracking (following the listeners successive visual fixations) Visual context affects ambiguity resolution (Spivey, Tanenhaus, Eberhard & Sedivy, 2002) Four conditions: Listeners heard either Put the apple on the towel in the boxor Put the apple thats on the towel in the box And sometimes what they saw was this: 3/23/12 Grammatical Complexity Workshop 51 B. on the towel is relevant: there are two apples

52 Complexity measures must be compatible with different performance metrics Put the apple on the towel in the box. Listeners eye movements were recorded to see whether they treated the PP modifier (e.g., …on the towel) in the instruction as a goal, in which case they would look to the empty towel, or as a modifier, in which case they would look to the apple on the towel, and not to the empty towel. 3/23/12 Grammatical Complexity Workshop 52 A. redundan t info: one apple B. relevant info: two apples

53 Complexity measures must be compatible with different performance metrics Participants heard instructions in one of these two forms: - temporarily ambiguous: Put the apple on the towel in the box or - unambiguous (control): Put the apple thats on the towel in the box Results: In the one-referent visual condition listeners were more likely to look towards the empty towel – that is, to treat the PP modifier on the towel in this temporarily ambiguous instruction as a goal - than in the two-referent instruction. 3/23/12 Grammatical Complexity Workshop 53 A. redundan t source info: one apple B. useful source info: two apples

54 Complexity measures must be compatible with different performance metrics Results - 2: In the two-referent condition (B), there was no difference between the responses to the temporarily ambiguous Put the apple on the towel in the box and the unambiguous Put the apple thats on the towel in the box. Participants looked to the correct apple, and then to the correct goal (the box), but not to the false goal (the towel). 3/23/12 Grammatical Complexity Workshop 54 B. useful source info: two apples A. redundan tsource info: one apple

55 Complexity measures must be compatible with different performance metrics So the visual context – the second apple – in display (B) resulted in a lower processing load for the listeners who heard the temporarily ambiguous (garden-path) Put the apple on the towel in the box instruction. The structural ambiguity (does on the towel modify the apple or describe what is to be done with it?) apparently added no complexity/processing load for them. 3/23/12 Grammatical Complexity Workshop 55

56 So weve seen that both speaker and hearer are influenced by their environments. Speakers may produce something thats more complex because its easier to say in the context, even though that context can make the utterance more difficult for the listener. (Wardlow Lane & Ferreira, 2008) Listeners can easily comprehend something thats more complex by making use of the visual context. (Spivey et al., 2002) 3/23/12 Grammatical Complexity Workshop 56 Complexity measures must be compatible with different performance metrics

57 7. Complexity measures must handle competition and how it gets resolved in both comprehension and production 3/23/12 Grammatical Complexity Workshop 57 What valid complexity measures would have to do:

58 7. Complexity measures must handle competition and how it gets resolved in both comprehension and production Processing is strongly influenced by competition – normal and aphasic speakers have both substitution errors and blends. Competing lexical items - substitutions. addressing a child using the name of their sibling tip-of-the-tongue (Bhatnagar – Baharav – Bhuvana!) aphasic word search: my right side was blunt- lump- trip- slip- Competing members of an inflectional paradigm Competing interpretations (visiting relatives…, garden paths) Competing possible continuations of a partially-produced structure. 3/23/12 Grammatical Complexity Workshop 58

59 7. Complexity measures must handle competition and how it gets resolved in both comprehension and production English-speaking aphasic speaker struggles: And the boy give to a cookiethe boy give to girl a cookie, apparently torn between double object and prepositional constructions. More errors of article choice among people with agrammatic aphasia in German than in Italian. A German speaker struggles to find the right form: die...der...das...die...den..den Hund ( Bates, Wulfeck, & MacWhinney 1991 ) More verb inflection errors among people with agrammatic aphasia in languages with richer verb paradigms. So: the paradigmatic axis, as well as the structural syntagmatic axis, affects the complexity of deploying a given structure. 3/23/12 Grammatical Complexity Workshop 59

60 8. Pragmatics/real-world knowledge are involved in resolving this competition 3/23/12 Grammatical Complexity Workshop 60 What valid complexity measures would have to accommodate:

61 8. Pragmatics/real-world knowledge are involved in resolving this competition Pragmatics has an enormous effect on processing We already saw this in the metaphor processing study Other speakers here have also made this point Lets look at a clear, reasonably controlled example… 3/23/12 Grammatical Complexity Workshop 61

62 Whats this? 3/23/12 Grammatical Complexity Workshop 62

63 Whats this? 3/23/12 Grammatical Complexity Workshop 63

64 Whats this? 3/23/12 Grammatical Complexity Workshop 64

65 8. Pragmatics/real-world knowledge is involved in resolving this competition Competing perspectives add to the difficulty of choosing among truth-value-equivalent constructions Theres a table with a lamp, A lamp on a table – were rarely tempted to say Theres a table under the lamp ! Theres a bed with a pillow at the wrong end or A pillow at the foot of a bed – we dont hesitate to choose the larger object as the location. In the third picture, were slowed up by the four-way competition among possible orders of mention and choice of location: –The footstool is behind the armchair –The armchair has a footstool behind it –Theres a footstool with an armchair in front of it –Theres an armchair in front of a footstool. 3/23/12 Grammatical Complexity Workshop 65

66 8. Pragmatics/real-world knowledge is involved in resolving this competition Performance data like this lead us to conceptualize complexity as requiring an integration of fluctuating levels of competition among different types of structures, varying from moment to moment as well as from speaker to speaker. 3/23/12 Grammatical Complexity Workshop 66

67 Wrapping up 3/23/12 Grammatical Complexity Workshop 67

68 To evaluate complexity measures for speaker- internal grammar, we need: Studies with systematic variation of structures and lexical items that would let us zero in on how construction frequency, lexical frequency, and other sources of processing difficulty interact. Should be carried out with normal speakers using ERP or other sensitive measures of processing load, because the number of linguistic variables is so high that a good design would impose serious burdens on a speaker with aphasia. 3/23/12 Grammatical Complexity Workshop 68

69 Implications for a valid complexity measure Point 1 – the need to work across unit sizes/levels and to predict the interaction of frequency and structure (including transition probabilities) are arguments for framing a processing account in terms of a formalization of grammar that takes surface structure (constructions) and usage into account Point 2 – compatibility with different performance metrics - implies that the goal of a creating formal processing complexity metric needs to be split into sub-goals for comprehension, production, and learning. Point 3 – that competition adds to complexity – implies that the paradigmatic axis as well as the structural syntagmatic axis is affects the complexity of a given structure. Point 4 – the need to deal with the impact of real-world situations on processing - suggests that any formal metric, even one that satisfies the above requirements - will be incomplete. But that doesnt mean it wont be interesting and useful! 3/23/12 Grammatical Complexity Workshop 69

70 How might a formal system handle this? The construction grammar approach treats language as an inventory of constructions (both lexical and combinatoric) (abstract entities that are the loci of constraints on the interface of form & meaning; Sag, Boas, & Kay 2012). Each construction has specific information about its properties; we dont assume generalizations across utterances simply because of structural similarity. For example, there are unique properties of various filler-gap constructions (Sag, 2010). By using a type hierarchy, we can represent different grains of constructions. 3/23/12 Grammatical Complexity Workshop 70

71 How might a formal system handle this? Furthermore, constructions contain more than just syntactic informationthey contain semantic and pragmatic information as well, thus formalizing these elements. These features may allow us to deal with the interaction of different levels (morphological, lexical, syntactic, semantic & pragmatic) although it does not directly address the issue of frequency and transition probabilities. The unique specification of construction properties organized in a type hierarchy could allow us to predict certain types of competition. For example, if two constructions are specified for the same lexical item, argument structure, or co-occurrence restrictions, we would expect there to be competition between those two constructionsalthough the competition itself is not formally represented. 3/23/12 Grammatical Complexity Workshop 71

72 its Complicated 3/23/12 Grammatical Complexity Workshop 72

73 Thank you! 3/23/12 Grammatical Complexity Workshop 73 Special thanks to Laura Michaelis-Cummings for steering us to good readings and giving feedback.


Download ppt "Looking for a gold standard to measure language complexity: What psycholinguistics and neurolinguistics can (and cant) offer to formal linguistics Lise."

Similar presentations


Ads by Google