1. Empirical Study of the Form and Function of Linguistic Elements July 2006 Jerry T. Ball Senior Research Psychologist Air Force Research Laboratory Mesa, AZ

2. 2 Basic Assumption Humans have explicit knowledge of the linguistic representations they construct during language comprehension – They know that in the man bit the dog The expression the man refers to a man The expression the dog refers to a dog The sentence describes an action of biting involving a man and a dog, and the man did the biting! Its OK to ask them about their explicit knowledge

3. 3 Why Isnt This Assumption Universally Shared? It may be regarded as conflicting with the assumption that there is an autonomous syntax module where syntactic representations are created that are impervious to higher level cognition and therefore implicit – If semantic representations are composed of abstract concepts which are non-linguistic and non-perceptual, and syntactic representations are implicit, then there are no explicit linguistic representations to reflect on Linguistic Competence and Performance are different things – Competence is implicit

4. 4 Why Isnt This Assumption Universally Shared? Normal humans are not good at explicitly verbalizing their linguistic knowledge Procedural knowledge is implicit – If knowledge of language is largely procedural knowledge as may be required for parsing, then that knowledge is implicit Children learn language implicitly, therefore their knowledge of language must be implicit The explicit knowledge of language we learned in grammar school is not the same knowledge we use to understand language

5. 5 Counter Arguments There is no autonomous syntax module – Much recent psycholinguistic evidence argues against the existence of an autonomous syntax module Visual World Paradigm – Tanenhaus et al. (2004); Henderson & Ferreira (2004) – There are explicit, perceptually-based linguistic representations and there are explicit, perceptually-based non-linguistic representations of the objects and situations the linguistic representations refer to Linguistic Competence and Performance are closely related – We are not syntactic processing automata disguised by performance limitations Ferreira (2003) vs. Townsend & Bever (2001)

6. 6 Counter Arguments Normal humans are not good at explicitly verbalizing their linguistic knowledge in ways that are consistent with prevailing linguistic theory – Maybe the prevailing theory is wrong! – Complicated linguistic representations exceed short-term working memory capacity. To the extent that they exist at all, they cannot be mentally manipulated all at once The process of constructing a linguistic representation may be in part implicit, but the resulting representation is declarative and explicit

7. 7 Counter Arguments The process of learning a language may be in part implicit, but the words, multiword units, and constructions that are learned are declarative and explicit Much of the knowledge of language we learned in school is used in understanding language and is explicitly available – We are capable of using explicit grammar rules, although it is relatively inefficient to do so Second language learners who rely on grammar rules cannot keep up with spoken input – Declarative lookup is more rapid and efficient Language comprehension is largely driven by identification of the largest chunks of meaningmultiword expressions and constructionsnot by composing the meanings of individual morphemes or words, which would be too slow and ambiguous

8. 8 Abstract Concepts vs. Perceptually Grounded Language pilot PILOT Real WorldMental Box Real World perception Language of Thought The Prevailing ViewAn Emerging View grounding perception Implicit (Abstract) Explicit (Perceptual)

9. 9 Abstract Concepts vs. Perceptually Grounded Language pilot PILOT Real WorldMental Box Real World perception The Prevailing ViewAn Emerging View grounding perception Implicit (Abstract) Explicit (Perceptual) Language of Thought

10. 10 Language is Grounded in a Situation Model the book is on the table obj subj head Clause Nominal Pred The book is on the table Words! refers

11. 11 Perceptually Grounded Language Harnads Symbol Grounding Problem Barsalous Perceptual Symbol Systems Lakoff & Johnson – Metaphors We Live BY – abstract concepts are understood via metaphorical extension of more concrete concepts – Good is Up; Bad is Down – Life is a Journey; Kintsch and Van Dijk – Situation Model Graesser, Zwann and others: Situation Model has visuo-spatial characteristics (not just abstract propositions) There must be a perceptual chain to support the understanding of abstract concepts

12. 12 Linguistic Representations Linguistic representations contains words (not abstract concepts) grouped into meaningful units These meaningful units are grounded in non-linguistic representations of the objects and situations to which they refer Linguistic representations encode multiple dimensions of meaning simultaneously – Variation in form reflects this trade-off in encoding Purely syntactic representations fail to capture important generalizations – She is smiling, happy and in a good mood conjunction of clausal heads, not verb, adjective and PP

13. 13 Linguistic Representations Assume a strong form of the Grammatical Constraint: – One should prefer a semantic theory that explains otherwise arbitrary generalizations about the syntax and the lexicon…a theorys deviations from efficient encoding must be vigorously justified, for what appears to be an irregular relationship between syntax and semantics may turn out merely to be a bad theory of one or the other (Jackendoff, 1983) Syntactic and Linguistic Semantic Representations are not distinct! Meaning is captured in the relationship between linguistic (form and function) and non-linguistic representations (objects and situations)

14. 14 Some Competing Representations There are multiple possible representations for even simple sentences – Subject-Predicate (Traditional Grammar) – SVO (Functional Grammar) – Mood-Residue (Hallidays Functional Grammar) Mood = subject + first auxiliary Residue = everything else – S NP VP (Early Generative Grammar) – X-Bar Theory YP = specifier X = head ZP(s) = complement(s) XP YP X XZP(s)

15. 15 Some Competing Representations What is the empirical evidence for these possibilities? Is one representation always preferred? Is there evidence for multiple representations? – Across subjects; Within subjects – Across sentences; Within sentences Halliday suggests three different tiers of representation for clauses – Clause as message: Theme and Rheme – Clause as exchange: Mood and Residue – Clause as representation: Subject Predicator Complement – These layers break the clause apart in different ways that can be empirically tested How do these representations interact with discourse contrasts like Topic/Focus and Given/New?

16. 16 Levelts Early Study (1970) Subjects show a high degree of reliability in making judgments about the cohesiveness of words in sentences Subjects were asked to rate the similarity of words in sentences – Word triads from sentences – + (plus) for most highly related pair in triad – - (minus) for least highly related pair in triad – Seven point rating scale of word pairs A relatedness matrix was generated from data and subjected to hierarchical cluster analysis Resulting trees reflect hierarchical knowledge of language

17. 17 Levelts Early Study the boy has lost a dollar 60 80 100 Relative frequency of more related in triadic comparison % Connectedness method

18. 18 Levelts Early Study Results consistent with basic subject-predicate and NP-VP structure of sentence Results inconsistent with the prevailing assumption that lost combines with a dollar forming an untensed VP before lost a dollar combines withhas forming a tensed VP Results (at least partially) inconsistent with basic SVO structure of sentence

19. 19 Levelts Early Study Jan (John) eet (eats) Piet (Peter) eet (eats) en (and) peren (pears) 3.0 5.0 7.0 Mean scale value (7-point rating scale) Connectedness method appels (apples)

20. 20 Levelts Early Study In the sentence Jan eet appels en Piet eet peren (John eats apples and Peter eats pears), Levelt found that Jan and Piet combined with the verb eet before the objects – Inconsistent with any prevailing linguistic theory Levelt claimed that results reflected deep structure not surface structure – I interpret this to mean meaningful units, not purely syntactic units

21. 21 Reasons for Reviving and Modifying Levelts Early Study Despite these interesting results, this line of empirical research was not pursued beyond about 1972 – Results did not match well to prevailing linguistic theory – Some results were counter intuitive, perhaps reflecting weaknesses in the methods of analysis or interaction with extraneous factors Hierarchical clustering tends to impose a binary structure which may not reflect human behavior – Hierarchical clustering provides an air of objectivity, but at the cost of imposing a fixed structure – Is it really needed? – Why not directly ask subjects to create meaningful groups? There are lots of sentences and linguistic expressions waiting to be empirically examined!

22. 22 Three Part Empirical Study Part 1: Subjects asked to group words into meaningful units within the context of a linguistic expression – Goal is to identify meaningful units Part 2: Subjects asked to identify the word (or words) which contributes most to the meaning of the expression, word (or words) which contributes second most, third most, etc. – Primary goal is to identify the head Part 3: Subjects asked to identify the part of speech of a word within the context of a linguistic expression – Goal is to see if head of NP is necessarily a Noun, head of VP is necessarily a Verb, etc

23. 23 Part 1: Preliminary Results Use sentences from Levelts study Compare results 19 subjects

24. 24 The boy has lost a dollar (n = 18) the boy has lost a dollar the boya dollarlost a dollar has lost 1211874 boy has lost boy dollar lost dollar the boy lost a dollar boy lost dollar the boy has lost 33322 2

25. 25 The boy has lost a dollar (n = 18) has lost a dollar boy hasboy has lost dollar boy has a dollar boy lost a dollar boy has a dollar the dollar 1 111111 has a dollar boy has lost a dollar the boy has a dollar boy a 1111

26. 26 The boy has lost a dollar (n = 18) Subjects are quite consistent at identifying the subject (11) and the clause (12) as meaningful groups The predication or untensed VP lost a dollar (7) may be treated as a multiword unit causing subjects not to see the object NP a dollar (8) which is less frequently identified than the subject (11) The predicator or verb group has lost is not consistently identified (4) The predicate or tensed VP has lost a dollar is only identified by 1 subject!

27. 27 Ball vs. Levelt Study boyhaslostadollarthe boyhaslostadollarthe Levelt Ball

28. 28 John eats apples and Peter eats pears (n = 19) John eats apples Peter eats pears John eats apples and Peter eats pears John eats Peter eats eats apples eats pears 161716665 5 John eats apples Peter eats pears apples pears John eats pears 42 2

29. 29 John eats apples and Peter eats pears (n = 19) John and Peter apples and pears apples Peter eats John eats apples and pears John eats apples pears eats apples Peter John Peter 11 1 111 1

30. 30 John eats apples and Peter eats pears (n = 19) Subjects are very consistent at identifying clauses (16, 17, 16) There is some evidence (6, 6) that subjects group the subject with the verb as in the Levelt study –eats can be used intransitively –John eats is a normal clause There is also some evidence (5, 5) that subjects group the verb with the object

31. 31 Ball vs. Levelt Study applesandPetereatspearsJohneats Levelt Ball appelsenPieteetperenJaneet

32. 32 The nurse likes the patient (n = 19) the nurse likes the patient the patient the nurse nurse patient nurse likes the patient nurse likes 13 1175 5 likes the patient the nurse likes likes patient nurse likes patient 4422

33. 33 The nurse likes the patient (n = 19) nurse the patient the nurse likes patient the nurse the patient 111

34. 34 The nurse likes the patient (n = 19) Subjects are very consistent at identifying the clause (13) and object (13) and quite consistent at identifying the subject (11) The identification of nurse patient (7) probably reflects their close semantic association The subject + verb group the nurse likes and the verb + object group likes the patient occur equally likely (4, 4) but not consistently The group nurse likes the patient (5) follows the determiner the suggesting a pattern like [ the X ] where X can be the rest of the sentence (not just the head of an NP). The somewhat frequent occurrence of the group nurse likes (5) is not expected

35. 35 Main Meaningful Groups thenurselikesthepatient

36. 36 The book was on the table (n = 19) the book was on the table the tablethe book on the table book was was on the table 1310 554 book table was onbook was on book on table book was on the table on table 3 222 22

37. 37 The book was on the table (n = 19) book was on table the book the table on thebook on the table the book was the book was on 1111 1 1

38. 38 The book was on the table (n = 19) Subjects are very consistent at identifying the clause (13) and quite consistent at identifying the subject (10) and object (10) Subjects do not consistently identify any other groups! – The PP on the table was only identified by 5 subjects! – The group book was was identified by 5 subjects – The predicate was on the table was only identified by 4 subjects – The predicator was on was only identified by 2 subjects

39. 39 Main Meaningful Groups thebookwasonthetable

40. 40 The dog is not very hungry (n = 19) the dog is not very hungry the dogvery hungry not very hungry not veryis not 199 873 3 dog not hungry dog very hungry dog hungry is hungry not hungry dog is not very hungry 33 22 22

41. 41 The dog is not very hungry (n = 19) the dog is hungry dog not dog is not hungry dog is not dog is dog not very hungry the dog is not 1111111

42. 42 The dog is not very hungry (n = 19) Subjects are extremely consistent at identifying the clause (19) and somewhat consistent at identifying the subject (9), the modified predicate adjective very hungry (8), and the negated modified predicate adjective not very hungry (7) Subjects also occasionally group not with is not (3) and not very (3) and other groups – This may reflect the status of not as a clausal modifier

43. 43 Main Meaningful Groups thedogisveryhungrynot

44. 44 Group Selection Percentage (more than 1 word) Subject: 55% (41 of 75) Object: 55% (31 of 56) Clause: 80% (106 of 132) Predication (untensed VP): 34% (19 of 56) Predicator (verb group): 16% (9 of 56) Predicate (tensed VP): 17% (19 of 113)

45. 45 Prototypical Meaningful Groups subjectobject clause The predicator (verb group) of a transitive clause does not typically form a meaningful group by itself! subject predicator aux predication clause The predication (untensed VP) tends to be grouped independently from the auxiliary verb!

46. 46 Questions?