Presentation on theme: "Double R Theory January 2011 Jerry Ball Human Effectiveness Directorate 711 th Human Performance Wing Air Force Research Laboratory."— Presentation transcript:
Double R Theory January 2011 Jerry Ball Human Effectiveness Directorate 711 th Human Performance Wing Air Force Research Laboratory
2 Theoretical Foundations Language Representation and Processing Double R Grammar – Cognitive Linguistic theory of the grammatical encoding of referential and relational meaning Double R Process – Psycholinguistic theory of the processing of English text into Double R Grammar based representations Double R Model – Computational implementation using the ACT-R cognitive architecture and modeling environment DoubleRTheory.com
3 Theoretical Foundations Language Representation and Processing Double R Grammar – Cognitive Linguistic theory of the grammatical encoding of referential and relational meaning Double R Process – Psycholinguistic theory of the processing of English text into Double R Grammar based representations Double R Model – Computational implementation using the ACT-R cognitive architecture and modeling environment DoubleRTheory.com
4 Theoretical Foundations Grounding Language in Experience Symbol Grounding (Harnad) – Ungrounded symbols are meaningless – There must be a chain from abstract to perceptually grounded concepts that provides the grounding for abstract concepts Perceptual Symbol Systems (Barsalou) – No purely abstract concepts – The brain is a highly evolved perceptual (motor) organ – Imagery simulates perceptual experience Embodied Cognition (Lakoff et al.) – Abstract concepts are often understood via metaphorical association with more concrete concepts Good is up—Bad is down; Life is a journey
5 Theoretical Foundations Situation Model Situation Model (Kintsch et al.) – Originally viewed as a propositional text base (van Dijk & Kintsch) Elaboration of propositions in linguistic input – Now viewed as a Spatial-Imaginal (and Temporal) representation of the objects and situations described by linguistic expressions and encoded directly from the environment (Zwann et al.) Non-propositional (in part) Non-textual No available computational implementations – Provides grounding for linguistic representations
6 Abstract Concepts vs. Perceptually Grounded Language “pilot” XY-123 (aka PILOT) Real WorldMental Box Cognition The Prevailing “Cognitive Psychological” View Concept ~ abstract amodal fixed point in conceptual space pilot “pilot” Perception
7 Abstract Concepts vs. Perceptually Grounded Language “pilot” pilot “pilot” Mental BoxReal World perception An Emerging “Embodied Cognition” View grounding perception Explicit (Perceptual) Perceptual Symbol Do we really need abstract concepts? How are they learned? Cognition is the simulation of perceptual experience Concept ~ dynamic and tangled interconnections of associated experiences
8 Language is Grounded in a Situation Model The horse runs the horse runs subj headSRE ORE PRED Dynamic mental simulation of horse running would be better! SRE: Situation Referring Expression ORE: Object Referring Expression PRED: Predicate refers
9 Language is Grounded in a Situation Model The paint runs the paint runs subj headSRE ORE PRED Dynamic mental simulation of paint running would be better! refers Each experience of a running event changes the RUN concept!
10 Guiding Linguistic Principles Jackendoff’s (1983) Grammatical Constraint: …one should prefer a semantic theory that explains otherwise arbitrary generalizations about the syntax and the lexicon…a theory’s deviations from efficient encoding must be vigorously justified, for what appears to be an irregular relationship between syntax and semantics may turn out merely to be a bad theory of one or the other
11 Guiding Linguistic Principles Langacker’s Cognitive Grammar (1987, 1991) – Grammar is simply the structuring and symbolization of semantic content – Exclusionary Fallacy – one analysis, motivation, categorization, cause, function or explanation for a linguistic phenomenon necessarily precludes another – Rule/List Fallacy – the assumption, on grounds of simplicity, that particular statements (i.e. lists) must be excised from the grammar of a language if general statements (i.e. rules) that subsume them can be established
12 Construction Grammar (Fillmore, Goldberg, Sag, etc.) Constructions—the basic units of grammar— are pairings of form, function and meaning the man hit the ballform function meaning subject predicator object HIT ( AGENT : MAN PATIENT : BALL ) “concepts” uppercase word syndrome semantic roles
13 Construction Grammar Declarative Clause + Intransitive Verb construction – The woman sneezed Decl Clause + Transitive Verb construction – The man hit the ball Wh-Question + Ditransitive Verb + Passive constr. – Who was given the ball? Decl Clause + Intrans Verb + Causative constr. – The woman sneezed the napkin off the table
14 X-Bar Theory Key element of Chomsky’s Generative Grammar from the 1970’s to the 1990’s Theory of the universal structure of all languages – Autonomous from meaning X-Bar structure presumed to be innate (not learned) Replaced Phrase Structure Grammar component of earlier theory (e.g. S NP VP; NP Det N; …) Has gone thru several major revisions resulting in more and more complex syntactic representations Subsumed by other theoretical considerations in Chomsky’s Minimalist Program (circa. 1995)
15 X-Bar Theory (Chomsky 1970) SpecifierX-Bar XP X (head) Complement(s) Generalization over Syntactic Categories – NP, VP, AP, PP XP Spec X-Bar X-Bar X (Head) Comp(s) Universal structure of all languages except that relative locations can vary (e.g. complements may occur before or after head) Universal structure of all languages – very strong claim – generative linguists spent next 20+ years trying to demonstrate it!
16 X-Bar Theory ~ 1993 Universal structure of all languages Something went seriously wrong! XP (X’’) SpecX-Bar (X’) XComp (YP) Locally adheres to X-Bar Schema Globally very complex! X-Bar schemaUniversal structure of clause
17 X-Bar Theory (adapted in Ball 2007) SpecifierX-Bar XP X (head) Complement(s) Generalization over grammatical categories – referring expression referential layer relational layer What’s right about X-Bar Theory: 1.Referential layer 2.Relational layer 3.Grammatical functions: specifier, head, complement, modifier (but need to be semantically motivated) head – semantically most significant element specifier – indicates referential function complements – arguments of relational head
18 Simpler Syntax (Culicover & Jackendoff 2005) Reaction against the complex syntactic representations of modern mainstream generative grammar – Against syntactocentrism If there is a level of meaning representation, then syntactic representations can be simpler – Flat as opposed to deeply nested syntactic representations Culicover & Jackendoff are former students of Chomsky
19 Comprehensive Grammars of English Cambridge Grammar (Huddleston & Pullum, 2002) – Informed by linguistic theory, but attempts to cover most of English with all its exceptions – Adds functional categories to syntactic representations Longman’s Grammar (Quirk et al., 1985) – Focus on basic functions of linguistic elements – In the spirit of Functional Grammar as opposed to Chomsky’s Generative Grammar
20 Double R Grammar Theory of the grammatical encoding of Referential and Relational meaning Derived from X-Bar Theory prior to the introduction of functional heads (Chomsky, 1970) Grammatical Functions (GFs) explicitly represented – Phrase Level: Specifier, Head, Complement, Modifier – Clause Level: Specifier, Head, Subject (Comp), Modifier Specifier + Head Referring Expression (Max Proj) – All the grammatical info needed to support reference Specifier = locus of Referential meaning Head = locus of Relational meaning
21 Basic Nominal – X-Bar Theory (Chomsky 1970) thecaptain D N NP Lexical Item Syntactic Category NP D N-Bar N-Bar N the captain N-Bar Head (implicit) Specifier (implicit) Maximal Projection Grammatical Functions are implicit in syntactic representation Noun is head of nominal (NP) N-bar level is required Later – D reanalyzed as head of DP (functional head) DP D-bar NP
22 Basic Nominal – Simpler Syntax thecaptain DN NP Double Line marks head Lexical Item Noun is head of nominal (NP) No N-bar level Syntactic Category One (explicit) phrase level GF: 1.Head NP D N (head)the captain
23 Basic Nominal – Cambridge Grammar thecaptain Det: D Head: N NP GF: Syntactic Category Lexical Item Four phrase level (NP) GF’s: 1.Head 2.Determiner 3.Complement 4.Modifier Noun is head of nominal (NP) N-Bar level allowed, but not required Note: Nominal = N-bar, not NP for H&P NP Det:D Head:Nthe captain
24 Nominal ~ Referring Expression John Lyons, Semantics, Vol 2, 1977, p. 445 “Looked at from a semantic point of view, nominals are referring expressions” “They are expressions which have a certain potential for reference”
25 Basic Nominal – Double R thecaptain SpecHead Object Referring Expression (ORE) Referential poleRelational pole DN Grammatical Function (GF) Grammatical/ Lexical Construction Lexical Item Four phrase level GF’s: 1.Head 2.Specifier 3.Complement 4.Modifier Nominal ~ Object Referring Expression Noun is head of nominal (NP) No N-bar level ORE Spec Head; Spec D; Head Nthe captain
26 Basic Clause X-Bar Theory ~ 1970s Joerun VP S NP NV S NP VP VP Spec v V-Bar V-Bar V Joe runs Structure of S not explained by X-Bar Theory circa no specifier or head of S N-barV-barSpec v TENSE pres Deep Structure gets transformed into Surface Structure (Transformational Grammar) -- TENSE pres + run runs Deep Structure
27 Basic Clause – Simpler Syntax Joerun VP S NP NV Syntactic Tier: GF Tier:Subject Head of S not specified in Culicover (2009) In Jackendoff (2002), no lexical items in syntactic tier S NP AUX VPJoe runs Clause level GF’s: 1.Subject 2.Object 3.Second Object CS: RUN(AGENT:X) AUX TENSE pres affix hopping Vestige of Transformational Grammar
28 Basic Clause – Cambridge Grammar Joeruns Predicate: VP Clause Subj: NP Head: N Predicator: V Clause level GF’s: 1.Predicate ~ Head of Clause 2.Subject ~ External Complement 3.Modifier Additional phrase level (VP) GF: 1.Predicator ~ Head of VP Clause Subj:NP Predicate:VP Predicate:VP Predicator:V Joe runs No equivalent to determiner at clause level!
29 Basic Clause – Double R (Spec+)Head Situation Referring Expression (SRE) Subj | ORE | (Spec+)Head | PN | Joe V fin | runs Clause level GF’s: 1.Head 2.Specifier 3.Subject ~ External Complement 4.Modifier SRE ~ Clause or S SRE Subj (Spec+)Head Subj ORE (Spec+)Head V fin Joe runs Grammatical Construction Specification fused with Head
30 Basic Clause X-Bar Theory ~ 1970s VP S NP | N-bar | N | Joe S NP VP VP Spec v V-Bar (head) V-Bar V (head) NP (comp) NP D (spec) N-bar (head) N-Bar N (head) Joe kicks the ball N-bar | N | ball D | the NP Later – VP reanalyzed as head of S & Subject NP reanalyzed as specifier of S – left of head so must be spec! S NP (spec) VP (head) Later – tense reanalyzed as head of IP; S reanalyzed as CP (complementizer phrase) with C-bar = IP CP IP = C-bar (head) IP (inflection phrase) NP (spec) I-bar (head) I-bar I (tense head) VP (comp) Spec v TENSE pres V-Bar V | kick
31 Basic Clause – X-Bar Theory ~ 1980s VP CP NP | N-bar | N | Joe CP IP = C-bar (head) IP NP (spec) I-bar (head) I-bar I (head) VP (comp) VP V-Bar (head) NP (comp) V-Bar V (head) Joe kicks the ball N-bar | N | ball D | the NP IP = C-bar Later – additional levels proposed: AgrP (agreement) AgrSP, AgrOP NegP (negation) ModP (modality) Etc. Complement of I-bar Spec of IP (subject) Head of CP I-bar V-Bar V | kick I | TENSE pres Sentence now adheres to X-Bar Theory!
32 Basic Clause – X-Bar Theory ~ 1993 C’ = C-bar TP = IP Subj Agreement Obj Agreement VP way down here! Structure below VP not shown Universal clausal structure of all languages! Some languages have object agreement, so universal, innate structure must have this layer! | Joe? kick the ball TENSE pres Joe kicks the ball
33 Basic Clause – Simpler Syntax VP || V | kick S NP || N | Joe Syntactic Tier: GF Tier:SubjectObject S NP AUX VP VP V (head) NP Joe kicks the ball CS: KICK(AGENT:X PATIENT:Y ) N | ball D | the NP AUX TENSE pres affix hopping
34 Basic Clause – Cambridge Grammar Predicator: V | kicks Clause Subj: NP | Head: N | Joe Head: N | ball Det: D | the Predicate: VP Obj: NP Clause Subj:NP Predicate:VP Predicate:VP Predicator:V Obj:NP Joe kicks the ball Additional phrase level (VP) GF: 1.Object ~ Complement
35 Basic Clause – Double R (Spec+)Head | V fin | kicks SRE Subj | ORE | (Spec+)Head | PN | Joe Head | N | ball Spec | D | the (Spec+)Head | Pred-Trans-Verb Obj | ORE SRE Subj (Spec+)Head Subj ORE Head Pred-Trans-Verb PTV Head Obj Joe kicks the ball Grammatical Construction Additional phrase level GF: 1.Object ~ Complement
36 Basic Clause with Auxiliary – Simpler Syntax VP [PROG-PART] || V [PROG-PART] | kick S NP || N | Joe Syntactic Tier: GF Tier:Subject N | ball Object AUX S NP AUX VP VP V (head) NP D | the NP Joe is kicking the ball CS: KICK(AGENT:X PATIENT:Y ) TENSE pres V AUX | be affix hopping
37 Basic Clause with Auxiliary – Cambridge Grammar Predicator: V | kicking Clause Subj: NP | Head: N | Joe Head: N | ball Det: D | the Predicate: VP Obj: NP Predicate: VP Predicator: V | is Clause Subj:NP Pred:VP Pred:VP Pred-or:V Comp:Clause bare Comp:Clause bare Pred:VP Pred:VP Pred-or:V Obj:NP Joe is kicking the ball Comp: Clause bare | catenative verbs bare clause (no subj or tense) No specifier GF head of clause!
38 Basic Clause with Auxiliary – Double R Head | V | kicking SRE Subj | ORE | Head | N | Joe Head | N | ball Spec | D | the Head | Pred-Trans-Verb Obj | ORE Spec | Aux | is SRE Subj Spec Head Subj ORE Spec Aux Head Pred-Trans-Verb PTV Head Obj Joe is kicking the ball head of clause
39 Possessive Nominal – Simpler Syntax Joebook N NP NP NP’s NJoe’s book ’s No label!
40 Possessive Nominal – Cambridge Grammar Joe’s book Head: N NP Plain Subj+Det: NP Gen Fused subject-determiner H & P allow GF’s to be fused – consistent with grammatical evidence NP Plain Subj+Det:NP Gen Head:NJoe’s book Additional phrase level GF: 1.Subj ~ Complement
41 Possessive Nominal – Double R Joe ’sbook RefPtHead Possessive Object Referring Expression (ORE) Spec Referential poleRelational pole NPoss-Mkr ORE – (Spec+)Head – PN Poss-ORE RefPt+Spec HeadJoe’s book Additional phrase level GF: 1.Ref Pt ~ Complement Grammatical Construction
42 Clause without Main Verb – Simpler Syntax PP || P | on S N | book Syntactic Tier: GF Tier:Subject N | table S NP AUX PP D | the NP D | the the book is on the table CS Tier: BE(THEME:X, ON(THEME:Y)) AUX TENSE pres V AUX | be
43 Clause without Main Verb – Cambridge Grammar Head: P | on Clause Head: N | book Head: N | table Det: D | the Comp: PP Obj: NP Predicate: VP Predicator: V | is Clause Subj:NP Pred:VP Pred:VP Pred-or:V Comp:PP Comp:PP Head:P Obj:NP Det: D | the Subj: NP the book is on the table head of clause!
44 Clause without Main Verb – Double R Head | P | on SRE Head | N | book Head | N | table Spec | D | the Head | Pred-Prep Obj | ORE Spec | Aux | is SRE Subj Spec Head Subj ORE Spec Aux Head Pred-Prep Pred-Prep Head Obj Subj | ORE Spec | D | the the book is on the table Grammatical Construction head of clause!
45 Clause without Main Verb – Simpler Syntax PP || P | on S N | book Syntactic Tier: GF Tier: Subject N | table S NP AUX PP D | the NP D | the the book’s on the table CS Tier: BE(THEME:X, ON(THEME:Y)) AUX TENSE pres V AUX | be
46 Clause without Main Verb – Cambridge Grammar Head: P | on Clause Head: N | book Head: N | table Det: D | the Comp: PP Obj: NP Predicate: VP Predicator: V | ’s Clause Subj:NP Pred:VP Pred:VP Pred-or:V Comp:PP Comp:PP Head:P Obj:NP Det: D | the Subj: NP Don’t see how H&P can allow GF’s to be fused – inconsistent with grammatical evidence the book’s on the table fused?
47 Clause without Main Verb – Double R Head | P | on SRE Head | N | book Head | N | table Spec | D | the Head | Pred-Prep Obj | ORE Spec | Aux | ’s SRE Subj+Spec Head Subj ORE Spec Aux Head Pred-Prep Pred-Prep Head Obj Subj | ORE Spec | D | the the book’s on the table
48 Passive Clause – Simpler Syntax VP S Syntactic Tier: GF Tier:Subject PP be v-aux S NP AUX VP be VP be be (head) VP[PASS] VP[PASS] V[PASS] (PP by ) PP by by NP V [PASSIVE] | take VP [PASSIVE] NP by p NP Object the book was taken by Joe CS: TAKE(AGENT:X, PATIENT:Y) AUX TENSE past the book Joe
49 Passive Clause – Cambridge Grammar Predicator: V | taken Clause Head: N | book Comp: NP Joe Comp: P | by Predicate: VP Comp: PP Predicate: VP Predicator: V | was Clause Subj:NP Pred:VP Pred:VP Pred-or:V Comp:Clause bare Comp:Clause bare Pred:VP Pred:VP Pred-or:V Comp:PP Det: D | the Subj: NP the book was taken by Joe Comp: Clause bare |
50 Passive Clause – Double R Head | V | taken SRE Head | N | book Obj ORE Head, PN Joe Head, P by Head | Pred-Trans-Verb Obj | Bind 1 Spec | Aux | was SRE Subj Spec Head Subj ORE 1 Spec Aux Head Pred-Trans-Verb PTV Head Obj Mod Obj Bind 1 Subj | ORE 1 Spec | D | the the book was taken by Joe Mod | Pass-By-RE Grammatical Construction
51 Yes-No-Question – Double R Y-N-Quest-SRE Head | Pred-Trans-Verb Obj | ORE | Operator | Aux | did Y-N-Quest-SRE Op Subj (Spec) Head Pred-Trans-Verb Head Obj Did he take it? Subj | ORE | Head | Pron | he Head, Pron | it Additional clause level GF: 1.Operator ~ Specifier Grammatical Construction Head | V | take
52 Yes-No-Question – Double R Head | V | taken Y-N-Quest-SRE Head | Pred-Trans-Verb Obj | ORE | Operator | Aux | could Y-N-Quest-SRE Op Subj (Spec) Head Pred-Trans-Verb Head Obj Could he have taken it? Subj | ORE | Head | Pron | he Head, Pron | it Spec | Aux | have
53 Wh-Question – Double R Wh-Quest-SRE Head | Pred-Trans-Verb Obj | Bind 1 Wh-Quest-SRE Wh-Focus Op Subj (Spec) Head Pred-Trans-Verb Head Obj Wh-Focus | Wh-ORE 1 | Head | Wh-Pron | what What did he take? Subj | ORE 1 | Head | Pron | he Additional clause level GF: 1.Wh-Focus ~ Complement Grammatical Construction Head | V | take Operator | Aux | did
54 Wh-Question – Double R Head | V | taken Wh-Quest-SRE Head | Wh-Pron | what Head | Pred-Trans-Verb Obj | Bind 1 Operator | Aux | could Wh-Quest-SRE Wh-Focus Op Subj (Spec) Head Pred-Trans-Verb Head Obj Wh-Focus | Wh-ORE 1 What could he have taken? Subj | ORE | Head | Pron | he Spec | Aux | have
55 Wh-Question + Passive + Ditrans – Double R Head | V | given Wh-Quest-SRE Head | Pred-Ditrans-Verb IObj | Bind 2 Operator | Aux | could Wh-Quest-SRE Wh-Focus Op Subj (Spec) Head Pred-Ditrans-Verb Head (IObj xor ) Obj (Recip xor ) Wh-Focus | Wh-ORE 1 | Head | Wh-Pron inan | what What could he have been given? Subj | ORE 2 | Head | Pron human | he Spec | Aux have been Obj | Bind 1 Grammatical Construction Animacy determines binding!
56 Wh-Question + Passive + Ditrans – Double R Head | V | given Wh-Quest-SRE Head | Wh-Pron human | who Head | Pred-Ditrans-Verb IObj | Bind 1 Operator | Aux | could Wh-Quest-SRE Wh-Focus Op Subj (Spec) Head Pred-Ditrans-Verb Head (IObj xor ) Obj (Recip xor ) Wh-Focus | Wh-ORE 1 Who could it have been given? Subj | ORE 2 | Head | Pron inan | it Spec | Aux have been Obj | Bind 2 Animacy determines binding!
57 Wh-Question + Passive + Ditrans – Double R Head | V | given Wh-Quest-SRE Head | Wh-Pron | who Head | Pred-Ditrans-Verb Recip | To-LRE Operator | Aux | could Wh-Quest-SRE Wh-Focus Op Subj (Spec) Head Pred-Ditrans-Verb Head (IObj xor ) Obj (Recip xor ) Wh-Focus | Wh-ORE 1 Who could it have been given to? Subj | ORE 2 | Head | Pron | it Spec | Aux have been Obj | Bind 2 P | to Obj | Bind 1
58 Grammatical Features of Nominals in English Definiteness – definite, indefinite, universal Number – singular, plural Animacy – human, animate, inanimate Gender – male, female Person – first, second, third Case – subj, obj, gen (2)
59 Why We Need Grammatical Features Definiteness: – Give me the ball (definite) – Give me a ball (indefinite) Number – The men (plural) kick the ball (sing). They (plural)… Animacy – The man (human) kicks the ball (inanimate). It (inanimate)… Gender – The man (male) likes the woman (female). She (female)… or
60 Simple Nominal the man singular human male “the” projects definite to obj-refer-expr “man” projects singular, human and male definite
61 Grammatical Features of Clauses in English Tense – present, past, non-finite Aspect – perfect, progressive Modality – “could”, “should”, “must”… Polarity – negative Voice – active, inactive, passive
62 Simple Clause …could not have gone finite present perfect active “could” negative “could” projects finite present tense and modality “not” projects negative polarity “have gone” projects perfect aspect and active voice “could not” recognized as a multi-word unit
63 Summary Representations matter! Language is complex! In complex systems, overall coherence is more important than overall simplicity! – Einstein: make your theory as simple as possible, but no simpler! – Computational implementation necessitates coherence If axioms + logical reasoning incoherence or a system that is obviously false, then question your axioms or your “logical” reasoning – E.g. if innateness assumptions lead to overly complex representations, then question the innateness assumptions or the reasoning
64 Theoretical Foundations Language Representation and Processing Double R Grammar – Cognitive Linguistic theory of the grammatical encoding of referential and relational meaning Double R Process – Psycholinguistic theory of the processing of English text into Double R Grammar based representations Double R Model – Computational implementation using the ACT-R cognitive architecture and modeling environment DoubleRTheory.com
65 Double R Process Serial, incremental, pseudo-deterministic language processor with a non-monotonic context accommodation mechanism (with limited parallelism) that is capable of making modest changes to the evolving representation Parallel, interactive, highly context sensitive, probabilistic mechanism which uses all available information to make the best choice at each choice point Processor presents the appearance and efficiency of deterministic processing, but is capable of handling the ambiguity which makes truly deterministic processing impossible
66 Double R Process Construction Driven Language Processing – Activation, Selection and Integration of constructions corresponding to the linguistic input Lexical items in the input activate constructions – Activation depends on current input, current context, and prior history of use – “give” activates ditransitive verb construction Most highly activated construction is selected Selected construction is integrated with evolving representation
67 Double R Process Adhere to well-established cognitive constraints on Human Language Processing Don’t use any obviously cognitively implausible mechanisms! Adhering to cognitive constraints may actually facilitate the development of functional NLP systems – Pushes development in directions that are more likely to be successful given inherently human nature of language processing – You don’t know what you’re giving up when you adopt cognitively implausible mechanisms!
68 ACT-R Cognitive Architecture Theory of human cognition based on 40+ years of psychological research (Anderson, 2007) – Computational implementation since 1993 Combines a symbolic procedural memory implemented as a production system with a symbolic frame based declarative memory (DM) Includes modules for vision, audition, and motor processing – Supports interaction with external world
69 ACT-R Cognitive Architecture Procedural memory is the central component – All modules interface to procedural memory via buffers (e.g. goal buffer, retrieval buffer, visual buffer) Productions have “subsymbolic” utilities – Productions match against buffers of other modules – Intentional module goal buffer is primary driver of behavior – Matching production with highest utility is selected for execution
70 ACT-R Cognitive Architecture DM contains chunks which are frame based – Chunk type + slot-value pairs (aka AVMs) Chunk types are organized into a single inheritance hierarchy Chunks have “subsymbolic” activations based on current input, current context and prior history of use Chunks are retrieved from memory by execution of a production which specifies a retrieval template – DM chunk with highest activation that matches retrieval template is retrieved (soft constraint retrieval)
72 ACT-R Cognitive Architecture Supports timing of cognitive processing – Production execution takes 50 ms – DM chunk retrieval time depends on level of activation of retrieved chunk – Timing of motor events based on Fitts’ Law – Used for empirical validation of models Provides a powerful debugging environment
73 Architectural Constraints No language specific module – although buffers and productions accessing buffers might be viewed as a module Forward chaining productions with no backtracking Limited pattern matching – not full unification Serial bottleneck – only one production can execute at a time Modules interact with production system via buffers – buffers have limited capacity for storing current context Activation spreads in parallel Activation and Utility subject to noise
74 Constraints on Human Language Processing Visual World Paradigm (Tanenhaus et al. 1995) – Subjects presented with a visual scene – Subjects listen to auditory linguistic input describing scene Immediate determination of meaning – Subjects look immediately at referents of linguistic expressions, sometimes before end of expression Incremental processing Interactive processing (Trueswell et al. 1999) – Ambiguous expressions are processed consistent with scene “the green…” “put the arrow on the paper into the box”
75 According to Crocker (1999), there are three basic mechanisms for dealing with ambiguity in natural language – Serial processing with backtracking or reanalysis – Deterministic processing with lookahead (Marcus 1980) – Parallel processing with alternative analyses carried forward in parallel (Gibson 1991; MacDonald, Pearlmutter & Seidenberg 1994; Trueswell & Tanenhaus 1994) According to Lewis (2000) “…existing evidence is compatible only with probabilistic serial-reanalysis models, or ranked parallel models augmented with a reanalysis component.” According to Gibson & Pearlmutter (2000) “noncompetitive ranked parallel models” are most consistent with the empirical evidence Constraints on Human Language Processing
76 Serial and deterministic with reanalysis for pathological input – Empirical evidence that we don’t carry forward all representations in parallel – Garden Path Sentences “The horse raced past the barn fell” (Bever 1970) “The old train the young” (Just & Carpenter, 1987) – Empirical evidence that we don’t retract previously built representations (Christianson et al. 2001) “While Mary dressed the baby sat up on the bed” – In a post test, a majority of subjects answered yes to the question “Did Mary dress the baby?” – Processing doesn’t slow down with increasing length of non-pathological input – Typically only aware of a single interpretation Constraints on Human Language Processing
77 Parallel and probabilistic with reanalysis for pathological input – Empirical evidence that we may carry forward multiple representations in parallel – Garden Path Effects can be eliminated with sufficient context – Empirical evidence that dispreferred representations can affect processing time (Gibson & Pearlmutter 2000) It’s extremely difficult to empirically falsify either – Could be parallel slow down or occasional switch between serial alternatives that causes effect Don’t have all the answers, but maybe it’s both! – A parallel, probabilistic substrate may make a pseudo- deterministic serial processing mechanism possible! Constraints on Human Language Processing
78 Cognitively Implausible Mechanism Serial processing with algorithmic backtracking – Algorithmically simple, but… Computationally intractable for NLP which is highly ambiguous Context which led to dead end is retracted on backtracking – Why give up the context? – How do we know it’s a dead end? Practical Consequences – No hope for on-line processing in real-time in large coverage NLP system – No hope for integration with speech recognition system – Performance degrades with length of input – Can’t easily handle degraded or ungrammatical input
79 Cognitively Implausible Mechanism Multiple pass or multi-stage parsing – Separate passes tokenize and assign part of speech Can’t use full context in each pass Errors get propagated – Separate pass builds structure Typically limited to using part of speech of words – Separate pass determines meaning Practical Consequences – Difficult to do on-line processing in real-time – Can’t easily integrate with speech recognition – Performance degrades with length of input – Limited context available to handle ambiguity at each stage
80 Outrageously Implausible Mechanism! Parsing input from right to left (Microsoft NLP system) – May have engineering advantages, but… Presumes a staged approach to NLP Completely ignores cognitive plausibility Practical consequences Impossible to do on-line processing in real-time – Must wait for end of input Nearly impossible to integrate with speech recognition
81 Cognitively Plausible Mechanism? Deterministic processing with lookahead – Many ambiguities resolved by looking ahead a few words, but… Don’t know how far to look ahead – Cognitive plausibility improved by limiting amount of lookahead 3 constituent lookahead (Marcus 1980) 1 word lookahead (Henderson 2004) Practical consequences – Difficult to use with eager algorithms for which there is good empirical evidence (immediate determination of meaning) – The smaller the lookahead, the less deterministic
82 Cognitively Plausible Mechanism? Parallel processing with multiple analyses carried forward – “Full parallelism – where every analysis is pursued – is not psychologically possible” (Crocker 1999) – Cognitive plausibility improved by limiting number of analyses carried forward and ranking alternatives (bounded ranked parallelism) and not having analyses compete Practical Consequences – The longer and more ambiguous the input, the less likely to have the correct representation in the parallel spotlight – necessitating a reanalysis mechanism – Impractical if multiple representations must be built at each choice point as opposed to just being selected
83 Cognitively Plausible Mechanism Pseudo-deterministic, serial processing mechanism with context accommodation operating over a parallel, probabilistic substrate – Parallel, probabilistic substrate proposes best alternative given current context – Processor proceeds as though it were serial and deterministic, but accommodates the subsequent input as needed – Integrates the advantages of parallel processing with an essentially serial processing mechanism Practical Consequences – How to accommodate when things go seriously wrong? – Mechanism is essentially non-monotonic
84 Cognitively Plausible Mechanism Serial, Pseudo-deterministic processing and Context Accommodation – Uses ACT-R’s production system – Builds structure – Limited parallelism Parallel, Probabilistic processing – Uses ACT-R’s declarative memory – Retrieves existing structure from memory
85 If current input is unexpected given the prior context, then accommodate the input – Adjust the representation – Coerce the input into the representation The following example demonstrates the context accommodation mechanism – “no target airspeed or altitude restrictions” Context Accommodation
86 no “no” object referring expression “no” projects obj-refer-expr and functions as specifier “head-indx” indicates head expected “bind-indx” provides index for binding
87 no target “target” head tree structures generated automatically with dynamic visualization tool (Heiberg, Harris & Ball 2007) based on phpSyntaxTree software (Eisenberg & Eisenberg) integration
88 no target airspeed “airspeed” head Accommodation of second noun via function shift and overriding override function shift integration
89 no target airspeed or altitude “or altitude” conj Conjunction integrated into noun integration
90 no target airspeed or altitude restrictions “restrictions” head Appearance of parallel processing! Accommodation of new head via function shift and override function shift override integration
91 Coercion – “the hiking of Mt Lemon” – head of nominal “hiking” construed objectively, arguments not expressed (“of Mt Lemon” functions as a modifier) – “a Bin Laden supporter” Proper Noun functions as modifier – “you’re no Jack Kennedy” Proper Noun functions as head (following specifier) – “the newspaper boy porched the newspaper” – nonce expression (H. Clark 1983) “porched” construed as transitive action Types of Accommodation
92 Override – Single word vs. Multi-Word Expression (MWE) “kicked…” transitive verb – “kicked the bucket” idiomatic expression “take…” transitive verb – “take a hike” “take five” “take time” “take place” “take out” “take my wife, please” “take a long walk off a short pier” … many idiomatic expressions Not possible to carry all forward in parallel – Morphologically simple vs. complex “car…” noun (sing) – “carpet…” noun (sing) – “carpets” noun (plur) – “carpeting” noun (sing) or verb Types of Accommodation
93 Grammatical Function Shift – “he gave it to me” direct object (initial preference due to inanimacy) – “he gave it the ball” direct object (initial preference) indirect object – “he gave her the ball” indirect object (initial preference due to animacy) – “he gave her to the groom” indirect object (initial preference) direct object Types of Accommodation
94 Nominal Head Override “he gave her the dog biscuit” head = her “he gave her dog the biscuit” head = dog Grammatical Function “Juggling” – “he gave the…” indirect object – “he gave the very old bone…” direct object – “he gave the very old bone collector…” indirect object – “he gave the very old dog…” indirect object – “he gave the very old dog collar…” direct object – “he gave the very old dog to me” direct object Types of Accommodation
95 Grammatical Function Shift – “he said that…” In context of “said”, “that” typically functions as a complementizer – But subsequent context can cause a function shift from complementizer – “he said that she was happy” To nominal specifier to – “he said that book was funny” To nominal head – “he said that.” Types of Accommodation
96 Grammatical Function Shift – “pressure” vs. “pressure valve” vs. “pressure valve adjustment” vs. “pressure valve adjustment screw” vs. “pressure valve adjustment screw fastener” vs. “pressure valve adjustment screw fastener part” vs. “pressure valve adjustment screw fastener part number” Serial nouns (and verbs) incrementally shift from head to modifier function as each new head is processed Functions like lookahead, but isn’t limited Not clear if a bounded ranked parallel mechanism can handle this! – 2 n possibilities if head or modifier at each word Types of Accommodation
97 Modulated Projection – “the rice” vs. “rice” – “the” projects a nominal and functions as a specifier – In the context of “the” “rice” is integrated as the head of the nominal – When there is no specifier, “rice” projects a nominal and functions as the head without separate specification Types of Accommodation nominal spec the head rice vs. head rice nominal “the rice”“rice”
98 Grammatical Feature Accommodation Grammatical features may be redundantly encoded and may conflict without the expression being ungrammatical – a indef+sing few indef+plur books indef+plur – the def books indef+plur – some indef+plur book sing – a indef+sing Ronald Reagan def+sing republican sing – he has pres+act given pass+perf me the ball – he is pres+inact given pass+perf the ball – he has pres+act been pass+perf given pass+perf it
99 Context Accommodation is part and parcel of the pseudo-deterministic processing mechanism – Not viewed as a repair mechanism (Lewis 1998) Processor proceeds as though it were deterministic, but accommodates the input as needed Gives the appearance of parallel processing in a serial, deterministic mechanism Summary of Context Accommodation
100 Combining Serial, Deterministic and Parallel, Probabilistic Mechanisms Tree Supertagging Construction Activation & Selection Supertag Stapling Construction Integration Rule ApplicationLexical Rule Selection Rule Selection Rule Application Rule Selection & Application Parallel Probabilistic Serial Deterministic Parallel Distributed Processing CFG PCFG Lexicalized PCFG Double R Probabilistic LTAG PDP Pseudo Deterministic Range Nondeterministic The parallel probabilistic substrate makes a pseudo-deterministic serial processing mechanism possible!