Presentation is loading. Please wait.

Presentation is loading. Please wait.

Mémoire, Apprentissage et Langage: Aspects Cognitifs Peter Ford DOMINEY Institut des Sciences Cognitives, CNRS UMR 5015, UCB Lyon 1.

Similar presentations


Presentation on theme: "Mémoire, Apprentissage et Langage: Aspects Cognitifs Peter Ford DOMINEY Institut des Sciences Cognitives, CNRS UMR 5015, UCB Lyon 1."— Presentation transcript:

1 Mémoire, Apprentissage et Langage: Aspects Cognitifs Peter Ford DOMINEY Institut des Sciences Cognitives, CNRS UMR 5015, UCB Lyon 1

2

3 Plan Langage: Constructions grammaticales Structure des séquences sensorimotrices Réseau Neuronal « Equivalence » apprentissage langage et séquences abstraites Bases neurophysiologiques Ré-éducation Apprentissage multi-langue Apprentissage robotique Fin

4 Constructions Grammaticales et mots de fonction : n Same meaning different syntactic structure n Different transformation from sentence to meaning n Signaled by function words (the, was, to, by) n Function words trigger transformation/mapping rules to associate open class nouns to their thematic roles (agent, object, recipient). The dog gave the elephant to the monkey. Gave (Agent, Object, Recipient) The elephant was given to the monkey by the dog.

5 Structure Abstraite et Constructions Grammaticales Constructions Grammaticales –Mapping Langage –Ensemble structuré de constructions grammaticales Correspondance entre –Constructions grammaticales & –Structures abstraites Hypothèse dÉquivalence Forme: NP 1 V T NP 2 à NP 3 Sens: Act T (Agent, Obj, Recip) Forme: Jean donne la balle à Marie Sens: Donne(Jean, Balle, Marie) Construction Di-transitif J D B M – D J B M Structure Abstraite « Di-Transitif » Dominey Evolution of Communication 02, Dominey et al., Brain and Language 03 J D B M D J B M

6 Structure des Séquences Sensorimotrices 1.Sérielle (ordre) –ABCBACB –CABACBA 2.Temporelle (rythme) –A BCAB AC A 3.Abstraite (règle) –ABC-BAC –DEF-EDF –…. –123 – 213 Selection WM ABCABC Recog State Output Input Cortex Recurrent Connections Striatum Reward-related dopamine synaptic plasticity A B C BAC Prefrontal Working Memory 1.Dominey et al. JCogNeuro 95, Dominey BioCybernetics 95 Dominey & Boussaoud CogBrRes 97 2.Dominey JEP 98, Dominey CogBrainRes 98 3.Dominey Brain&Lang 97, Dominey et al. JCogNeuro 98, Dominey & Ramus Lang & Cog Proc. 2000, Dominey et al. 1997, Lelekov et al Réseau Neuronal

7 Model learns to perform thematic role assignment nTraining: n Present sentence one word at a time n Indicate: agent, object and recipient n Learn transformational relations so State can Modulate appropriate STM to Output nTesting & Correction: n Present new sentences n Model emits the correct roles n Error-Learning n100% Correct Caplan State Selection Output Input The elephant was given to the monkey by the dog. Closed Class Stream (Function): the, was, to, by Open Class Stream (Content): Canonically ordered nouns: agent(dog), object(elephant), recipient(monkey) STM elephant monkey dog Recog Dominey, et al, Brain and Language, 2003.

8 Theoretical Implications Functional relation between linguistic and non-linguistic abstract sequence processing Common neurophysiological basis for rule-based structural transformation processing ABCx Closed Class Stream (Function): x Open Class Stream (Content): A,B,C State Selection Output STM/Rec Input Symbols in transformed order: BAC The elephant was transferred to the monkey by the dog. Closed Class Stream (Function): the, was, to, by Open Class Stream (Content): elephant, dog, monkey Canonically ordered nouns: agent(dog), object(elephant), recipient(monkey) State Selection Output STM/Rec Input

9 Corrélats Neuro-Linguistiques: chez des patients aphasiques agrammatiques 1.Perturbations Corrélées entre Séquences Abstraites - Langage 2.Transfert dapprentissage entre Séquences Abstraites - Langage 1. Perturbations Corrélées Lelekov, Dominey et al. 00; Dominey et al. Brain&Language 03, Hoen et al. NeuroReport Transfert dapprentissage

10 Corrélats Neuro-Linguistiques: chez des patients aphasiques agrammatiques

11 Language Abstract Sequences ERP Common Cortical Network Langage/Abstract Seq LAN, P600, N400 ABCXBAC vs ABCYABC (Michel Hoen - Thesis 2003) Neurophysiology - ERP: Neurophysiology - ERP: Shared cortical networks for abstract sequence and language Compare Sentence and Sequence Processing for closed class elements Sentence: The faucet was soldered by the plumber Sequence: ABC X BAC

12 Corrélats Neurophysiologiques en IRM fonctionnelle Réseau cortical commun pour le traitement des transformations structurelles Réseau cortical distinct pour lintégration de la semantique dans ces transformations Newman 2003 IRMf LangageLangage & Seq Hoen, Dominey et al. Cortex 2005 – special issue on Integrative models of Brocas area and the ventral premotor cortex

13 Modèle Actualisé: Traitement des Constructions Grammaticales IRMf donné Marie balle Jean ACTION AGENT OBJECT RECIPIENT __ a été __ à __ par __ balle donné Jean Marie balle donné Jean Marie La balle a été donné à Jean par Marie OpenClassArray Predicted Referents Array SceneEvent Array ConstructionIndex Construction Inventory SentToSceneWordToReferent Langage & Séquence (Pars opercularis) Ba 44 Transformations Structurelles Langage spécifique (Pars triangularis) Ba 45 Intégrations Sémantiques B A C __ __ __ X __ __ __ A B C ABC X BAC ConstructionIndex Construction Inventory Dominey & Hoen Cortex 2005 Special Issue on Integrative Models of Brocas Area LangageSéquences

14 Grammar and Grammatical Constructions Learned 1. S NP + VP 2. NP Det + N 3. NP NP+Rel+ VP 4. VP Va + NP 5. VP Va + NP + PP 6. VP Aux + Vp + PP 7. VP Aux + Vp + PP + PP 8. PP Prep + NP 9. Det the, a 10. N cylinder, block, moon 11.Va touched, pushed, took, gave 12.Vp touched,pushed, taken, given 13. Aux was 14. Prep to, by, from 15. Rel that The block pushed the triangle. The block gave the triangle to the moon. The triangle was pushed by the block. The moon was given to the triangle by the block. The block that pushed the triangle touched the moon. The block pushed the triangle that touched the moon. The block pushed the triangle that was touched by the moon. The block that was pushed by the triangle touched the moon. 35+ syntactic structures

15 Cross-Linguistic Validation: Japanese Free word order Grammatical roles of NPs indicated by post-positions Will the cue competition model adapt to this configuration? -ga, -ni, -yotte, -to and – wo markers as closed class elements

16 Cross-Linguistic Validation: Japanese Block-ga circle-wo oshita triangle-ni-yotte tatakareta. Circle-wo oshita triangle-ni-yotte block-ga tatakareta. The block was hit by the triangle that pushed the circle. Pushed(triangle, circle), Hit(triangle, block) Circle-wo tataita block-ga triangle-ni-yotte osareta. Triangle-ni-yotte circle-wo tataita block-ga osareta. The block that hit the circle was pushed by the triangle. Hit(block, circle), Pushed(triangle, block) Block-ga circle-wo oshita triangle-wo tataita. Circle-wo oshita triangle-wo block-ga tataita. The block hit the triangle that pushed the circle. Pushed(triangle, circle), Hit(block, triangle) N = 26 constructions

17 Using These Learned Grammatical Constructions for Event Description and Interrogation Human narrator Vision, "Meaning" Extraction Spoken Language Interface (CSLU RAD) Grammatical Construction Model: Sentence to Meaning The moon gave the cylinder to the block. The block was gave the cylinder by the moon. The cylinder was gave to the block by the moon. Event(Agent, Object, Recipient) E(A,O,R) Sentence CCD Camera Gave(moon, cylinder, block)

18 Grammatical Constructions for Robot Command Grammatical constructions –Link between command meaning (predicate-argument structure) and sentence structure –Sentence - Meaning mappings Event description –Action(Agent, object, recipient) –Sentence Commanding –Sentence –Move(Object, location)

19 Plan Langage: Constructions grammaticales Structure des séquences sensorimotrices Réseau Neuronal « Equivalence » apprentissage langage et séquences abstraites Bases neurophysiologiques Ré-éducation Apprentissage multi-langue Apprentissage robotique Fin


Download ppt "Mémoire, Apprentissage et Langage: Aspects Cognitifs Peter Ford DOMINEY Institut des Sciences Cognitives, CNRS UMR 5015, UCB Lyon 1."

Similar presentations


Ads by Google