Presentation is loading. Please wait.

Presentation is loading. Please wait.

B RAIN SIGNATURES OF GAME PLAY AND LANGUAGE PROCESSING Robert V. Reichle Department of Foreign Languages Center for the Interdisciplinary.

Similar presentations


Presentation on theme: "B RAIN SIGNATURES OF GAME PLAY AND LANGUAGE PROCESSING Robert V. Reichle Department of Foreign Languages Center for the Interdisciplinary."— Presentation transcript:

1 B RAIN SIGNATURES OF GAME PLAY AND LANGUAGE PROCESSING Robert V. Reichle Department of Foreign Languages Center for the Interdisciplinary Study of Language and Literacy Northern Illinois University

2 For late L2 learners, L2 processing is difficult Corresponding features or constructions in the native language do not ensure success in the L2 Variability is typical Native-like attainment is not typical Previously thought to relate to putative critical period for L2 acquisition L2 PROCESSING

3 Among other explanations for divergence in L1/L2 processing, Ullman’s (2001, 2004) Declarative/Procedural model Two domain-general memory systems Declarative: facts, associations between form and meaning, non-compositional words Procedural: rule-governed processes, morphosyntactic composition, cognitive-motor skills Some L2 learners become more nativelike as they come to rely more on procedural memory in the L2 L2 PROCESSING

4 (e.g., graphs from Osterhout et al., 2006, p. 204 and Neville et al., 1991; Friederici, Steinhauer, & Frisch, 1999; Friederici, 2002; Hahne & Friederici, 1999, 2001; Hagoort & Brown, 1999; Kaan et al., 2000; Kutas & Hillyard, 1980; Osterhout, Bersick, & McLaughlin, 1997; Osterhout & Holcomb, 1992; Osterhout & Mobley, 1995; Steinhauer, Halter, & Friederici, 1999; Friederici, Pfeifer, & Hahne, 1993; Kluender & Kutas, 1993; Münte, Heinze, Matzke, Wieringa, & Johannes, 1998) Lexical/semantic processing: N400 Morphosyntactic/grammatical processing: LAN EVENT-RELATED POTENTIALS (ERPs) Differential ERPs for L1 syntax/semantics interpreted as evidence for DP model (Ullman, 2004)

5 Proficiency effects on sensitivity to inflectional morphology and syntax Artificial language learning: with language training, N400  LAN/P600 - (e.g., Friederici, Steinhauer, & Pfeifer, 2002; Morgan-Short, Sanz, Steinhauer, & Ullman, 2010; Morgan- Short, Steinhauer, Sanz, & Ullman, 2012; Morgan-Short, Finger, Grey, & Ullman, 2012) Real language learning: with increasing proficiency, N400  LAN/P600 - (e.g., Bowden et al., 2007; Hahne et al., 2006; Osterhout et al., 2006; Rossi et al., 2006; Steinhauer et al., 2006; Gillon Dowens, Vergara, Barber, & Carreiras, 2010; McLaughlin, Osterhout, & Kim, 2004; McLaughlin, Tanner, Pitkänen, Frenck-Mestre, Inoue, Valentine, & Osterhout, 2010; Tanner, Osterhout, & Herschensohn, 2009; Ojima, Nakata, & Kakigi, 2005) EVENT-RELATED POTENTIALS (ERPs)

6 Proficiency effects on sensitivity to inflectional morphology and syntax Artificial language learning: with language training, N400  LAN/P600 - (e.g., Friederici, Steinhauer, & Pfeifer, 2002; Morgan-Short, Sanz, Steinhauer, & Ullman, 2010; Morgan- Short, Steinhauer, Sanz, & Ullman, 2012; Morgan-Short, Finger, Grey, & Ullman, 2012) Real language learning: with increasing proficiency, N400  LAN/P600 - (e.g., Bowden et al., 2007; Hahne et al., 2006; Osterhout et al., 2006; Rossi et al., 2006; Steinhauer et al., 2006; Gillon Dowens, Vergara, Barber, & Carreiras, 2010; McLaughlin, Osterhout, & Kim, 2004; McLaughlin, Tanner, Pitkänen, Frenck-Mestre, Inoue, Valentine, & Osterhout, 2010; Tanner, Osterhout, & Herschensohn, 2009; Ojima, Nakata, & Kakigi, 2005) EVENT-RELATED POTENTIALS (ERPs)

7 BROCANTO TRAINING GAME Artificial language (BROCANTO) Fulfills syntactic requirements of natural language Contains morphosyntactic features that differ from L1 of participants Participants trained in lexicon of artificial language One group trained to proficiency in BROCANTO via a two-player computer board game Played against each other in pairs Orally expressed all game moves in BROCANTO Received feedback on valid game moves and grammatical BROCANTO utterances The structure of the language not dependent on the structure of the game  (Friederici, Steinhauer, & Pfeifer, 2002; for similar BROCANTO2 design, see also Morgan-Short, Sanz, Steinhauer, & Ullman, 2010; Morgan-Short, Steinhauer, Sanz, & Ullman, 2012; Morgan-Short, Finger, Grey, & Ullman, 2012)

8 BROCANTO TRAINING GAME ERPs ERP data collection using syntactically felicitous and anomalous items took place after participants in the training group reached a 95% accuracy threshold. Training group had biphasic response to violations that resembled LAN/P600 of L1 processing aaf trul prez nöri aak füne plox the trul-piece captures horizontally the round plox- piece aaf trul prez nöri *rix füne ploxthe trul-piece captures horizontally *buy round plox-piece

9 BROCANTO TRAINING GAME Why was this a successful training paradigm?

10 BROCANTO TRAINING GAME Why was this a successful training paradigm? Gee (2003): active, critical learning cooperation competition a requirement of successful learning in order to progress feedback meta-awareness of gameplay mechanics

11 BROCANTO TRAINING GAME Why was this a successful training paradigm? Declarative/Procedural model nativelike syntactic processing should be handled by procedural memory learning a cognitive-motor skill (like the rules of the training game) should also be handled by this system Did these two types of learning engage similar cognitive processes? Does “words and rules” have an equivalent “sprites and rules” (Reichle, 2012)? P600 not necessarily fully language-specific — also observed for structural revision in music (e.g. Patel et al., 1998)

12 R ESEARCH QUESTIONS Does the execution of gameplay skills engage the same processes as syntactic processing? Specifically, examining violations of syntax and violations of expected gameplay behavior. If the same processes, then could aid learning, but could also present confound to interpretation of results. Does processing linguistic stimuli in the context of a game lead to a task effect compared to linguistic stimuli in isolation?

13 H YPOTHESES Violations of expected gameplay behavior will elicit LAN and P600 effects comparable to those seen for violations of morphosyntax. Processing violations of morphosyntax in the context of a game causes a task effect compared to violations of morphosyntax in isolation.

14 ERP E XPERIMENT 1 23 participants (5 rejected due to artifacts) native speakers of English pre-screened for knowledge of rules of movement in chess EEG data were recorded from nine sites (F3, Fz, F4, C3, Cz, C4, P3, Pz, P4)Four conditions A: Control. Legal moves followed by grammatical sentences B: Morphosyntax violation. Legal moves followed by sentences with a morphosyntax violation C: Gameplay mismatch. A chess move that is legal for a piece other than the one shown in the image, followed by a grammatical sentence D: Gameplay violation. A chess move that is never legal for any piece, followed by a grammatical sentence

15

16

17 A: The white queen takes the black pawn.

18 B: The white queen *taking the black pawn.

19

20

21 C: The white queen takes the black pawn.

22

23

24 D: The white queen takes the black pawn.

25 ERP E XPERIMENT 1 Stimulus presentation each image onscreen for 1 sec 600ms pause word-by-word presentation: 290 ms plus 30 ms per letter, up to a maximum of 590 ms, with a 150 ms interval between words Analysis At fourth word (takes/*taking) Mean amplitude data in ms and ms windows Repeated-measures ANOVAs: 4 (Condition) x 3 (Anteriority) x 3 (Laterality)

26 F3 FzF4 C3CzC4 P3PzP4

27

28 Omnibus ms: Cond * Lat, p <.05 F3 FzF4 C3CzC4 P3PzP4

29 Morph. vs Control ms: Cond * Lat, p <.05 F3 FzF4 C3CzC4 P3PzP4

30 Game rule mismatch vs Control ms: NS ms: NS F3 FzF4 C3CzC4 P3PzP4

31 Game rule violation vs Control ms: Cond * Ant * Hem, p =.022 F3 FzF4 C3CzC4 P3PzP4

32 R ESULTS P600 effect for morphosyntax violations No effect for game rule mismatch Game rule violation Negativity effect, strongest in left and mid central and frontal sites Interpreted as possible variation of LAN Extended into later time window than typical biphasic LAN But, not biphasic — no P600 effect followed

33 D ISCUSSION Hypothesis: Violations of expected gameplay behavior will elicit LAN and P600 effects comparable to those seen for violations of morphosyntax. Not fully supported — no P600. Consistent with generally language-specific nature of P600. Partially supported by presence of LAN. But work still remains on reproducibility and functional significance of LANs (e.g. Steinhauer, Drury).

34 D ISCUSSION Implications Possibility remains that processing gameplay rules engages similar cognitive processes as processing morphosyntax (LAN) No evidence that gameplay elicits P600 Doubtful that P600s seen in BROCANTO studies were indices of gameplay-related processing

35 H YPOTHESES Violations of expected gameplay behavior will elicit LAN and P600 effects comparable to those seen for violations of morphosyntax. Processing violations of morphosyntax in the context of a game causes a task effect compared to violations of morphosyntax in isolation.

36 ERP E XPERIMENT 2 Show morphosyntactic anomalies within and without game context 2 groups Image group: Saw chess move after reading sentence NoImage group: Did not see chess moves 12 participants (6 per group) native speakers of English pre-screened for knowledge of rules of movement in chess EEG data were recorded from nine sites (F3, Fz, F4, C3, Cz, C4, P3, Pz, P4)

37 ERP E XPERIMENT 2 2 Conditions Control: The white queen takes the black pawn. Morphosyntax violation: The white queen taking the black pawn. 2 fillers Progressive filler: The queen is taking the black pawn. Game rule violation: The black queen takes the white pawn.

38 ERP E XPERIMENT 2 Stimulus presentation word-by-word presentation: 290 ms plus 30 ms per letter, up to a maximum of 590 ms, with a 150 ms interval between words for Image group, presentation of game images 1 sec after final word Analysis At fourth word (takes/taking) Mean amplitude data in ms and ms windows Repeated-measures ANOVAs: 2 (Condition) x 3 (Anteriority) x 3 (Laterality)

39 Control Morphosyntax violation NoImage group

40 Control Morphosyntax violation Image group

41 Control Morphosyntax violation Between-group omnibus: ms: Cond * Group, p =.26 Image group

42 Control Morphosyntax violation Image group: ms: Cond, p = ms: Cond * Lat, p = ms: Cond * Lat * SRTT, p =.2 Image group

43 R ESULTS Trend toward between-group difference (task effect) Trend toward P600 for Image group Data collection still in progress — more participants needed

44 D ISCUSSION Hypothesis: Processing violations of morphosyntax in the context of a game causes a task effect compared to violations of morphosyntax in isolation. Preliminary results trend toward support of hypothesis. Attributable to “motivation” in the broadest sense. Additional studies needed to separate out Gee’s factors (competition, meta-awareness, etc.) from simpler explanations (i.e. more attentive due to presence of images)

45 CONCLUSIONS Evidence of an LAN for the processing of gameplay violations suggests gameplay may engage cognitive processes also used in language processing. Lack of P600 for gameplay processing indicates that L2 studies using game training paradigms are not likely to confound indices of gameplay with indices of morphosyntactic processing. Preliminary data suggest trend toward game-related task effect, suggesting increased motivation or attention within game context. Additional studies needed to further tease apart variables.

46 A CKNOWLEDGMENTS This research has been supported by the NIU Division of Research and Graduate Studies, the NIU Center for the Interdisciplinary Study of Language and Literacy, and the NIU Center for the Study of Family Violence and Sexual Assault. The author thanks Cody Happ and Regina Hiraoka for their assistance with materials creation and data collection, and all participants for their cooperation.

47 T HANK YOU


Download ppt "B RAIN SIGNATURES OF GAME PLAY AND LANGUAGE PROCESSING Robert V. Reichle Department of Foreign Languages Center for the Interdisciplinary."

Similar presentations


Ads by Google