Presentation on theme: "Task-internal and task-external readiness: A report of the effects of topic familiarity and strategic planning on task performance by L2 learners of different."— Presentation transcript:
Task-internal and task-external readiness: A report of the effects of topic familiarity and strategic planning on task performance by L2 learners of different proficiency levels Gavin Bei Xiaoyue The Chinese University of Hong Kong firstname.lastname@example.org 3rd Biennial International Conference on Task-Based Language Teaching, Lancaster University, Sept 15, 2009
Contextualizing Task-based instruction research looks at: 1. Task characteristics: Subjective or objective, structural or non-structural, familiar or unfamiliar… 2. Task conditions: Monologic or interactive, Pre/Post task activities, planning or non-planning… 3. Participants: Gender, motivation, learning style, proficiency…
Research background 1: topic familiarity 1. Comprehension (many) facilitative e.g., Shimioda, 1993; Barry and Lazarte, 1995; Bügel & Buunk, 1996; Chen and Donin, 1997; Johnson, 1982; Lee, 1986; and Chang, 2006 no effect on comprehension e.g., Hammadou, 1991; Peretz & Shoham, 1990; and Carrell (1983) 2. Production (few) Mostly in L1 research by psychologists Higher fluency, but inconsistent in accuracy or complexity
Research Background 2: planning types Two macro and four micro types of planning (Ellis, 2005) 1. pre-task 1) rehearsal 2) strategic planning 2. within-task 1) pressured 2) unpressured Or simply three micro types (Ellis, in press) 1) rehearsal 2) strategic planning 3) within-task planning
Research background 3: strategic planning Ample studies (e.g., Skehan, Foster, Ellis, Crookes, Wigglesworth, etc.) with quite some consistent results. Planning raises: Fluency + Complexity (sometimes, but usually not) Accuracy Skehan: Trade-off of between Comp. and Accu. Robinson: Planning does not lead to Comp, no trade-off. Is proficiency important here?
Research background 4: Proficiency and Familiarity Hudson (1982): In Reading: Familiarity > Proficiency. Schmidt-Rinehart (1994): In Listening: Familiarity > Proficiency. Carrell (1983): In Reading: Proficiency > Familiarity (NS:NNS) Chern (1993): In Reading: Proficiency > Familiarity.
Research background 5: proficiency and planning Wigglesworth (1997): low proficiency did not benefit from planning. Tavokoli and Skehan (2005): planning drove high and low learners for better performance. Kawauchi (2005): more Flu. and Comp. for higher learners, more Accu. for the lower. The advanced gained the least. Most other studies did not consider proficiency.
1.Participants and proficiency test Participants: 80 HK Cantonese- speaking undergraduates volunteers to participate. A C-test as proficiency test to group participants ------- borrowed from Dornyei and Katona (1992). --------The validity and reliability are good in the literature and in the present context. See appendix 1
3. Independent Variables 1. Topic familiarity (within): 2 levels familiar VS unfamiliar task 2. Planning (between): 2 levels non-planning VS 10-min planning 3. Proficiency (between): 2 levels intermediate VS high
Study design Planning (between) Proficiency (between) Topic familiarity (within) FamiliarUnfamiliar Planners High20 Intermediate20 Non- planners High20 Intermediate20 Each cell consists of 10 computer majors and 10 medicine majors as counterbalancing to rule out the topic effect.
4. Dependent Variables Fluency: pausing, speech rate, MLR, phonation time, repairs, etc. Accuracy: error-free clauses ratio, length of correct clause, and errors per 100 words. Complexity: Clauses per AS unit, AS unit length, and clause length Lexis: lexical diversity, lexical sophistication, and lexical density. Formality: F-score, DB-score Appendix 2 Appendix 2Totally 21 measure were employed. See Appendix 2 for a detailed description.Appendix 2 P value: the significance level to tell whether there is an effect. Cohens D value: the effect size to tell how big the effect is.
5. Statistical procedures A repeated measures analysis of variance (ANOVA) was employed.
Breakdown fluency main effects 1 T. FamiliarityPlanningProficiency pDpDpD Speech rate.000.26.000.58 ns/ Phonation time.000.35.000.62 ns/ Mean length of run.016.17.046.32 ns/ No. Mid-clause pauses.000.38.001.54 ns/ No. End-of-clause pauses ns/ /.027.48 Table 1. p and Cohens D value Means omitted due to the space limit. The means show that the familiar topics and the planning time improve fluency. Same directions below unless there is a note.
Breakdown fluency main effects 2 T. FamiliarityPlanningProficiency pDpDpD Mid-clause silence total.000.38.000.61 ns/ End-of-clause silence ttl..014.19.007.59 ns/ Av. mid-clause pause.019.28.000.71 ns/ Av. End-of-clause pause ns/.004.64 ns/ No. filled pauses.085 /ns/ / Table 2. p and Cohens D value
T. Familiarity has significant interactions with Planning in: 1) speech rate 2) phonation time 3) No. Mid-clause pauses 4) Mid-clause silence total (per 100 words) 5) End-of-clause silence total (per 100 words) Breakdown fluency: interactions All showing one pattern: planning compensates for the unfamiliar topics.
Measures: speech rate and mid-clause silence total
Breakdown fluency: a summary 1) T. Familiarity affects fluency in a strikingly similar way as Planning does. 2) Approximately, the effect sizes of T. Familiarity is half as big as those of Planning. 3) Planning mitigate the difference between familiar and unfamiliar topics. 4) The effects of Proficiency is marginal, and probably overridden by T. F. and Planning.
Repair fluency T. FamiliarityPlanningProficiency pDpDpD False starts ns/.0001.02 ns/ Reformulations.088 /.001.53 ns/ Replacements.077 /.008.43 ns/ Repetitions.001.40.000.75 ns/ Table 3. p and Cohens D value Note: planning induced more replacements, though reducing others.
Accuracy Table 4. p and Cohens D value T. FamiliarityPlanningProficiency pDpDpD Error-free clauses ratio.02.22ns/.000.69 70% accuracy clause length ns/ /.000.57 Errors per 100 words.000.38ns/.000.77
Complexity Table 5. p and Cohens D value T. FamiliarityPlanningProficiency pDpDpD Clauses per AS unit ns/.018.39.067/ Words per AS unit ns/.000.81.000.52 Words per clause ns/ / /
Lexis Table 6. p and Cohens D value T. FamiliarityPlanningProficiency pDpDpD Lexical diversity.018.29ns/ / Lexical sophistication.000.41ns/ / Lexical density ns/.008.43.031.39
Formality Table 7. p and Cohens D value T. FamiliarityPlanningProficiency pDpDpD F-score.000.49.003.48ns/ DB-score ns/.019.38ns/
Conclusions 1 1. Planning is more powerful in driving fluency than T. Familiarity. It can reduce the differences between familiar and unfamiliar topics in breakdown fluency. 2. Topic familiarity and planning seem to be more concerned with meaning expression (similar). 3. T. familiarity and planning affect different syntactic areas (different). 4. Proficiency affects mostly forms, esp. accuracy, but not so much meaning expression (fluency and lexis). 5. Higher proficiency does not appear to remove the trade-off effects. So L2 learners are L2 learners!
Conclusion 2 6. Factor analyses of all measures show: 1) theres probably an end-of-clause fluency different from breakdown and repair fluency. (Av. Pause and total silence at the end of a clause, and phonation time. ) 2) theres probably a noun-phrase complexity as compared to the syntactic complexity. (words per clause, F-score, DB-score, and Lambda) 6. A broader perspective on planning stems from the similarities and differences between T. familiarity and strategic planning in this study, in which I argue that T.F. can be regarded as a kind of implicit planning (see next page).
A general framework of task-readiness Macro- dimension Micro-dimensionSample studies Learner readiness for a task Task- internal readiness (implicit planning) Topic familiarity (prior domain knowledge) This study Schematic familiarity (story structure) Skehan and Foster (1999) Task familiarity (task types) Bygate (2001) Task- external readiness (explicit planning) Rehearsal ( content repetition) Bygate (1996) Strategic (pre-task) planning Foster and Skehan (1996) Within-task (on-line) planning Yuan and Ellis (2003)
Appendix 1: reliability and validity of C-test Dornyei and Katona (1992) found that the C-test is reliable (the internal consistency coefficients are very consistent,.75 and.77 respectively, for university English majors and secondary students) and valid (C-test is significantly highly correlated with different other proficiency tests like the General Language Proficiency and TOEIC). Cronbachs alpha reached.84 in Daller and Phelan (2006). Klein-Braley and Raatz (1984), Klein-Braley (1985), Cohen, Segal and Bar-Siman-Tov (1984), Klei-Braley (1997), and Grotjahn, 1995 generally supported such a claim on written tasks. More importantly here, the C-test was reported to be highly correlated with oral tasks as well in recent studies (e.g., r=.64 in Arras, Eckes and Grotjahn, 2002, and also in oral lexical performance in Daller and Xue, 2007). More recently Dai (this conference) reported in Chinese context, Cronbachs Alpha=.770, Concurrent validity r=.633, p<0.01 (correlated with CET-4). In this study, the Cronbach Alpha is=.64 in the pilot study, but =.74 in the main study. back
Appendix 2: description of dependent variables General CategoryVariable NameDescription Fluency PausingThe number of pauses and the amount of silence. In the present study it is operationalized as any break of 0.4 second or longer. Repair FluencyThis measure is orthogonal to breakdown fluency and should be treated separtely. In the present study it is operationalized as the total number of repetitions, replacements, false starts and reformulations. Speech RateA pruned speech rate is investigated here because it shows the real speed of the speaker. It is operationalized as the total words per minute after deletion of reformulations, replacements, false starts, repetitions, pauses and silence total. Mean Length of RunThe number of words uttered before any breakdown or repair fluency is encountered. Phonation timeThe ratio of voicing time to the total time of utterance.
Accuracy Error-free RatioThe ratio of error-free clauses to all clauses. Errors per 100 WordsThe number of errors in every pruned one hundred words. Length AccuracyThe length of a clause with 50% of all clauses of the same length correct is set as the cut-off point beyond which the participant cannot produce correct clause at 50% level. Complexity Subordination RatioThe ratio of subordinate clauses per AS unit. Words Per AS UnitThe average word number in all AS units. Words Per ClauseThe Average word number in all clauses Lexis Lexical diversity: the D value Corrected Type-token ratio, an index of the extent to which the speaker avoid returning to the same set of words. Lexical sophistication: the Lambda value The extent to which speech contains difficult or rare words. Lexical DensityThe ratio of content words to the total words.
FormalityF-score(Noun frequency+adjective freq.+preposition freq.+article freq.-pron. Freq. –verb freq.- adverb freq. – interjection freq. +100) / 2 From F. Heylighen and J. Dewaele (1999). DB-scoreThe involved style words in Biber, Conrad and Reppen (1998). back
References Barry, S. & Lazarte, A. (1995). Embedded clause effects on recall: Does high prior knowledge of content domain overcome syntactic complexity in students of Spanish? Modern Language Journal, 79, 491–504. Biber, D., Conrad, S. & Reppen, R. (1998). Corpus Linguistics: Investigating Language Structure and Use. Cambridge: Cambridge University Press. Bügel, K. & Buunk, B. (1996). Sex differences in foreign language text comprehension: The role of interests and prior knowledge. Modern Language Journal, 80, 15–31. Bygate, M. (1996). Effects of task repetition: appraising the developing language of learners. In Jane Willis & Dave Willis (Eds). Challenge and change in language teaching. Oxford : Heinemann. Bygate, M. 2001. Effects of task repetition on the structure and control of oral language [A]. In M. Bygate, P. Skehan & M. Swain (eds). Researching Pedagogical Tasks: Second Language Learning, Teaching and Testing [C]. Harlow, England: Longman. Carrell, P. L. (1983). Three components of background knowledge in reading comprehension. Language Learning, 33, 183-207. Chang, C. (2006). Effects of topic familiarity and linguistic difficulty on the reading strategies and mental representations of non-native readers of Chinese. Journal of Language and Learning, 4, 172-198. Chen, Q. & Donin, J. (1997). Discourse processing of first and second language biology texts: Effects of language proficiency and domain-specific knowledge. Modern Language Journal, 81, 209– 227. Chern, C. (1993). Chinese students word-solving strategies in reading in English. In T. Huckin, M. Haynes, & J. Coady (Eds). Second language reading and vocabulary learning. Pp. 67–82. Westport, CT: Ablex. Crookes, G. (1989). Planning and interlanguage variation. SSLA, 11, 367-383. Ellis, R. (2005). Planning and task-based performance: Theory and research. In R. Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins.
Ellis, R. (in press). The Differential Effects of Three Types of-Task Planning on the Fluency, Complexity and Accuracy in L2 Oral Production. Applied Linguistics. Hudson, T. (1982). The effects of induced schemata on the short-circuit in L2 reading: non- decoding factors in L2 reading performance. Language Learning, 32/1, 1-31. Hammadou, J. (1991). Interrelationships among prior knowledge, inference, and language proficiency in foreign language reading. The Modern Language Journal,75: 27-39. Johnson, P. (1982). Effects on reading comprehension of language complexity and cultural background of text. TESOL Quarterly, 16, 169–181. Kawauchi, C. (2005). The effects of strategic planning on the oral narratives of learners with low and high intermediate proficiency. In R. Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins. Lee, J. F. (1986). Background knowledge and L2 reading. Modern Language Journal, 70, 350–354. Peretz, A., & Shoham, M. (1990). Testing reading comprehension in LSP: Does topic familiarity affect assessed difficulty and actual performance? Reading in a Foreign Language, 7, 447–455. Shimoda, T. A. (1993). The effects of interesting examples and topic familiarity on text comprehension, attention, and reading speed. Journal of Experimental Education, 61, 93-103. Schmidt-Rinehart, B. C. (1994). The effects of topic familiarity on second language listening comprehension. The Modern Language Journal, 78, 179-189. Skehan, P. & Foster, P. (1999). The influence of task structure and processing conditions in narrative retellings. Language Learning, 49 (1): 93-120. Tavakoli, P. & Skehan, P. (2005). Strategic planning, task structure, and performance testing. In R. Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins. Foster, P. & Skehan, P. (1996). The influence of planning and task type in second language performance. SSLA, 18, 299-323. Wigglesworth, G. (1997). An investigation of planning time and proficiency level on oral test discourse. Language Testing, 14 (1): 85-106. Yuan, F. & Ellis, R. (2003). The effects of pre-task and on-line planning on fluency, complexity and accuracy in L2 monologic oral production. Applied Linguistics 24(1): 1–27.