Presentation on theme: "Assessing L2 writing performance: Reconsidering writing prompts as genre-based tasks Heidi Byrnes German Department Georgetown University"— Presentation transcript:
Assessing L2 writing performance: Reconsidering writing prompts as genre-based tasks Heidi Byrnes German Department Georgetown University firstname.lastname@example.org TBLT 2007 University of Hawai’i at Manoa Sept. 21, 2007 GUGD curriculum web page: http://www3.georgetown.edu/departments/german/programs/curriculum/index.html email@example.com
Underlying Assumptions L2 writing prompts can/should - be reconsidered as “tasks”, particularly as genre-based tasks - be linked to quality of performance - facilitate the assessment of L2 writing performance L2 writing in genre-based tasks can - Provide a favorable environment for integrating “language” and “content” Assessment of L2 writing performance should - Consider language knowledge and content knowledge in an integrated fashion
“Tasks” in L2 education (1) An educational “activity ” With a particular objective, appropriate content, specified work procedure, range of outcomes, necessitating the use of language (Breen 1997, Van den Branden 2006) In the target language, focused on meaning rather than form ( Nunan 1989) Meaning-oriented language use, resembling language use in the real world, often occasioned by a problem, involving a workplan toward achieving a certain outcome ( Ellis 2003, Skehan 1998)
“Tasks” in L2 education (2) Users of tasks Teachers and learners, curriculum developers, researchers, assessment specialists (Bygate et al. 2001) Uses of tasks, as a way of –Specifying learning goals –Designing and organizing educational activities in support of learning –Ascertaining ability to use language (assessment) –Exploring implications of findings for curriculum and instruction
“Tasks”: Relating “language” and “content/meaning ” Language AND content Language knowledge AND content/background knowledge Focus on form AND focus on meaning Ability to use/perform AND ability to develop Additive conceptualization of task No conceptual reach into content/meaning in social context
Needed: New conception of “language” “Our approach to the teaching of any phenomenon depends critically on our conception of this phenomenon. Unless we can base language teaching and learning on a richly revealing comprehensive account of what kind of phenomenon language is, we are not in a position to answer the many questions that arise in educational contexts. Similarly, the value and success of any approach designed to support second or foreign language teaching … will depend critically on the conception of language that informs it.” ( Matthiessen 2006: 31)
Probing two constructs of “task” (1) Language analyzed as Formal system of rules Fundamentally functional social semiotic system Meaning making Expressing things; Focus on personal meanings; “Negotiation of meaning”. Construing/creating that which is being expressed; Developing a voice within socially situated, socially preferred forms of language use. Semantics and lexicogrammar Inventories Strategic resources
Probing two constructs of “task” (2) Context and content Given “out there” Socially constructed; Language as primary semiotic system; “Context of culture” + “context of situation” participating in the construal of meaning Language learning as Gains in accuracy, fluency, complexity of processing in all modalities Learning how to mean; Building up ‘meaning potential’ in a range of registers and text types/genres Focus of attention Sentence-level processing Oral/written texts as environment arising from and construing social structures and meanings
Probing two constructs of “task” (3) Processing orientation Syntagmatic syntactic rules orientation Paradigmatic meaning and choice orientation “Knowledge” imagined and valued as Cognition; “Cognitively demanding;” “Higher-order thinking” Language-based shaping of world; Expressed in texts from 2 semiotic perspectives congruent semiosis non-congruent (“metaphorical”) semiosis Integration through Psycholinguistic mechanisms focused on individual language user Semiotic processes focused on construing and shaping the physical and social world = meaning making
Needed: New understanding and descriptions of language The challenge is to relate sociocultural context and the “content” that is likely to be enacted to its particular wording in language, that is, we need rich accounts of semantics and lexicogrammar. “On the one hand, we need semantics as an ‘interface’ between context and lexicogrammar … a fully fledged semantics of text. … Language learners need to learn semantics as a strategic resource … a resource for transforming what is not meaning into meaning, construing their experience of the world as meaning and enacting social roles and relations as meaning; and this will provide them with the ‘bridge’ to lexicogrammar. On the other hand, both semantics and lexicogrammar need to be learned as resources rather than as inventories.
Needed … ... To meet the challenge of relating language to context, we also need to have a systematic and comprehensive account of context … Like language, context is a semiotic system, but it is a different kind of semiotic system … language is a denotative semiotic system (that is, a semiotic system that has its own expression plane), whereas context is a connotative semiotic system (that is, a semiotic system that has other semiotic systems as its expression plane). Context is realized by language, and by other denotative semiotic systems such as gesture and facial expressions; and also by non-semiotic, social systems.” (Matthiessen 2006, 37-38)
SFL approach to relating meaning/content and language The process of meaning making is multifunctional in all languages Three metafunctional modes of meaning –Construing our experience in the world around us and inside us as meaning = ideational metafunction (Meaning as “representation”: participants, processes, circumstances) –Enacting our social roles and relations as meaning = interpersonal metafunction (Meaning as “exchange”) –Presenting the meaning construed and enacted as a flow of information = textual metafunction (meaning as “message/text”: “coherence, cohesion”)
SFL approach (ctd.) These resonate with three contextual variables - field: “what” is being talked about (social activity) (participants, processes, circumstances) - tenor: “how” the participants relate to each other in relation to the activity (mood, modality, appraisal) relation to the activity (mood, modality, appraisal) - mode: “how” the oral/written text is structured (theme and information structure; logical relationships in clause complexes; grammatical metaphor)
A trinocular conception of meaning/context Language comprises Ideational resources naturalize reality “Field knowledge” enables participation in domestic, recreational, academic, public, and professional activities Interpersonal resources negotiate social relations ”Interpersonal knowledge” enables a way of valuing and assessing these activities and enacting power and solidarity in relation to shared values Textual resources manage information flow “Textual knowledge” enables ways of phasing the ideational and interpersonal meanings into textures that are responsive/sensitive to the communicative demands of oral and written discourse.
Metafunction in relation to language, register, and genre
Register and genre in SFL Register – a “constellation” A constellation of lexicogrammatical features that realizes a particular situational context, characterized in terms of ranges of field, tenor, and mode values in the contexts within which they are located Genre – a “text type” Recurrent configuration of meanings Culture as a system of genres Staged, goal-oriented social process, realized in text Staged: more than one phase of meaning to work through the genre Goal-oriented: unfolding phases toward meeting expectations Social: genres occur in social settings and are interactively undertaken The “teachability” of genres
Big Challenges What constitutes good L2 writing? How can we link content knowledge and language knowledge while setting some limits to the assessment of “content”? How does instructed L2 writing develop? Does it/should it figure into the assessment of L2 writing performance?
Writing development in the GUGD: a “case study” Writing tasks as genre-based writing tasks Embedded in a four year integrated content oriented and task-based curriculum Learning goal of advanced multiple literacies in several languages Explicit pedagogy for writing Writing development integrated into L2 development in all modalities Assessment of writing development in curricular context Example: Level III writing task, within 5-level curriculum. Course focus “German stories, German histories” (Post WWII – present). Unit: fall of the wall.
Foci for genre-based writing tasks Task Appropriateness: Understood in terms of –Generic moves –Positioning of writer toward an imagined reader audience Content: Understood in terms of –Information provided within each of the generic moves Language Focus : at 3 strata of language system –Discourse, sentence, lexicogrammatical system
Move to handout for End of curricular level expected writing performance profile: Levels I – IV of the five-level curriculum Specification of language foci at 3 strata of language system: Level III Genre “(Political) appeal” = Aufruf. Text used in class to exemplify the genre. Analysis of the text’s generic moves Lexicogrammatical analysis of the moves “problem statement” and “appeal for action” in terms of field, tenor, and mode. General guidelines for the creation of genre-based tasks Aufruf task sheet, in terms of task appropriateness (genre), content, and language focus Feedback sheet for the writing task Guidelines for the assessment of writing performances
Linking content/context and language in genre-based writing tasks: Pedagogical considerations Purposeful texts approximating real-life writing in terms of topics, possible relationships between writer and audience, purposes, textual demands in terms of level of L2 ability Focus on creating meaning (content/knowledge) in a textual world through and in and with language “forms” at all strata of the system that should be chosen for their meaning-making capacity in relation to a genre. Culturally authentic language use within the genre as culturally validated forms of texts and registers with their preferred bundling of lexicogrammatical features Development of a competent non-native voice within stable, yet flexible generic structures
Pedagogical considerations (ctd.) “Conceptual space” through metalanguage for a focus on content: manifestations of “field” in the various genre moves Motivated development from more narrative genres, which privilege congruent semiosis, to more abstract, essayistic, public genres, which privilege noncongruent (metaphorical) semiosis long-term trajectory toward the secondary discourses of the academy (see e.g., Byrnes & Sprang, Christie, Macken-Horarik, Martin, Rothery, Schleppegrell) “ Plausibility” for preferred lexicogrammatical choices – Pawley and Syder’s “native-like selection”, particularly at the phrasal and clausal level, increasingly more important for more advanced learners. Link between L1 and L2 literacies; between learning a language and learning in other disciplinary areas.
Implications for assessment The previous categories and pedagogical approaches to L2 writing provide ways of considering writing development as well as writing quality Assessment can be imagined at the intersection of a number of the constructs SFL makes available: What metafunctions are handled and how well? What major subaspects of these metafunctions are handled and how well?
Implications for assessment (ctd.) What language strata are competently used? from word to phrase, to clause, to clause complex, to text, - -- linking into cultural context, and from cultural context back to language What is the breadth and depth of registerial and generic resources available for use? What is the user’s location on the cline between the resources of the system and instantiation of those resources in a text?
Assessment (ctd.) What kind of knowledge is being created: congruent – noncongruent forms of semiosis? What kinds of logical relationships are established and how? In simplex clauses? In complex clauses? What forms of grammatical are deployed? For what functions? How is the phrase level expanded? Expansion of clausal and phrasal capacity How is information structure textured? themes, hyperthemes; different forms of coherence
Assessment (ctd.) … and many more! In each case, language use is linked to meaning, content, context. BUT … There are also necessary limits set to what is being assessed
Benefits of assessment for teachers and learners Predicts systematically the likely linguistic features that must be in focus Spells out the criteria for successful writing Helps teachers formalize and be more objective about their intuitive judgments about writing quality Is formative and diagnostic with high potential for washback into educational processes
Benefits (ctd.) Clarifies the demands associated with a particular writing task Therefore has a high likelihood of enhancing L2 writing development Provides a language for reflecting on writing Links the goals of L2 writing with those in other subject matter areas
How to take up the challenge … Until we know more about instructed L2 writing development, perhaps inquiry is best located within an educational context -With an explicit curricular trajectory -That integrates all aspects of educational work: curriculum, materials development, pedagogy, assessment -Uses to advantage the specificity that assessment demands regarding what constitutes competent L2 writing while also informing assessment on the nature of L2 development in writing, -And makes appropriate adjustments to improve educational outcomes.
… and gain large benefits It constitutes Socially responsive work –Outcomes of our programs: what is the value added on the basis of our educative work Socially responsible work –Development of sophisticated, multiple literacies in a multilingual and multicultural world Enhances L2 learning –Links content and language learning, thereby providing perhaps the most deep-seated form of motivation for learning languages.