Presentation on theme: "Assessing Writing – assessing tools and teaching action Barbeiro, Luís Filipe LITERACY ACROSS THE CURRICULUM IN DIFFERENT LANGUAGES AND CONTEXTS. Functional."— Presentation transcript:
Assessing Writing – assessing tools and teaching action Barbeiro, Luís Filipe LITERACY ACROSS THE CURRICULUM IN DIFFERENT LANGUAGES AND CONTEXTS. Functional approaches to reading and writing Madrid 17-19 Oct 2013 Ferreira, Alexandra Oliveira, Ana Cristina Barbeiro, Célia Correia Mateus School School of Education and Social Sciences, ‒ Polytechnic Institute of Leiria
■ In this presentation, we will compare the potential and use of different writing assessment tools: the evaluation grids in use at schools participating in the TeL4ELE project, in Leiria, and criteria that have been adopted since the implementation of the R2L programme. 2 Introduction ●
► Enable teachers to see the progress students are making, and how far they need to grow to achieve their grade standard; ► Provide a record of achievement for students, their family and the state, ► Show what language resources and skills students have, so teachers can plan teaching program, and explicitly discuss language and skills with them. 3 Introduction ● ► Background ♦ Purposes of writing assessment ► Rose (2010)
► Portuguese Ministry of Education has adopted a policy of national examinations in order to evaluate students’ achievement (writing included), at different school years: 4 th, 6 th, and 9 th. ► Accompanying the national exams, the Ministry of Education provides assessment guidelines and criteria to be used by the teachers who are involved in the process of marking students’ exams. 4 Introduction ● ► Background ♦ Portuguese context
► Extension ► Theme and Typology ► Coherence and Content Information ► Structure and Cohesion ► Syntax ► Vocabulary ► Punctuation ► Spelling 5 Introduction ● ► Background ♦ The evaluation grid of the Ministry of Education (national examinations) With slight modifications over the years, this grid has adopted the following criteria: 0/1-4/5
6 Introduction ● ► Background ▪ Since these guidelines and criteria were created by the Ministry of Education and since they are used in national examinations, they were adopted by many schools and teachers, aiming to keep up to date and improve students’ preparation for the national examinations. ▪ Therefore, in Portuguese schools, these criteria have acquired a regulatory dimension, in relation to writing assessments and ultimately in teaching (functioning as guidelines to teachers’ evaluation and feedback to students). It is important to reflect on the potential and limitations of these criteria in comparison with other grids
► The assessment grids adopted in the schools were also largely based on the criteria of the Ministry of Education: 7 ♦ … Schools where TeL4ELE was implemented, in Leiria ■ Correia Mateus School (5th - 9th) ▪ Type of text and theme ▪ Coherence and pertinence of information ▪ Structure and cohesion ▪ Morphology and syntax ▪ Vocabulary ▪ Spelling ▪ Punctuation ■ Vale Sumo School (1st - 4th) ▪ (Extension) ▪ Theme ▪ Sequencing ▪ Ideas / Creativity ▪ Vocabulary ▪ Spelling ▪ Punctuation Var. 0-2/3/4 0-3
8 ♦ During the implementation of TeL4ELE, we applied the criteria of R2L: Levels Criteria Quick judgementScores CONTEXT Purpose 0-3 Staging 0-3 Phases 0-3 Field 0-3 Tenor 0-3 Mode 0-3 DISCOURSE Lexis 0-3 Appraisal 0-3 Conjunction 0-3 Reference 0-3 GRAMMAR 0-3 GRAPHIC FEATURES Spelling 0-3 Punctuation 0-3 Presentation 0-3
The most salient differences refer to the higher levels of text construction: the level of the relationship with Context and the Discourse organization In the previous grids, the criteria referring to higher levels merge different levels (e.g. Structure and Cohesion; Sequencing; Coherence and pertinence of information) In contrast, the R2L grid offers more specific criteria/parameters to deal with text, at the higher levels of text adequacy to objectives and text organization beyond the sentence 9 ► Contrasting the grids
Do they result in different global evaluations of students’ texts? Do the more specific parameters of R2L grid, namely at Context and Discourse levels, reflect on scores? Do R2L criteria reveal the need for specific action at these levels? 10 Study ● ► Method ■ We compared the application of the two grids: ► Comparing the total score obtained with each one of the grids (text genre: narrative; scores were assigned by teachers of the classes) ► Comparing particular texts under the perspective (and scores) of the different grids
11 Results ● ► Correlations ♦ Very high correlation between the total scores of the two grids Levelsr Sig. All cases (n=37) 0,863 ** (p<,01) 5 th (n=25)0,985 ** 4 th (n=6 ≡ 2H+2M+2L)0,951 ** 3 rd (n=6 ≡ 2H+2M+2L)0,984 **
12 ♦ Both grids discriminate high, medium and low achieving students. Levels Grid 1 (av.) R2L (av.) High achievement92 %91 % Medium71 %70 % Low achievement55 %56 % ► The purposes of writing assessment tools are not limited to evaluating students’ achievement at the end of each term or school year (to inform the families and the educational authorities). ▪ Two other major purposes are ► to show the language resources and skills that students have, so teachers can plan teaching program; ► to function as tools of interaction in order to explicitly discuss language and skills with students.
► Working with R2L proposals and criteria provided us the specific terms to guide students to improve their texts at the level of Text organization and Discourse; ► At the level of Text organization / Context, the work on Stages and Phases became prominent. 13 ● ♦ Implications of R2L criteria for teachers’ practice
► During the implementation of TeL4ELE, namely through the R2L writing assessment criteria, we became aware that some of the criteria related to CONTEXT and DISCOURSE were not specifically present in our assessment grid and, as a consequence, did not acquire prominence in our instruction or our interaction with children; ► A significant example concerns text genre and its organization through Stages and Phases; ► Talking about text organization, the prominent terms were the common and general terms Introduction; Development and Conclusion; 14 ● ♦ Implications of R2L criteria for teachers’ practice
► Working Stages and Phases benefited the students at all levels of achievement; ► The students who already performed at the highest levels found new challenges and new fields to improve their writing: ◊ the challenge of developing their text, according to the Stages of a specific genre (the terms “Orientation”, “Complication”, “Resolution”, for example, becoming usual in the classroom interaction, and allowing to establish the differences between narratives and recounts); ◊ the challenge of incorporating specific phases, in order to improve their texts — and, by doing this, finding the pathway to creativity, as modelled in the texts of the authors; ► The students at lower achievement levels took part in the classroom interaction and revealed (namely through explicit guidelines and the strategies of Rewriting and Joint Construction) textual and linguistic devices to improve their writing. 15 ● ♦ Implications of R2L criteria for teachers’ practice
16 Examples ● ♦ Sara’s narrative “Adventure on the beach“ Sara, 5 th grader, is a high-achievement student (level 5/5); Very often, her texts receive the maximum scores; The first evaluation of her text, using the usual Grid, assigned it the maximum score. The text does not present formal problems: spelling, syntax, etc. are OK. We cannot say it is not organised or it has sequencing, coherence and cohesion problems; R2L criteria revealed that the text could be improved through a better management of Stages (namely working on Complication and Resolution) and Phases (inserting a real Problem, Reactions, Solution…) — subsequent work allowed to improve even more Sara’s writing, according to the genre challenges. An adventure at the beach It was a beautiful sunny day. Carlota, the female crab, woke up with the sound of ocean waves. Carlota was very curious and as she was still small, she did not know well the beach where she lived. After having breakfast, Carlota was calling her friend Diogo, the hermit crab, to play with her by the seaside. They were both entertained playing a game, when suddenly, they heard a whistle. The whistle came from behind a rock, and Carlota and Diogo went out to see what was happening. They were amazed when they saw a fish trapped between the rocks whistling. The three became great friends and they played altogether until sunset. After sunset, they went back to their homes. Carlota never forgot that fantastic adventure at the beach.
17 Examples ● ♦ Patricia’s story “The horse“ Patricia, 5 th grader, is a low- achievement student (level 2/5); The first evaluation of her text, using the usual Grid, penalized the formal problems (8/20 ≡ 40%); R2L criteria confirmed the formal (spelling, …), discursive problems (Lexis, Reference, Appraisal), and genre problems (although a rough mastery of Stages) — 19/42 ≡ 45%). Instead of relying on generic concepts (“organization”, “structure”, …) it showed the way of the specific work with Stages and Phases, and the strategies of Rewriting to improve Patricia’s writing at the levels of Discourse and Context.
18 Examples ● ♦ During the implementation, the work with Stages acquired prominence. Orientations attained a high level of elaboration, even at the lower school grades: 3 rd grade 4 th grade Once upon a time, there was a beautiful and colourful mermaid of the sea. The mermaid had got blue eyes and her tail had the colors of the rainbow. The sea where she lived was quiet and beautiful, with blue water and many fish. On a cloudy day, the sea was rough, and down there in the deep ocean was a beautiful mermaid with pink hair and a dark green tail, the most beautiful colour of that ocean.
19 Examples ● ♦ Complications and Resolutions present a lower level of elaboration. However, they became present and established the difference with the genre Recount (which previously occupied almost all the work with stories). 3 rd grade 4 th grade Examples of Resolutions Then the elf became the wise. On the big day she won and became amazed /openmouthed with the prize. Thereafter there was never lack of food in that place.
Conclusion 20 ■ Writing assessment criteria ▪ are a powerful tool to regulate and influence the process of teaching writing skills ■ Portuguese language teachers ▪ have adopted the criteria of the Ministry of Education for national examinations; ▪ this grid does not provide enough and adequate criteria to assess students’ texts at the levels above the sentence (some of the criteria merge different aspects or levels) ■ Working with R2L criteria ▪ in association to a genre based pedagogy, revealed some of the weaknesses that students’ writing commonly still present, at the level of text organization, ▪ and guided us in the process of improving writing, focusing on genres, Stages and Phases, benefiting students at all achievement levels. ▪ Alongside with Lexis, this was one of the most remarkable improvement of students’ writing, in our TeL4ELE classes.