1. Fremdsprachenkompetenz – Der Schlüssel zur Tür nach Europa Kloster Banz 26. Oktober 2006 Language Testing and Teacher Training — A Swedish Example Gudrun Erickson Göteborg University, Sweden Dept. of Education; Language and Literature Unit gudrun.erickson@ped.gu.se

2. The Swedish School System 9-year compulsory school — Gesamtschule 98% continue to three-year upper secondary education (c. 75 % graduate) — Core subjects National curricula and syllabuses for subjects Grading from year 8 — to be year 6… Goal/criterion referenced grading system; four grade levels — to be more… Teachers responsible for grading Grades used for selection for higher education Extensive national testing programme

3. ”Levels” in national test development process Government The Ministry of Education The Swedish National Agency for Education Different universities in the country Göteborg University: Foreign languages www.ped.gu.se/sol

4. Aims of national assessment — [Strong] Advisory function Enhance educational achievement Clarify goals; indicate strengths and weaknesses in individual learner profiles Concretize goals and grading criteria Enhance equity and fairness in assessment and grading Provide statistics for local and national analyses of educational achievement Formative and summative materials

5. Foreign Languages in Swedish Schools English compulsory from primary level [Optional] Second FL in compulsory school; approx. 80 % start, 62 % complete their studies – German, Spanish, French English compulsory in upper secondary education — a core subject Second FL compulsory in more academically oriented programs in upper sec. (Fr, Ger, Sp + e.g. Italian, Russian, Chinese…)

6. National Test Materials of FL English Subject test for grade 5 (11-year-olds) Diagnostic material for grades 6-9 (12-15-year-olds) Subject test for grade 9 (16-year-olds) / end of compulsory school Summative tests for two courses in upper sec. ed. (A + B) Self-assessment material for course A Illustrative materials for [advanced] productive skills (C) French, German, Spanish Summative materials for two courses in upper sec. ed. Illustrative materials for productive skills Under development: formative (diagnostic) + summative materials for compulsory school

7. Focal areas in the national syllabuses of FL Receptive competence Productive competence Interactive competence Intercultural communicative competence Reflective competence 7 stages of proficiency

8. The Swedish syllabus for FL and the CEFR: A tentative [textual] comparison Stage Level (entrance levels) 1A1 – A2 2A2 3A2 –B1 4B1 5B1 – B2 6B2 7B2 – C1

9. The typical national FL test Four subtests — Target language only (instructions, questions, answers) Oral production and interaction (paired or group oral) Reception: Listening comprehension Reception : Reading comprehension Written production (and interaction) — — — — — — — — Culture in a wide sense, an integrated aspect of test content and design Reflection, e.g. self assessment, focused upon in several materials Results / Sub-scores presented in profiles Model provided for aggregation into total score Extensive guidelines for teachers. Benchmarked examples of oral and written production and interaction, incl. reasoning and comments based on goals, grading criteria and pre-defined analytic factors.

10. Basic principles for national test development Pedagogical considerations – Assessment of and for learning Enhancing validity and reliability, avoiding bias Broad coverage of the construct – multi-faceted, varied, monolingual tests Giving students the opportunity to show what they can do with their language, rather than focusing on what they cannot do Making what is most important assessable, not prioritizing what is easily measurable

11. Basic principles for test development (cont’d) Presenting individual results in profiles Focusing first on strengths, then on weaknesses Distinguishing between errors that [may] disturb, and errors that destroy communication ********** A collaborative approach to test development Large-scale pretesting of all materials An eclectic approach to method (“Q +Q”)

12. Development Process Analyses of relevant literature and research Continuous cooperation with ’broad’ expert groups Small-scale piloting > Adjustments Large-scale pretesting in randomly selected groups of students in the country (n ≈ 400 per task) Analyses of results AND of students’ and teachers’ views and suggestions (“the q + q approach”) Compilation of tests Standard setting and benchmarking in broad groups of experts

13. Reactions to national tests Students generally positive to varied, authentic tasks – not too easy, not too difficult > 95 % of teachers positive to the national tests function, content, level of difficulty, guidelines, profiles, layout [Teacher] discussions about workload; practical aspects of oral testing; standards; linguistic requirements [Political] discussions about role of national tests (correlation with final grades); confidentiality; costs

14. Four Lines of Research Language Validation Test-taker feedback Dimensionality

15. National assessment in Sweden: Challenges Coping with multiple aims Balancing implementation and prescription Bridging the gap between formative and summative assessment Further developing methods of collaboration with different stakeholders, e.g. students Emphasizing further the pedagogical potential of assessment for efficient and positive learning Increasing assessment competence and confidence among teachers Analyzing further the relationship between the Swedish syllabuses and the CEFR

16. Language (Subject) Teacher Education in Sweden: Two models I.Language studies, didactics and teaching practice integrated (4.5 – 5.5 years) II.Language studies — Didactics and teaching practice as a separate programme (1.5 years) (often older students with work experience) Subject teachers normally teach two subjects

17. Testing and Assessment in Teacher Training Considerable variation between different universities Strong criticism from the Swedish National Agency for Higher Education of too little emphasis on assessment and grading in teacher training programmes Growing interest in assessment issues, in pre- and in-service training Considerable increase of budget from the new government for teacher pre- and in- service training, including assessment and grading

18. Example of structure Pre- and inservice training General lecture(s) on assessment and testing (construct/rubric; validity/reliability; formative/summative; “the joint venture”; assessment of/for learning; washback; rating/grading, … ) Literature seminars and workshops (Designing tasks for different constructs and different purposes; rating tasks — focusing on inter- and intra- rater reliability; national tests; benchmarks/ standards; CEFR / ELP, … ) Assignments during teaching practice / in practice Follow-up seminars based on practical examples

19. EALTA* Guidelines for Good Practice in Language Testing and Assessment Three categories/“audiences”: Teacher pre- and in-service training Classroom assessment Test development in national or institutional testing units or centres Basic principles the same for all categories: Respect for students/examinees, responsibility, fairness, reliability, validity, collaboration among the parties involved * European Association for Language Testing and Assessment www.ealta.eu.org

20. EALTA Guidelines Considerations for teacher pre-service and in- service training in language testing and assessment Examples How aware are trainees made of the range of assessment procedures appropriate to their present or future needs? What is the balance between theory and practice in the training? How far are the trainees involved in developing, trialling and evaluating assessment procedures? How far do assessment procedures used to evaluate the trainees follow the principles they have been taught?

21. EALTA Guidelines Considerations for classroom testing and assessment Examples ASSESSMENT PURPOSE(S) AND SPECIFICATION What is the purpose of the assessment? How does the assessment purpose relate to the curriculum? ASSESSMENT PROCEDURES Who designs the assessment procedures? What account is taken of students’ views on the assessment procedures? CONSEQUENCES What use is made of the results? What are the consequences of the results of the assessment for learners?

22. EALTA Guidelines Considerations for test development in national or institutional testing units or centres Examples TEST PURPOSE AND SPECIFICATION TEST DESIGN and ITEM WRITING QUALITY CONTROL and TEST ANALYSES TEST ADMINISTRATION REVIEW WASHBACK LINKAGE TO THE COMMON EUROPEAN FRAMEWORK

23. European student 2005: “I think that a good language test/ assessment should get students chance to show what they know and devolope their skills. It should contain of listening and reading comprehations, as well as writing and grammar part. But the most important thing is to speak in that language, so I think there should be also oral exams. Speaking is the most important, because without it, we wouldn’t communicate. Because of that students should also learn pronunciation.”

24. Matters to discuss Formative and summative assessment / classroom assessment and exams: similarities and differences? Influence from tests and testing on learning and teaching: positive, negative — inevitable…? Effective ways of supporting teachers in their different assessment activities and practices The role of students in testing and assessment “ Making what is most important assessable, not prioritizing what is easily measurable” – effects and implications? The EALTA Guidelines for Good Practice in Language Testing and Assessment