From Learning to Testing

Slides:

Advertisements

Similar presentations

Consistency of Teacher Judgement

Advertisements

Training teachers to use the European Language Portfolio Former les enseignants à lutilisation du Porfolio européen des langues.

European Frameworks of Reference for Language Competences Waldemar Martyniuk Language Policy Division, Council of Europe / Jagiellonian University, Poland.

© Stichting CITO Instituut voor Toetsontwikkeling 1 Mapping the Dutch Foreign Language State Examinations onto the Common European Framework of Reference.

Spiros Papageorgiou University of Michigan

Centre for Applied Linguistics Dr Claudia Harsch Centre for Applied Linguistics University of Warwick From Norm- to Standards-based assessment What role.

Using the CEFR in Catalonia Neus Figueras

Consistency of Assessment

| ERK/ CEFR in Context 23 January 2015, Groningen Estelle Meima Language Centre.

Prepared by Jan Sheinker, Ed.D Points of view or opinions expressed in the paper are not necessarily those of the U.S. Department of Education, or Offices.

Setting Alternate Achievement Standards Prepared by Sue Rigney U.S. Department of Education NCEO Teleconference March 21, 2005.

Classroom Assessment A Practical Guide for Educators by Craig A

Questions to check whether or not the test is well designed: 1. How do you know if a test is effective? 2. Can it be given within appropriate administrative.

Consistency of assessment Technology subjects (7-12)

Relating language examinations to the Common European Framework of Reference for Languages (CEFR) Waldemar Martyniuk Waldemar Martyniuk Language Policy.

Common European Framework of Reference for Languages (CEFR): Learning, Teaching, Assessment Nuppu Tuononen Palmenia Centre for Continuing Education

Quality in language assessment – guidelines and standards Waldek Martyniuk ECML Graz, Austria.

Quality Assurance: Looking for Quality Data 1 I know it is in here somewhere Presented by The Early Childhood Outcomes Center Revised January 2013.

1 Use of qualitative methods in relating exams to the Common European Framework: What can we learn? Spiros Papageorgiou Lancaster University The Third.

Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.

CCSSO Criteria for High-Quality Assessments Technical Issues and Practical Application of Assessment Quality Criteria.

Chap. 2 Principles of Language Assessment

STANDARD SETTING Prepared by Ludmila Kozhevnikova and Viktoria Levchenko Based on material by Anthony Green.

Military Language Testing at the National Defence University and the Common European Framework BILC CONFERENCE BUDAPEST.

DEPARTMENT OF EDUCATION AND TRAINING Assessment using the Australian Curriculum 2012.

Testing and Evaluation

ACCEPTABLE METHODOLOGY AND STANDARDS Carey Edwards CRM Advisory Panel.

Curriculum Framework for Romani Seminar for decision makers and practitioners Council of Europe, 31 May and 1 June 2007 Introduction to the Common European.

Relating examinations to the CEFR – the Council of Europe Manual and supplementary materials Waldek Martyniuk ECML, Graz, Austria.

Maurice Grinberg evaluation.nbu.bg ALTE meeting, Lisbon, November

ARE WE TESTING WHAT WE WANT TO TEST? Carole Thomas Bilkent University, Turkey.

RelEx Introduction to the Standardization Phase Relating language examinations to the Common European Framework of Reference for Languages José Noijons.

Australian Curriculum English Achievement Standards PRESENTER: CHRIS THOMPSON SAETA COUNCIL MEMBER.

Presentation to the Nevada Council to Establish Academic Standards Proposed Math I and Math II End of Course Cut Scores December 22, 2015 Carson City,

Colorado Academic Standards Colorado English Language Proficiency (CELP) Standards There are now five English language development standards: Standard.

RelEx Introduction to the Standardization Phase Relating language examinations to the Common European Framework of Reference for Languages Gilles Breton.

Setting Performance Standards EPSY 8225 Cizek, G.J., Bunch, M.B., & Koons, H. (2004). An NCME Instructional Module on Setting Performance Standards: Contemporary.

Glyn Jones Product Development Manager Dr John H.A.L. De Jong Director of Test development Pearson Language Assessments, London Linking Exams to the Common.

Language Assessment.

Relating Foreign Language Curricula to the CEFR in the Maltese context

EVALUATING EPP-CREATED ASSESSMENTS

Sustained excellence in HR - Sanjay Chaturvedi

Quality Assurance: Looking for Quality Data

Introduction to the Specification Phase

ECML Colloquium2016 The experience of the ECML RELANG team

Introduction to the Validation Phase

ECML and Lithuanian Educational Institutions: Experence of Cooperation

Key findings on comparability of language testing in Europe ECML Colloquium 7th December 2016 Dr Nick Saville.

The RELANG initiative is part of a new cooperation agreement, signed in May 2013 between the Council of Europe (represented by the European Centre for.

Introduction to the Validation Phase

Training in Classroom Assessment Related to the CEFR

Introduction to the Validation Phase

RELATING NATIONAL EXTERNAL EXAMINATIONS IN SLOVENIA TO THE CEFR LEVELS

Learning About Language Assessment. Albany: Heinle & Heinle

EALTA MILSIG: Standardising the assessment of writing across nations

Aligning Curricula and Examinations to the CEFR Malta 2015 – 2017

A brief presentation on:

ECML and NEC: Experience of Cooperation

MINISTRY OF NATIONAL EDUCATION

Testing Productive Skills

Specification of Learning Outcomes (LOs)

Module 5: Relating foreign language curricula

Testing Receptive Skills

Relating Examinations to the CEFR Empowering Language Professionals

RELANG Relating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating.

ECML October 2017 Ireland.

Institute of Education Development

Deanna L. Morgan The College Board

Training Chapter for the Advanced Field Experience Form (Pre-CPAST

Presentation transcript:

From Learning to Testing RELANG Relating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility From Learning to Testing European Centre for Modern Languages and European Commission cooperation on Innovative Methodologies And Assessment In Language Learning

Linking Procedures in the Manual Familiarisation with the CEFR Linking on the basis of specification of examination content Standardization and Benchmarking Standard setting Validation: checking that exam results relate to CEFR levels as intended

Claims through Specification and Standardization Claimed link to CEFR is based on specification only no indication of the quality of a performance or the score on a test required for CEFR-level in question Claims to be substantiated through standardisation Learning objectives in relation to CEFR levels.

Steps in the standardization phase Adequate familiarisation with the CEFR. Training in rating productive skills tables and scales in the CEFR or the Manual & scales or specific rating scales Training in rating receptive tasks tables in the Manual & specifications developed for the examinations or tests in question Benchmarking performance samples Standard setting of receptive tasks

Group Decisions Benchmarking and standard setting are procedures requiring group decisions to be carefully prepared by appropriate training.

Standard Setting for Receptive Skills For Learning Objectives that are tested through indirect tests with a numerical score, performance standards have to be set Receptive skills (reading, listening) Underlying competences (grammar, vocabulary) A performance standard is the boundary (cut-off score) between two levels on the scale reported by a test Process to arrive at a cut-off score: standard setting

How to arrive at Standards? Group decisions (panel) Group is familiar with CEFR Test content has been specified in terms of the CEFR Standard setting procedures have been formalized Careful selection and training of panel members

Benchmarking of Productive Skills The judgment on the CEFR level is direct Assistance needed for raters in giving valid judgments Benchmarking: providing one (or more) typical sample(s) to illustrate performance at a given level

Validation What is Validation? The degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. Does the test measure what it intends to measure?

Content Validity Does the test accurately reflect the syllabus on which it is based AND reflect the descriptors in the CEFR? Does the content specification reflect all areas to be assessed in suitable proportions?