We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byShyann Hoyte
Modified about 1 year ago
© Stichting CITO Instituut voor Toetsontwikkeling 1 Mapping the Dutch Foreign Language State Examinations onto the Common European Framework of Reference Report of a Cito research project commissioned by the Dutch Ministry of Education, Culture and Science José Noijons & Henk Kuijper
© Stichting CITO Instituut voor Toetsontwikkeling 2 Project Goals To establish links between the existing examinations in French, German and English and the CEFR, following the steps as outlined in the Manual published by the Council of Europe. To study the possibilities of developing more comprehensive CEFR-related examinations in the foreign languages.
© Stichting CITO Instituut voor Toetsontwikkeling 3 Four Project Phases Phase 1: Familiarisation Phase 2: Specification Phase 3: Standardisation Phase 4: Empirical validation
© Stichting CITO Instituut voor Toetsontwikkeling 4 Phase 2: Specification Specification of texts Related to the reading scales for communicative language activities (CEFR chapter 4) Related to text type, source, topic, domain (Dutch Grid) Related to the scales for communicative language competence (CEFR chapter 5 & Dutch Grid)
© Stichting CITO Instituut voor Toetsontwikkeling 5 Phase 2: Specification Specification of items Question types Task dimensions 1 (recognising, inferences, evaluation) Task dimensions 2 (explicit vs. implicit) Task dimensions 3 (content of operations)
© Stichting CITO Instituut voor Toetsontwikkeling 6 Specification texts: reading scales communicative activities German
© Stichting CITO Instituut voor Toetsontwikkeling 7 Specification texts: text type, source, topic, domain Great variation in text source and communicative themes (topics) Increase of proportion of expository and argumentative texts from lower to higher educational levels Domain: mainly personal
© Stichting CITO Instituut voor Toetsontwikkeling 8 Specification texts: communicative language competences Level of abstraction Grammatical complexity Vocabulary Text length Dutch grid
© Stichting CITO Instituut voor Toetsontwikkeling 9 Specification texts: communicative language competences, level of abstraction Only concrete1 Mostly concrete2 Fairly extensive abstract3 Mainly abstract4
© Stichting CITO Instituut voor Toetsontwikkeling 10 Specification texts: communicative language competences, grammar Only simple structures1 Mainly simple structures2 Limited range of complex structures3 Wide range of complex structures4
© Stichting CITO Instituut voor Toetsontwikkeling 11 Specification items From lower to higher educational level: more inferences and evaluation, less recognising Relatively more implicit information Greater variation of operations
© Stichting CITO Instituut voor Toetsontwikkeling 12 Specification: general conclusions Increase of linguistic and cognitive complexity of texts Increase of variation of operations demanded in the items
© Stichting CITO Instituut voor Toetsontwikkeling 13 Claims through Specification and Standardisation If a claim of a link to the CEFR is based on specification only, we do not know what score a candidate needs to claim that the candidate’s ability is at the CEFR-level the test claims to be at. Claims can be further substantiated through standardisation of judgements: item- difficulties are judged in relation to CEFR levels.
© Stichting CITO Instituut voor Toetsontwikkeling 14 Standardisation Standard-setting Basket procedure: judges are to assign items (texts + tasks) to CEFR-levels. What minimum CEFR level does one need to master this item?
© Stichting CITO Instituut voor Toetsontwikkeling 15 Results for Exams in English Required minimum CEFR level for Exams in English
© Stichting CITO Instituut voor Toetsontwikkeling 16 Results for Exams in French Required minimum CEFR level for Exams in French
© Stichting CITO Instituut voor Toetsontwikkeling 17 Results for Exams in German Required minimum CEFR level for Exams in German
© Stichting CITO Instituut voor Toetsontwikkeling 18 Scores & Standards: English Distribution of scores & cut score vwo-exam English VWO English Score distribution
© Stichting CITO Instituut voor Toetsontwikkeling 19 Scores & Standards: French Distribution of scores & cut score vwo-exam French VWO French Score distribution VWO French
© Stichting CITO Instituut voor Toetsontwikkeling 20 Scores & Standards: German Distribution of scores & cut score vwo-exam German VWO English Score distribution VWO German
© Stichting CITO Instituut voor Toetsontwikkeling 21 Conclusions standard-setting In the Netherlands a candidate can “pass” an exam without attaining the relevant CEFR level for that exam. Only scores at, or higher than the CEFR cut score indicate that the candidate is at or above the CEFR level the exam has been estimated to be at. Empirical validation may help to show that a CEFR level (a score) attained on one exam is equivalent to a CEFR level on another exam.
© Stichting CITO Instituut voor Toetsontwikkeling 22 General Conclusions A test that is linked or validated through specification only, cannot provide sufficient information on how candidates need to perform on the test to claim they have reached relevant attainment targets. It is necessary through standard setting to compute minimum scores that are needed for candidates to claim they have reached relevant attainment targets. External validation is needed to verify claims of links to the CEFR.
1 C2 in English – can this level be tested? Welcome! Suzanne Vetter-MCaw, ELTAF Members Day,
Linking exams to the CEF levels: T he Manual and the Catalan experience Neus Figueras
Using Learning Outcomes and Assessment Criteria Peter Noakes Department of Electronic Systems Engineering University of Essex.
Session 3: Text Complexity Audience: K – 5 Teachers.
The Common Core State Standards Text Complexity 1.
Principles and Practice in Language Testing: Compliance or Conflict? J Charles Alderson, Department of Linguistics and English Language, Lancaster University.
TESTING SPEAKING AND LISTENING. Proficiency tests are designed to ensure people’s ability in a language regardless of any training they may have received.
UNIVERSITY OF JYVÄSKYLÄ L2 learners performance across L2 writing tasks: comparing tasks and language proficiency through CEFR scales Riikka Alanen
Making the most of assessment: designing multi-purpose tasks for young language learners Louise Courtney Warsaw 21 st October 2011.
Quality Control in Evaluation and Assessment J Charles Alderson, Department of Linguistics and Modern English Language, Lancaster University.
Leon County Schools Understanding Text Complexity August
1 Science as a Process Chapter 1 Section 2. 2 Objectives Explain how science is different from other forms of human endeavor. Identify the steps that.
Language Instruction Educational Programs (LIEPS): Lessons from the Research & Profiles of Promising Programs October 16,
Copyright © 2014 by Educational Testing Service. ETS, the ETS logo, LISTENING. LEARNING. LEADING. ERATER and GRE are registered trademarks of Educational.
1 Welcome Safety Regulatory Function Handbook April 2006.
Linking classroom activities and CEFR levels: how different kinds of assessment can support learning. Neil Jones Cambridge ESOL.
Balancing Quality Enhancement and Accountability Reforming the Dutch and Flemish accreditation system Stephan van Galen.
National Accessible Reading Assessment Projects PARA Project Overview, Results, Next Steps Martha Thurlow and Deborah Dillon.
The Common Core State Standards…. What are they? How will they inform instruction? What are the assessments going to be like? What does this mean for teachers.
1 The National Assessment of Adult Literacy: Health Literacy Results December 5, 2007 Peggy Carr Associate Commissioner National Center for Education Statistics.
KATIE BUCKLEY, HARVARD UNIVERSITY SCOTT MARION, CENTER FOR ASSESSMENT NATIONAL CONFERENCE ON STUDENT ASSESSMENT (NCSA) NATIONAL HARBOR, MD JUNE 22, 2013.
McGraw-Hill/IrwinCopyright © 2011 by The McGraw-Hill Companies, Inc. All Rights Reserved. fundamentals of Human Resource Management 4 th edition by R.A.
1 Consider the observation by Peter Senge: The quality of an organization is a direct reflection on the quality of the conversations that people have within.
1 Implementation of the No Child Left Behind (NCLB) Highly Qualified Teacher & Paraprofessional Requirements December 2010.
1 Teaching and learning about libraries and information: some experiments and some experience Vrije Universiteit Brussel,
COMMON CORE 101 Arizona Common Core Standards: What Governing Boards, Parents, and Community Members Need to Know... ©2012 AYLSTOCK CONSULTING, LLC West.
6.1 PPS t/a Carnegie et al; Accounting: Financial and Organisational Decision Making © 1999 McGraw-Hill Book Co. Aust. ACCOUNTING Financial and Organisational.
1 Developing Tests for Departmental Assessment Deborah Moore, Assessment Specialist Institutional Research, Planning, & Effectiveness University of Kentucky.
Ecdc.europa.eu … Referencing of national qualifications against the European Qualifications Framework for lifelong learning Carlo Scatoli European Commission,
National Accessible Reading Assessment Projects Defining Reading Proficiency for Accessible Large Scale Assessments Principles and Issues Paper American.
© 2016 SlidePlayer.com Inc. All rights reserved.