1. Ensuring quality in language testing and assessment: the contribution of the CEFR
ECML Colloquium2016
The experience of the ECML RELANG team
over the three years of delivering RELANG Training and Consultancy:
Aligning language curricula, tests and examinations to the CEFR
in a total of 20 European countries

2. on Innovative Methodologies and Assessment in Language Learning
RELANG
Relating language curricula and examinations to the common European reference levels of language proficiency
Promoting quality assurance in education and facilitating mobility
European Centre for Modern Languages and European Commission Cooperation
on Innovative Methodologies and Assessment in Language Learning

3. RELANG Team Members
Évelyne Bérard: (formerly) Université de Franche-Comté, Centre de linguistique appliquée, France
Jana Bérešová: Trnava University, Slovak Republic
José Noijons: (formerly) Cito, Institute for Educational Measurement, The Netherlands (Coordinator)
Gábor Szabó: University of Pécs, Hungary

4. Modules on Offer (1)
Developing valid language tests for the use in the classroom
Target group: teacher trainers (both pre-serviced and in-service)
Constructing valid language tests and examinations based on the CEFR
Target group: language testers and examiners, item writers and curriculum developers
Relating existing tests and examinations to the CEFR
Target group: language testers and examiners, item writers and curriculum developers, policy makers

5. Modules on Offer (2)
Screening existing tests and examinations for their relations to the CEFR
Target group: examination centres and policy makers
Relating foreign language curricula to the CEFR
Target group: curriculum developers, teacher trainers and policy makers
Applying the CEFR to the development of tests of second language competence
Target group: policy makers, language test developers, teacher trainers

6. Basic Documentation
The Training and Consultancy is based on two Council of Europe / ECML publications dealing with the construction and validation of language tests linked to the CEFR:
The ALTE Manual for Language Test Development and Examining
Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR): Highlights

7. Linking Procedures in the Manual
Familiarisation with the CEFR
Linking on the basis of specification of curriculum/examination content
Standardization and Benchmarking
Standard setting
Validation: checking that curricula and exam results relate to CEFR levels as intended

8. The CEFR Model of Language Use
Elements in action-oriented approach:
Actions performed by persons - individuals and social agents
A range of competences, both general and in particular communicative language competences
Various contexts under various conditions and constraints to engage in language activities
Language processes to produce and/or receive texts in relation to themes in specific domains

9. Frequently Asked Questions
How do I know whether a test is at the CEFR level it claims to be at?
Can an exam that tests linguistic competence (language use) be linked to the CEFR?
How do we make sure that our pass/fail decisions are related to the CEFR? 
Can we test at more than one CEFR level in one test?
Can we link our tests to the CEFR without statistics on student performance?
Can tests of different skills be summarized at one CEFR level?

10. How do I know whether a test is at the CEFR level it claims to be at?
Examination centres, testing organisations and publishing companies claim that the tests they administer or publish are at a given CEFR level
The validity of such claims may be very important for stakeholders
On the basis of their results, students may be admitted to further education or hired for a job.
There is also a need for institutes and employers to be able to depend on the validity of claims
Yet some published information on links should be available:
Field testing has shown the test is valid and reliable
The test has been specified in terms of the CEFR
The test has been standardized
There is empirical validity (from outside)

11. Can an exam that tests linguistic competence (language use) be linked to the CEFR?
The CEFR acknowledges that linguistic competence is an important aspect of language competence
It may be difficult to link sections in an exam that test subskills such as grammar or vocabulary to the CEFR
Such sections may focus on the structure of a language rather than on the communicative aspects of it.
Such sections do not necessarily focus on grammatical constructions that are typical of texts produced in various contexts at various levels, and the vocabulary going with those.
Such sections are not usually linked to specific linguistic CEFR descriptors at various levels.

12. How do we make sure that our pass/fail decisions are related to the CEFR?
Pass/fail scores in exams are often determined without reference to the CEFR
Students may pass an exam at a given CEFR level without attaining a score that indicates a proficiency at the desired CEFR level
Pass/fail scores related to the CEFR need
Standard-setting procedure (for receptive skills)
Benchmarking procedure (for productive skills)
A score on the exam may indicate two things:
the student has or has not passed the exam from a legal perspective
the student has or has not reached the desired CEFR level

13. Can we test at more than one CEFR level in one test?
In theory this is possible, in practice this may be challenging
A test that is linked to the CEFR is supposed to tap a representative number of descriptors at each desired CEFR level
Tests of Writing, Reading and Listening may become too long
Certain test formats for Speaking are suitable
A low score at a higher level tests does not mean a pass at a lower level
Insufficient score on a B1 test does NOT imply a pass at A2 level

14. Can we link our tests to the CEFR without statistics on student performance?
Test validity will be enhanced when on the basis of data on student performance (pre-testing) the test is adapted: quality of items, difficulty of tasks, type of tasks, length of the test etc.
However, there are situations where such a procedure is not possible or too costly, such as in classroom-based testing
If it is not possible to collect evidence that a test is valid and reliable through statistics, we can try and link the test to the CEFR through specification.
In fact specification is a phase in the linking process that always needs to be carried out.

15. Can tests of different skills be summarized at one CEFR level?
We cannot simply “average out” performances in different skills
In real life most learners are better at one skill than another, certainly at the lower CEFR levels
We may be able to average score points, but we cannot average CEFR levels
We may be able to say that the student is at B2 for reading and at A2 for writing. We cannot then say that the student is at B1 for reading and writing combined
The Council of Europe has propagated profiles, in which the student’s proficiencies in the various language skills are described.
The European Language Portfolio has also adopted this approach.
Suggestions for action: see