Presentation is loading. Please wait.

Presentation is loading. Please wait.

Applying CEFR principles to testing and assessment in practice

Similar presentations

Presentation on theme: "Applying CEFR principles to testing and assessment in practice"— Presentation transcript:

1 Applying CEFR principles to testing and assessment in practice
ACERT Conference 15th March 2014, Prague Mila Angelova

2 Aims of the session To give you a short update on EAQUALS operation and activities To introduce you to the EAQUALS Certification Scheme To (re)-introduce you to the most salient features of the six global levels according to CEFR To assess two oral performances for standardisation purposes

3 The Association Founded in 1991 A company registered in UK as charity
International NGO with participatory status at the Council of Europe (2006)

4 EAQUALS Mission The mission of EAQUALS is:
to contribute to the development of excellence in language education to be the internationally recognised authority for quality standards and quality assurance in language education and training

5 EAQUALS Values Plurilingualism Intercultural understanding
International cooperation Lifelong learning

6 EAQUALS Members A network connecting 35 countries
Accredited members x 112 25 languages, private and state sector Associate members x 27 British Council; Instituto Cervantes; CIEP France; Goethe-Institut; Cambridge Assessment; Trinity London; g.a.s.t. Germany; 12 national quality associations Project partners x 5 Centro di Ricerca e di Servizio + Universita per Stranieri di Perugia School of Languages, Sabanci, Turkey Individual membership (2011)

7 EAQUALS Expertise Quality management:
- design of quality accreditation schemes - training and standardisation - capacity building at local level The Common European Framework of Reference for Languages (CEFR): - descriptors, curriculum, assessment, - portfolio, certification Teacher management and professional development: - teacher competences (profiling grid) - teacher training and development - academic management

8 EAQUALS Activities International accreditation of language education
- standards, inspections; recommendations Support and pathways for institutional development - consultancy; self-assessment; training resources An international professional network - regular conferences; involvement in funded projects Certificates of achievement – CEFR levels - training support, numbered certificates, website archive A rewarding professional experience Consultancy for members and applicant members. Also for external national bodies – capacity building in country 8

9 EAQUALS Projects Quality management CEFR
- ISO: standard for providers of language learning - Council of Europe: code of practice for integration of adult migrants - Leonardo: Quality evaluation of on-line teacher training, Quality Guidelines CEFR - Core Inventories (English; French) - Listening and Reading scenarios (Eng, French) - EAQUALS Certificate of achievement Teacher management & development - profiling grid for teachers (EPG Project) - teacher development framework

10 EAQUALS Accreditation
international and plurilingual across educational sectors consultative and developmental based on EAQUALS Charters

11 EAQUALS Standards designed for language education services
12 main areas of assessment specific indicators of quality for each area 50 key assessment criteria: clear and transparent derived from the Charters

12 Becoming a Member Attend an EAQUALS Conference
and get to know EAQUALS: details on Complete a Self-assessment and identify areas that may need improvement Have a One-day Advisory Visit and receive consultancy on achieving EAQUALS standard Arrange your First Inspection after working through the points raised

13 Brief introduction to the scheme
The EAQUALS Certificate of Achievement A Guarantee for Quality Assessment Brief introduction to the scheme

14 Main aims of the scheme To increase EAQUALS’ profile: the first organisation to award CEFR certificates of achievement across different languages Europe-wide To help members differentiate themselves from their competitors and demonstrate their competence To open the possibility, at a second stage, of accrediting CEFR certificates awarded by schools that are not yet accredited members of EAQUALS 14

15 Core message A guarantee for quality assessment in the accredited institutions by thoroughly screening all assessment processes CEFR-implemented approach to syllabus and curriculum design Feasibility of standardisation and moderation techniques used by the institution

16 EAQUALS Certificate of Achievement
Phase 1: voluntary – Accredited Members only: accreditation carried out as a “top up” to the EAQUALS inspection Phase 2: compulsory for Accredited Members; voluntary for schools which are members of Associate Members: the accreditation process is fully integrated into the EAQUALS inspection and carried out as a top up to inspections of Associate Members 16

17 Why EAQUALS Certificate?
Benefits for: the institution the course participants the stake holders EAQUALS profile 17

18 Main prerequisites to apply for the Certification Scheme
A) CEFR-based curriculum B) Standardisation training C) Moderation techniques

19 CEFR-based curriculum
CEFR-related level descriptions and syllabus documents the levels referenced to CEFR (CEFR criteria grids adopted or adapted to local circumstances) learning aims specified by „can-do” statements - based on ELP or CEFR scales and adapted to the institution’s own needs 2. Written guidelines for teachers on levels, teaching and assessment 3. Application of CEFR in teaching coherence between curriculum and syllabus aims and classroom practice

20 Implementing CEFR-based curriculum and syllabuses
Core of the syllabuses is the practice-oriented background of the CEFR Can-do-statements developed for each type of course and level / sub-level Can-do-statements serve as a basis for tests development Can-do-statements used as a self-assessment instrument

21 Standardisation training - requirements
Familiarisation with CEFR levels and descriptors Systematic standarisation training with CEFR illustrative samples (DVDs or scripts: the EAQUALS Standardisation packs – for English, French and German) Procedures for CEFR-novice and CEFR-experienced teachers

22 Examples of standardisation training
Means of writing standardisation (making use of local samples) Means of speaking standardisation (with official samples) Standardisation of the placement testing (with standardisation videos and other CEFR-related tasks)

23 Systematic reduction of subjectivity in the assessment.
Moderation Systematic reduction of subjectivity in the assessment. 23

24 Moderation techniques - requirements
Collective assessment techniques grading by two assessors (all performances, a structured sample or with less-experienced staff), or assessing initially as a panel Quality control liaison between teachers systematic monitoring by academic managers

25 Examples of moderation techniques
Doing buzz observations during testing periods to monitor application of assessment standards Monitoring inexperienced teachers’ assessment of spoken production and interaction / written work

26 Assessment procedures should include at least two of:
a) teacher judgements through continuous assessment in relation to CEFR/ELP descriptors b) performances in specific spoken/writen tasks assessed with CEFR-related criteria c) use of tests (progress test giving an indication of good performance - e.g.15/20), or an independent, validated “CEFR level test” reporting a CEFR level

27 Procedure to accredit assessment processes
Step 1: Introduction of a CEFR-based assessment approach Standardisation training Assessment procedures Moderation techniques Step 2: A formal application Step 3: Evaluation by the EAQUALS Assessment Panel 27

28 Introduction Please, discuss the following questions with the person sitting next to you: How often do you do standardisation training at your institutions? Is it enough? Do all teachers have to take part in it? If not, why? What problems do you face when you do standardisation training?

29 Warmer Look at the descriptors for spoken production for B1 level. Fill in the gaps with the missing salient features for the level. Compare your ideas with the person sitting next to you.

30 CEFR Speaking Skills overview
Arrange the descriptors for the six basic levels in the correct order. Compare with your answers from the previous stage with the actual descriptor for B1

31 Bell Krakow_Anna & Magda.mpeg
Oral sample Watch a short performance for 3-4 min. Decide on the speakers’ level. Bell Krakow_Anna & Magda.mpeg Write a short description of the speaking abilities of the students in the video. Compare your answers with the CEFR descriptor. Provide a rationale for your decision by referring to the most salient features of the descriptors.

32 Criteria for speaking assessment
List the basic criteria that you use for speaking assessment. Compare your ideas with a suggested set of criteria, used at one EAQUALS institution.

33 Sample mark sheet Criteria
Candidate A Candidate B Discourse management Can manage very short, isolated, mainly pre-packaged utterances, with much pausing to search for expressions, to articulate less familiar words, and to repair communication. Can link words or groups of words with very basic linear connectors like “and” or “then” Pronunciation: Can produce a limited number of words / phrases in a relatively recognizable manner and make her/himself understood in simple everyday situations but there is very noticeable interference from mother tongue in terms of individual sounds, word and sentence stress Interaction: Can ask and answer questions about personal details. Can interact in a simple way but communication is totally dependent on repetition, rephrasing and repair Grammar and vocabulary: Has a very basic repertoire of words and simple phrases related to personal details and particular concrete situations. Shows only limited control of a few simple grammatical structures and sentence patterns in a memorized repertoire. Global achievement: Overall impression mark

34 Initial standardisation
Task 1: Watch the standardisation video again and assess the speakers’ performance using the mark sheet Discuss your grades with a partner, then compare ideas with another pair

35 Thank you!

Download ppt "Applying CEFR principles to testing and assessment in practice"

Similar presentations

Ads by Google