Presentation is loading. Please wait.

Presentation is loading. Please wait.

Applying CEFR principles to testing and assessment in practice ACERT Conference 15 th March 2014, Prague Mila Angelova.

Similar presentations


Presentation on theme: "Applying CEFR principles to testing and assessment in practice ACERT Conference 15 th March 2014, Prague Mila Angelova."— Presentation transcript:

1 Applying CEFR principles to testing and assessment in practice ACERT Conference 15 th March 2014, Prague Mila Angelova

2 Aims of the session To give you a short update on EAQUALS operation and activitiesTo give you a short update on EAQUALS operation and activities To introduce you to the EAQUALS Certification SchemeTo introduce you to the EAQUALS Certification Scheme To (re)-introduce you to the most salient features of the six global levels according to CEFRTo (re)-introduce you to the most salient features of the six global levels according to CEFR To assess two oral performances for standardisation purposesTo assess two oral performances for standardisation purposes

3 The Association Founded in 1991Founded in 1991 A company registered in UK as charityA company registered in UK as charity International NGO with participatory status at the Council of Europe (2006)International NGO with participatory status at the Council of Europe (2006)

4 EAQUALS Mission The mission of EAQUALS is: to contribute to the development of excellence in language educationto contribute to the development of excellence in language education to be the internationally recognised authority for quality standards and quality assurance in language education and trainingto be the internationally recognised authority for quality standards and quality assurance in language education and training

5 EAQUALS Values PlurilingualismPlurilingualism Intercultural understandingIntercultural understanding International cooperationInternational cooperation Lifelong learningLifelong learning

6 EAQUALS Members A network connecting 35 countries Accredited members x 112 25 languages, private and state sector Associate members x 27 British Council; Instituto Cervantes; CIEP France; Goethe-Institut; Cambridge Assessment; Trinity London; g.a.s.t. Germany; 12 national quality associations Project partners x 5 Centro di Ricerca e di Servizio + Universita per Stranieri di Perugia School of Languages, Sabanci, Turkey Individual membership (2011)

7 EAQUALS Expertise Quality management: - design of quality accreditation schemes - training and standardisation - capacity building at local level The Common European Framework of Reference for Languages (CEFR): - descriptors, curriculum, assessment, - portfolio, certification Teacher management and professional development: - teacher competences (profiling grid) - teacher training and development - academic management

8 International accreditation of language education - standards, inspections; recommendations Support and pathways for institutional development - consultancy; self-assessment; training resources An international professional network - regular conferences; involvement in funded projects Certificates of achievement – CEFR levels - training support, numbered certificates, website archive EAQUALS Activities

9 EAQUALS Projects EAQUALS Projects Quality managementQuality management - ISO: standard for providers of language learning - ISO: standard for providers of language learning - Council of Europe: code of practice for integration of adult migrants - Council of Europe: code of practice for integration of adult migrants - Leonardo: Quality evaluation of on-line teacher training, Quality Guidelines - Leonardo: Quality evaluation of on-line teacher training, Quality Guidelines CEFRCEFR - Core Inventories (English; French) - Core Inventories (English; French) - Listening and Reading scenarios (Eng, French) - EAQUALS Certificate of achievement - EAQUALS Certificate of achievement Teacher management & developmentTeacher management & development - profiling grid for teachers (EPG Project) - profiling grid for teachers (EPG Project) - teacher development framework - teacher development framework

10 EAQUALS Accreditation international and plurilingualinternational and plurilingual across educational sectorsacross educational sectors consultative and developmentalconsultative and developmental based on EAQUALS Charterbased on EAQUALS Charters

11 EAQUALS Standards designed for language education servicesdesigned for language education services 12 main areas of assessment12 main areas of assessment specific indicators of quality for each areaspecific indicators of quality for each area 50 key assessment criteria:50 key assessment criteria: -clear and transparent -derived from the Charters

12 Becoming a Member Attend an EAQUALS Conference and get to know EAQUALS: details on www.eaquals.org Complete a Self-assessment and identify areas that may need improvement Have a One-day Advisory Visit and receive consultancy on achieving EAQUALS standard Arrange your First Inspection after working through the points raised

13 The EAQUALS Certificate of Achievement A Guarantee for Quality Assessment Brief introduction to the scheme

14 Main aims of the scheme To increase EAQUALS’ profile: the first organisation to award CEFR certificates of achievement across different languages Europe- wideTo increase EAQUALS’ profile: the first organisation to award CEFR certificates of achievement across different languages Europe- wide To help members differentiate themselves from their competitors and demonstrate their competenceTo help members differentiate themselves from their competitors and demonstrate their competence To open the possibility, at a second stage, of accrediting CEFR certificates awarded by schools that are not yet accredited members of EAQUALSTo open the possibility, at a second stage, of accrediting CEFR certificates awarded by schools that are not yet accredited members of EAQUALS

15 Core message A guarantee for quality assessment in the accredited institutions by thoroughly screening all assessment processes CEFR-implemented approach to syllabus and curriculum design Feasibility of standardisation and moderation techniques used by the institution

16 EAQUALS Certificate of Achievement EAQUALS Certificate of Achievement voluntary – Accredited Members only: accreditation carried out as a “top up” to the EAQUALS inspectionPhase 1: voluntary – Accredited Members only: accreditation carried out as a “top up” to the EAQUALS inspection Phase 2: compulsory for Accredited Members; voluntary for schools which are members of Associate Members: the accreditation process is fully integrated into the EAQUALS inspection and carried out as a top up to inspections of Associate MembersPhase 2: compulsory for Accredited Members; voluntary for schools which are members of Associate Members: the accreditation process is fully integrated into the EAQUALS inspection and carried out as a top up to inspections of Associate Members

17 Why EAQUALS Certificate? Benefits for: the institution the course participants the stake holders EAQUALS profile

18 Main prerequisites to apply for the Certification Scheme A) CEFR-based curriculum B) Standardisation training C) Moderation techniques

19 CEFR-based curriculum 1.CEFR-related level descriptions and syllabus documents o the levels referenced to CEFR (CEFR criteria grids adopted or adapted to local circumstances) o learning aims specified by „can-do” statements - based on ELP or CEFR scales and adapted to the institution’s own needs 2. Written guidelines for teachers on levels, teaching and assessment 3. Application of CEFR in teaching o coherence between curriculum and syllabus aims and classroom practice

20 Implementing CEFR-based curriculum and syllabuses Core of the syllabuses is the practice- oriented background of the CEFRCore of the syllabuses is the practice- oriented background of the CEFR Can-do-statements developed for each type of course and level / sub- levelCan-do-statements developed for each type of course and level / sub- level Can-do-statements serve as a basis for tests developmentCan-do-statements serve as a basis for tests development Can-do-statements used as a self- assessment instrumentCan-do-statements used as a self- assessment instrument

21 Standardisation training - requirements Familiarisation with CEFR levels and descriptors Familiarisation with CEFR levels and descriptors Systematic standarisation training with CEFR illustrative samples Systematic standarisation training with CEFR illustrative samples (DVDs or scripts: the EAQUALS Standardisation packs – for English, French and German) Procedures for CEFR-novice and CEFR-experienced teachers Procedures for CEFR-novice and CEFR-experienced teachers

22 Examples of standardisation training Means of writing standardisation (making use of local samples)Means of writing standardisation (making use of local samples) Means of speaking standardisation (with official samples)Means of speaking standardisation (with official samples) Standardisation of the placement testing (with standardisation videos and other CEFR-related tasks)Standardisation of the placement testing (with standardisation videos and other CEFR-related tasks)

23 Moderation Systematic reduction of subjectivity in the assessment Systematic reduction of subjectivity in the assessment.

24 Moderation techniques - requirements Collective assessment techniquesCollective assessment techniques o grading by two assessors (all performances, a structured sample or with less-experienced staff), or o assessing initially as a panel Quality controlQuality control o liaison between teachers o systematic monitoring by academic managers

25 Examples of moderation techniques Doing buzz observations during testing periods to monitor application of assessment standardsDoing buzz observations during testing periods to monitor application of assessment standards Monitoring inexperienced teachers’ assessment of spoken production and interaction / written workMonitoring inexperienced teachers’ assessment of spoken production and interaction / written work

26 Assessment procedures should include at least two of: teacher judgements through continuous assessment in relation to CEFR/ELP descriptors a) teacher judgements through continuous assessment in relation to CEFR/ELP descriptors b) performances in specific spoken/writen tasks assessed with CEFR-related criteria c) use of tests (progress test giving an indication of good performance - e.g.15/20), or an independent, validated “CEFR level test” reporting a CEFR level

27 Procedure to accredit assessment processes Step 1: Introduction of a CEFR-based assessment approach »Standardisation training »Assessment procedures »Moderation techniques Step 2: A formal application Step 3: Evaluation by the EAQUALS Assessment Panel

28 Introduction Please, discuss the following questions with the person sitting next to you: oHow often do you do standardisation training at your institutions? Is it enough? oDo all teachers have to take part in it? If not, why? oWhat problems do you face when you do standardisation training?

29 Warmer Look at the descriptors for spoken production for B1 level. Fill in the gaps with the missing salient features for the level.Look at the descriptors for spoken production for B1 level. Fill in the gaps with the missing salient features for the level. Compare your ideas with the person sitting next to you.Compare your ideas with the person sitting next to you.

30 CEFR Speaking Skills overview Arrange the descriptors for the six basic levels in the correct order. Arrange the descriptors for the six basic levels in the correct order. Compare with your answers from the previous stage with the actual descriptor for B1 Compare with your answers from the previous stage with the actual descriptor for B1

31 Oral sample Watch a short performance for 3-4 min. Decide on the speakers’ level.Watch a short performance for 3-4 min. Decide on the speakers’ level. Bell Krakow_Anna & Magda.mpeg Bell Krakow_Anna & Magda.mpeg Write a short description of the speaking abilities of the students in the video.Write a short description of the speaking abilities of the students in the video. Compare your answers with the CEFR descriptor.Compare your answers with the CEFR descriptor. Provide a rationale for your decision by referring to the most salient features of the descriptors.Provide a rationale for your decision by referring to the most salient features of the descriptors.

32 Criteria for speaking assessment List the basic criteria that you use for speaking assessment. Compare your ideas with a suggested set of criteria, used at one EAQUALS institution.

33 Sample mark sheet Criteria Candidate A Candidate B Discourse management Can manage very short, isolated, mainly pre-packaged utterances, with much pausing to search for expressions, to articulate less familiar words, and to repair communication. Can link words or groups of words with very basic linear connectors like “and” or “then” 1 2 3 4 5 Pronunciation: Can produce a limited number of words / phrases in a relatively recognizable manner and make her/himself understood in simple everyday situations but there is very noticeable interference from mother tongue in terms of individual sounds, word and sentence stress 1 2 3 4 5 Interaction: Can ask and answer questions about personal details. Can interact in a simple way but communication is totally dependent on repetition, rephrasing and repair 1 2 3 4 5 Grammar and vocabulary: Has a very basic repertoire of words and simple phrases related to personal details and particular concrete situations. Shows only limited control of a few simple grammatical structures and sentence patterns in a memorized repertoire. 1 2 3 4 5 Global achievement: Overall impression mark 1 2 3 4 5

34 Initial standardisation Task 1: Watch the standardisation video again and assess the speakers’ performance using the mark sheetWatch the standardisation video again and assess the speakers’ performance using the mark sheet Discuss your grades with a partner, then compare ideas with another pairDiscuss your grades with a partner, then compare ideas with another pair

35 Thank you! www.eaquals.org


Download ppt "Applying CEFR principles to testing and assessment in practice ACERT Conference 15 th March 2014, Prague Mila Angelova."

Similar presentations


Ads by Google