Language teaching and language assessment in a multilingual Europe. A case of quality, diversity and transparency Piet van Avermaet Centre for Language and Migration KULeuven

Economical, cultural and social integration (unification?) in Europe Language Policy Context in Europe National interests (19th century: one nation, one state, one language)

Multilingual policy in Europe Language Policy Context in Europe: paradoxes Monolingual policy in different European countries

Positive attitude in Europe towards use of more than one language Language Policy Context in Europe: paradoxes Negative attitude in European countries towards use of non European languages

Society handles with language diversity in a schizophrenic way Language Policy Context in Europe Argue in favour of multilingualism Demand/require the use of one language Diversity is important and has an extra value Unity, nation building, uniformity

Language Policy Context in Europe Striving for monolingualism is easier to argue for Surplus value of multlingualism is difficult to make a case for; difficult to bring up arguments Economical,practicalVague surplus value of cultural and linguistic diversity

Language and dealing with languages is a continuous balancing between Language Policy Context in Europe Uniformity clarity transparency economical way of thinking consistency straightforwardness Diversity identity be opposed to react against

There will always be a tendency towards uniformization and there will always be a counter reaction by people Language Policy Context in Europe Process of language evolution and language change. Languages are not static but dynamic. It is important to have people who stress nowadays the value of diversity

Now two major tendencies Language Policy Context in Europe In a climate of globalisation and economic thinking there is a tendency towards uniformization and uniformity (one language/one standard) People and media making more and more use of different communication modes: pictures/graphs/…/ linguistic code mixing Policy / perception / discourseDaily behaviour

In the context just described, recent developments in Europe like Bologna agreement, Barcelona indicators, the CEF and the Manual have to be situated and interpreted. Bologna agreement, Barcelona indicators, CEF and Manual Keywords are diversity and transparency Often interpreted and translated in many different ways: diversity often only at level of European official languages transparency often as uniformity and loss of specificity

One concept people often forget to mention or it is often taken for granted Bologna agreement, Barcelona indicators, CEF and Manual quality in language teaching quality in language assessment

Define for yourself: quality, diversity and transparency Organise teaching and assesment along these lines: levels and content Make a clear statement Consequences of Bologna and CEF for University Language teaching Institutes

Original aims What? CEF as POINT OF REFERENCE Background and role of CEF

Only the overall proficiency scale is known/used Educational misuse Assessment misuse Ethical misuse Misinterpretations, misuse and impact of CEF

Qualityneeds, context and functionality reliability, validity, efficiency, impact/effect Diversity (linguistic and language) and transparency needs analysis domains description of aims/context/level relate to CEF Bologna - CEF and language teaching/assessment

Instruments CEF itself Manual* Population/students ALTE/COP/checklists/minimum standards* Bologna - CEF and language teaching/assessment

Evaluation is gathering and judging (assessing) students PERFORMANCES 3 central questions: Why? What? How? Language assessment: tests and alternative assessment

Shifting on 4 axes. Position on each axis depends on answer on 3 question TIME INVOLVEMENT IN SCORING/MARKING TYPE OF DATA/INFORMATION GATHERED WAYS OF GATHERING INFORMATION/DATA Alternative assessment

Alternative assessment is NEW and NOT NEW NOT: we do it daily NEW:try to make it more formal/explicit. Give it a higher status in making/taking decisions for reasons of reliability, validity and efficiency Alternative assessment

Two examples ALTECode of Practice Checklists Minimum standards CEF Manual to relate examinations to CEF Quality, diversity and transparency

The ALTE Code of Practice Aim - to guarantee fairness for the users of the examinations Two main groups of user: Primary users are the candidates who take the exams and whose lives are affected by the results Secondary users include: sponsors of the candidates, teachers, parents, funding agencies, employers, language teaching institutions, etc.

The Code of Practice identifies the roles of three groups of stakeholder in the testing process: the examination developers - e.g. members of ALTE the examination takers - primary users - who take the examinations by choice, direction or necessity the examination users – secondary users - who require the examination for some decision-making or other purpose Developing the ALTE Code of Practice

The Code of Practice lays down four broad areas of responsibility: developing examinations interpreting examination results striving for fairness informing examination takers Developing the ALTE Code of Practice

Quality Considerations for ALTE Members Two aspects of Quality Better understanding of Principles of Good Practice - theoretical concerns and knowledge The practical application of the principles within each ALTE organisation developing better systems managing innovation and change monitoring standards

The ALTE Principles of Good Practice Based on VRIP features Validity Reliability Impact Practicality Plus additional feature Quality of service

How to set ALTE Quality Standards?

QMS approach: Monitor quality through self-assessment Seek confirmation that standards are being met e.g. through peer review within ALTE ALTE Quality Management System - QMS

Discuss and agree on minimum standards B ut establish best practice models as long-term targets Establish desired outcomes and impacts within each member organisation Move towards good practice Aim at continuous improvement Applying an ALTE QMS

BUT each ALTE organisation is different !

Adopt the QMS approach and the revised Code of Practice checklists Apply the checklists to the 4 aspects of the Test Development and Administration Cycle Test Design and Construction Administration Processing - marking, grading, issue of results Analysis and Review ALTE QMS

Identify within each organisation: Current strengths Areas in need of immediate improvement Areas for long-term development Jointly agree on activities to support ALTE Members in raising standards: e.g. workshops, sub-groups projects etc. ALTE QMS

Manual Preliminary Pilot Version Relating Language Examinations to the CEF

What the brief was….. Provide guidelines and suggest procedures to facilitate a common understanding of the CEF levels.

Project Timeline July 2002 Helsinki seminar September Project approval November Authoring group starts work Nov-January 2003 Consultation on structure Jan-June Consultation on content June Consultants meeting (v.9) September Preliminary Pilot version January 2004 Standardised videos and benchmarked items

The Authoring Group Coordinator: Brian North Neus Figueras Sauli Takala Piet van Avermaet Norman Verhelst + Sounding board Consultants

The Manual offers guidance to users to... describe the examination coverage, administration and analysis procedures, relate results reported from the examination to the Common Reference Levels presented in Chapter 3 of the CEF, provide supporting evidence tht reports the procedures followed to do so.

The Manual …. provides a guide specifically focussed on procedures involved in the validation of a claim that a certain examination or test is linked to the CEF, but

The Manual does not…. provide a general guide how to construct good language tests or examinations, or prescribe any single approach to constructing language tests or examinations.

Structure of the pilot version Chapter 1: Introduction Chapter 2: The Common European Framework Chapter 3: Familiarisation Chapter 4: Specification Chapter 5: Standardisation Chapter 6: Empirical validation Chapter 7: Guidelines for Reporting Appendix: Report forms

To be followed by..... Benchmarked performances Benchmarked reading items Reference supplement

Approach adopted Four inter-related sets of procedures which contribute to the validation process: Familiarisation Specification Standardisation Empirical Validation

Graphical presentation of examination(s) related to CEF levels C2 C1 B2 B1 A2 A1 overalllistening reading spoken spoken written written interaction production interaction production

Contributions most welcome! Feedback, on the reading of the Manual Piloting, on the basis of using the Manual A Case Study, writing up the experience in using the Manual Collecting Performance Samples for Speaking and Writing

Language teaching and language assessment in a multilingual Europe. A case of quality, diversity and transparency Piet van Avermaet Centre for Language and Migration KULeuven