1. Bureau for International Language Coordination
Julie J. Dubeau
Jana Vasilj-Begovic
BILC Secretaries
Varna, Bulgaria
October 11, 2010

2. Outline What/Who is BILC? What do we do? Custodians of STANAG 6001
Assistance to Training Programmes
Visits
Assistance to Testing Programmes
Visits, Conferences, Seminars & Workshops
Special Projects and Events
Projects

3. BILC History
NATO’s consultative and advisory body for language training and testing issues
Established in The founding members are France, Germany, Italy, the United Kingdom, and the United States. Subsequently, the following countries joined:
1967: Belgium, Canada, Netherlands 1975: SHAPE and IMS/NATO (non-voting members) 1978: Portugal 1983: Turkey 1984: Denmark and Greece 1985: Spain 1993: Norway 1999: Czech Republic, Hungary, Poland 2004: Bulgaria, Estonia, Latvia, Lithuania, Romania, Slovakia, Slovenia
2009: Albania and Croatia
PfP countries routinely participate. MD countries are welcome. 3

4. BILC STRUCTURE The BILC Chairman presides over the Steering Committee
The voting members of the Steering Committee are the heads of each full member delegation.
Secretariat staffed by:
United Kingdom ( )
Germany ( )
United States ( )
Canada (2008- )
Projects

5. BILC Mission and Vision
To promote and foster interoperability among NATO and PfP nations by furthering standardization of language training and testing. To support the Alliance's operations through the exchange of knowledge and best practices, IAW established procedures and agreements.
Vision
To achieve levels of excellence where progress made by one is shared by all.

6. Two Main Events May 2011 hosted by Lithuania -
Annual Conference in Spring (May)
May 2011 hosted by Lithuania -
“Defining Operational Proficiency”
Professional Seminar in Fall (Oct)
October 2011 hosted by U.S.A. –
“Furthering Our Training Goals Through Research”

7. STANAG 6001 - Custodian Edition 3 promulgated by NSA in Feb 2009
English and French versions can be downloaded from the BILC website
Secretariat proposed admin change during Istanbul conference last May
Change is in progress at NSA
6. At Appendix 1 to Annex A, a series of plus (+) descriptions is provided. The para in ed 3 said : A plus indicator may be added to a base level for training and evaluation purposes, to indicate a level of proficiency that substantially exceeds a 0 through 4 base skill level, but does not fully or consistently meet all of the criteria for the next higher base level. and now it will say : A plus indicator may be added to a base level for training, evaluation, recording or reporting purposes, to indicate a level of proficiency that substantially exceeds a 0 through 4 base skill level, but does not fully or consistently meet all of the criteria for the next higher base level. This will better reflect the fact that nations can report plus profiles if they wish to do so. It came to our attention that one nation had encountered roadblocks due to this technicality.

8. What does BILC do? Collaborative Language Training Assessments
(K. Wert)
Goal: to assess language training and testing programmes and offer recommendations
2010: Georgia, Macedonia, Serbia
2011: Albania
One type of assistance BILC has been providing to nations under the leadership of our associate secretary Mr. Keith Wert have been collaborative language training assessments. As their name reveals, the goal of these assessments is to look evaluate language training an testing programmes and offer recommendations on possible or desirable improvements. A team of BILC experts led by Mr Wert pays visits to requesting nations. In 2010, such visits were paid to Geo Mac and Serbia, and in 2011 Albania will be the recipient of this form of assistance.

9. What does BILC do?
Assistance to National Testing Programmes (P. Garza)
Goal: to assist nations in establishing, restructuring or improving their testing systems
Bosnia-Herzegovina: 2008-present
Workshops and assistance visits
Spain: 2009, 2010
Workshop and assistance visit
Georgia: 2009-present
BiH and Georgia: June 2010
Cooperative event on STANAG 6001 testing
Our other associate secretary, Peggy Garza is responsible for assistance to national testing programmes. The goal of this programme is to assist nations in the establishing, restructuring or improving their testing systems. This is accomplished through visits, workshops, and seminars. BIH, Spain and Geo were the nations who benefitted from this programme in An event worth mentioning was the meeting between the BIH and Georgian test specialists during which they exchanged experiences on STANG testing.

10. Language Testing Seminars (LTS) and (ALTS)
Developed and conducted by BILC testing experts
Objective: To provide NATO/PfP language testing professionals with the theory and practice of test development based on STANAG 6001
LTS- attended by more than 325 graduates from NATO/PfP/MD nations
ALTS- originated from need for further professional development/piloted in 2009
The LTS was designed and developed in 1999 by a group of BILC experts. The rationale behind this course was to train testers or teachers who were about to become testers in a common interpretation of the STANAG 6001, as well as to provide them with the theory and practice of language test development in all four skills. The need for further professional development in the field of testing was the force behind the creation of the Advanced language testing seminar.

11. LTS ALTS Introduction to STANAG 6001, Ed. 3
Focus on L2
All four skills integrated
Practical, interactive small group exercises
Emphasis of test development process
Limited statistical analysis
STANAG Standardization exercises
Focus on L3
Modular structure
Practical, interactive small group exercises
Moderation of already developed items
More in-depth statistical analysis
This slide outlines the basic differences between the two seminars. While the LTS introduces the STANAG 6001, ed. 3, through standardization exercises, the ALTS aims at fortifying the participants’ understanding of how this scale is interpreted. The LTS focuses on L2, whereas the focus of the ALTS is level 3 testing. And as mentioned, the ALTS has 3 distinct modules, whereas the skills are integrated in the LTS. Module 1, in the ALTS deals with the receptive skills (listening and reading) and is one week lone, module 2, also a week long, deals with the productive skills (speaking and writing), module 3 focuses on test analysis over 3 days (qualitative and quantitative) and managerial issues over two days of training. One significant difference is that in the LTS, participants practice test item development, while in the ALTS, they moderate the already developed test items. The statistical analysis of tests is limited in the LTS, and is more detailed in the ALTS.

12. Special Projects and Events: Level 4 Analysis
In Istanbul, BILC Steering Committee recommended to form a working group to:
Analyse Level 4
Study feasibility of testing at Level 4
Develop test specifications if need established
Status: Level 4 Working Group formed
Bulgaria, Canada, Croatia, Denmark, Germany, Italy, Netherlands and USA
Parallel work to be done on PE posts analysis in order to establish whether level 4 linguistic designations match STANAG 6001 level 4 descriptors (in planning)
Presently, a special project is in progress. Based on the outcome of a study group that convened at the Istanbul BILC conference and the SC recommedations a working group was formed the objective of which has been to analyse level 4 proficiency, study feasibility of testing at this level and eventually, if a clear need for testing at this level is established to develop test specifications. The rationale for doing this work stems from the fact that certain nations expressed their concern that they are expected to conduct tetsing at this level and that they do not feel well prepared or resourced for this task.
The WG was formed and is comprised of representatives from BUL, CA, CRO, DEN, ITA, Netherlands and USA
It’s important to mention that parallel to this project, an analysis of PE posts is to be conducted in order to establish if the posts designated as level 4 do indeed match the tasks as described in the STANAG 6001 as level 4.

13. Level 4 Analysis First meeting held:
Sarajevo STANAG 6001 Speaking Conference, August 2010
WG members to:
produce a draft paper on the conceptual model for all four skills
revise proposed models through online collaboration
meet at the BILC Conference in Vilnius to finalize documents and formulate recommendations
The WG held their first meeting in Sarajevo.
The WG now plans to produce a draft paper describing the conceptual model of level 4 proficiency for all 4 skills. Until the group meets at the BILC conference in Vilnius, work will be done online. We hope that we’ll be able to finalize this work in Vilnius and present our findings at the next BILC seminar.

14. Testing Conference Sarajevo 2010
Standardizing Approaches to Testing Speaking
Specific goals
Increase understanding of the STANAG 6001 descriptors for speaking
Compare STANAG 6001 speaking test protocols
Improve elicitation techniques
Standardize the rating of speaking tests
Identify the challenges of testing speaking and discuss possible solutions
In collaboration with the US Marshall Center, BILC organized a testing conference in Sarajevo under the theme title Standardizing Approaches to Testing Speaking. The specific goals were to:
It gives me much pleasure to report that this event was highly successful and provided the participants with the opportunity for further professional development and exchange of best practices. Through a mixture of presentations and hands-on workshops, the conference represented another step toward standardizing the way we test speaking and interpret speaking proficiency levels.

16. QUESTIONS?
Enjoy the Seminar!