1. Bureau for International
Language Co-ordination
BILC Standardization Efforts and a Preview of the Advanced Language Testing Seminar (ALTS)
Peggy Garza
Associate BILC Secretary for
Testing Programs

2. Bureau for International Language Co-ordination
Standardization Efforts
History of STANAG 6001
STANAG 6001, Edition 2
Language Testing Seminar (LTS)
STANAG 6001, Edition 3
Benchmark Advisory Test (BAT)
Other Initiatives
Advanced Language Testing Seminar (ALTS)

3. HISTORY OF STANAG 6001

4. STANAG 6001 Language proficiency descriptions developed by BILC
Adopted by NATO in 1976
- SLP (Standardized Language Profile)
- Listening, speaking, reading, and writing
- SLPs in job descriptions, prerequisites for training, Force Goals, Partner Goals

5. Standardization Challenges
“…English language is the foundation of interoperability…” Gen Ralston, former SACEUR
Testing is a national responsibility
NATO became more concerned with enforcing SLP requirements

6. Standardization Challenges
STANAG 6001 used without modifications since 1976, although there had been many refinements in the field to language proficiency descriptions since then
NATO/PfP interoperability issues raised common understanding of STANAG 6001 to forefront
Testing protocols diverged greatly in NATO and PfP countries

7. BILC Response Emphasis on testing at PfP Seminars
BILC Steering Committee approved formation of a Working Group (WG) on testing in 1999
To interpret and elaborate STANAG 6001 language descriptions multilateral consensus
Testing Working Group (WG) involved academics from 11 countries
STANAG 6001 Interpretation and Elaboration document produced

8. STANAG 6001, EDITION 2

9. Custodian of STANAG 6001
Interpretation and Elaboration Document became STANAG 6001, Edition 2
Promulgated by the NATO Standardisation Agency (NSA) in 2003
NATO Training Group (NTG) asked BILC to
Periodically review and update STANAG 6001
Develop a seminar to familiarize Partner nations with the STANAG 6001 standards and the methodology for testing to the standards

10. LANGUAGE TESTING SEMINAR (LTS)

11. Language Testing Seminar (LTS)
BILC collaborative effort
Developed by testing experts from 5 BILC member nations
Aim: To give NATO/PfP language teaching professionals the opportunity to practice developing test items based on STANAG 6001
Started in Fall 2000
Sponsored by Allied Command Transformation (ACT)
Hosted at the Partner Language Training Center Europe of the George C. Marshall Center

13. Language Testing Seminar (LTS)
Two-week graduate level seminar on language testing
Faculty from Canada, Denmark, Germany, Hungary, Italy, Romania, the Netherlands, Slovak Rep, UK, and USA
Since 2000, almost 300 attendees from 38 nations and 4 NATO offices

14. Language Testing Seminar
June 2009

15. STANAG 6001, EDITION 3

16. Custodian of STANAG 6001
NATO Training Group (NTG) asked BILC to update STANAG 6001
BILC Study Group reviewed STANAG 6001 in 2006 and recommended new labels for each level
BILC Testing WG developed descriptors for
plus levels

17. STANAG 6001 Label Harmonization
FORMER
0 No Proficiency
1 Elementary
2 Limited Working
3 Minimum Professional
4 Full Professional
5 Native/Bilingual
CURRENT
0 No Proficiency
1 Survival
2 Functional
3 Professional
4 Expert
5 Highly-articulate native

18. Plus Level Descriptors
Lack of intermediary levels in the STANAG 6001 brought to the attention of the Steering Committee
Range between L 2 and L3 is significant
Descriptors developed by multinational WG of BILC testing experts
Use is optional per STANAG 6001 Edition 3

19. STANAG 6001, Edition 3
Ratified by the nations and promulgated by NATO Standarization Agency (NSA) in February 2009
English and French versions
Can be downloaded from the BILC website

20. BENCHMARK ADVISORY TEST
(BAT)

21. Benchmark Advisory Test (BAT)
NTG tasking to standardize testing
BILC proposed a benchmark test against which national tests can be calibrated
Purpose: to ensure equivalency of national tests
Advisory in nature

22. Benchmark Advisory Test (BAT)
History
Survey conducted on need for a NATO-wide Benchmark Advisory Test (BAT)
Items collected from BILC member nations (Voluntary National Contributions)
Allied Command Transformation (ACT) funding for BAT was approved in December 2006

23. Benchmark Advisory Test (BAT)
Is a general language proficiency exam
Everyday survival and work-related topics
Some NATO-related topics at higher levels
Not job-specific
The Reading and Listening tests
Computer-delivered, computer-scored

24. Benchmark Advisory Test (BAT)
The Speaking test
Telephonic person-to-person
Human-scored
The Writing test
Computer-delivered
Levels reported: 0, 1, 2, 3 with optional plus levels 0 – 2

25. Benchmark Advisory Test (BAT)
Multinational BILC WG wrote test specifications
All countries invited to submit items
Web collaboration
Items posted to a secure website
BILC WG and leadership team met 2x year
14 members from 8 nations and SHAPE
Contributions from many other nations

26. BILC Benchmark Test (BAT)
Validation of Listening and Reading tests
Modified modified Angoff method
Conducted by BAT WG
Speaking and Writing tests
Tester/rater norming workshop for BAT WG

27. OTHER INITIATIVES

28. ADVANCED LANGUAGE TESTING SEMINAR (ALTS)

29. Advanced Language Testing Seminar
Need for continuing professional development for test developers identified
Stakeholders’ meeting in September 2008
Result: tentative plan for the Advanced Language Testing Seminar (ALTS)

30. Advanced Language Testing Seminar
Target audience: LTS graduates who continue to work on STANAG 6001 testing
STANAG 6001 Edition 3 standardization sessions
Focus on L3 for practical exercises
Opportunity to exchange best practices
Modular approach to seminar

31. Advanced Language Testing Seminar
Three-week seminar
Module 1: Productive skills: Speaking and Writing (one-week)
Module 2: Receptive skills: Listening and Reading (one week)
Module 3: Test Analysis (3 days)
Module 4: Managerial Issues (2 days)

32. Advanced Language Testing Seminar
Development (Jan-Aug 2009)
Each module is being developed by a two-person multinational team of BILC testing experts
Validation (24 Aug-11 Sept 2009)
Module developers will teach their modules
ALTS participants will provide feedback on the objectives, content and delivery of the modules

33. ADVANCED LANGUAGE TESTING SEMINAR (ALTS)
TOPICS
AND
ACTIVITIES

34. Discussion Questions
1. What should the job qualifications be for members of a STANAG 6001 testing team?
2. How should work responsibilities be divided up among testing team members? Discuss the tasks of test developers vs test administrators.
3. Make recommendations for recruiting and retaining testing team members.
4. Discuss the relationship between the testing team and the teachers.
5. Make recommendations for training, norming and renorming members of a testing team.
6. How can conflict arise in test within a testing team? Make recommendations for dealing with this conflict. .