3 SOME BASIC ISSUES Aviation English testing: what is required? Aviation English training: How much?
4 Language Proficiency Testing Misleading to speak about ‘a test.’Testing services are requiredCapability to constantly develop new test promptsAble to maintain high level of security in administration, record keeping, scoring
5 High Stakes Testing Safety Economics Airline industry health Careers 06/29/103High Stakes TestingCareersSafetyEconomicsAirline industry healthLong note section here: be sure to scroll all the way to //end//: That testing in the aviation industry is very HIGH STAKES. We call a test high stakes when it has a serious impact. Controllers/pilots are often mid career. They have until 2008 to learn English. Just stop for a moment. Consider your job. Your life. Your family. Now imagine if learning French or Russian to the ICAO Level 4 became an overnight requirement for you, and you had until 2008 to learn French or Russian. It would be no joke. We can make a rough, rough, estimate of 1000 hours from beginner to Level 4, minimum. (This explained more later.)Elaine Shohamy has published a recent book on the the Power of language Tests to affect people’s lives. She correctly points out the potentially negative impact tests can have on the lives of test-takers. The kind of language testing with which those of us in academia are most familiar, college entrance exams TOEFL or IELTS. These tests impact on the lives of 18, 19 year old college applicants, or graduate students. Failure means missed opportunity to attend university or to receive graduate stipends. But, now rachet up the ante. Our target audience are 35, 45 year old men and women, professionals in the middle of their careers, with a very short time frame in which to learn or acquire a fairly high level of English proficiency. How far away from Level 4 are pilots and professionals today? Depends on where you look, but one survey in Latin America found that 70 – 80% of the controllers were at the ICAO level 1 or 2. How many hours of training will they need?Training need is enormous and important.Secondly, the impact on professional careers of pilots and controllers is counter-balanced by the high-stakes impact of passenger safety. I think by and large the flying public has been mostly unaware of language issues in aviation. Assumeed English is and has been official language of aviation for many years. Without measurable criteria, difficult to talk nail down lack. Now that the ICAO has implemented proficiency criteria, this area is likely to receive more attention. Airlines will have to demonstrate compliance by 2008.Besides obvious safety impact, there also the impact on the perceived safety of flight by flying public.Another high stakes issue is that language testing and training is relatively expensive, primarily bc of time factor. Pilots, controllers, airlines deserve best practice in language teaching and testing. //END//9
6 Stakes are high‘too hard’-- a threat to airline economic health and pilot career‘too easy’-- a threat to airline safety standards
7 Aviation-Specific Testing Existing general English tests not appropriateNot aviation specificNot proficiency testsProficiency testingActual speaking performanceNot pen and paper tests
8 Test development Rater training and retraining Quality control Needs analysisSpecificationsItem writingOperational expertsLanguage testing specialistsTrialingAnalysisConstructionStandard settingGuidance materialRater training and retrainingQuality controlAdminstrativeAdministering testTest securityRecord keepingOn going test item development, including trials
9 Level 6 Level 5 English Level 4 Level 3 Level 2 Level 1 Aviation EnglishRadiotelephony CommunicationICAO PhraseologyLevel 1Level 2Level 3Level 4Level 5Level 6English“ATC, do you see fire?”“Uh, who’s ahead? Us or Ryan Air?”“There is a dog on the runway”
11 There is no magic bullet “Learn English while you sleep.”Contrary to what some programs would have you believe, there is no magic bullet for language learning. There is no substitution for time and effort. There is no substitution for time and effort.However, there are good, sound teaching methods. Pilots and controllers are busy professionals. And they deserve the highest quality, most efficient methods that the field of professional language teaching can provide.As you move to implement your language training programs, look to work with professionals with standardized, certifiable qualifications.“Speak like a diplomat in thirty days.”
13 Learner Variables Starting point Time Learner style Motivation PersonalityEnvironmentTeacher led is best.CALL or CBT only as support to real training.CALL or CBT unproven in language learning: research on teacher led. Little research on CALL: 30 minutes max= boring. Not interactive and ll all about interactions. Some assistance, but maybe not all. Children, no effect.
14 Aviation English Training How long to Level 4?200 hours for measurable improvementImmersion environmentsIntensive learningTeacher-ledAcademic programsReading; writing, grammar, listening/speakingWhat will the necessary aviation English language training look like? How long? How delivered? Where?Categorically: language learning is not like flight training: can’t predict total number of hours with any certainty. Way too many variables.However, if I have made that clear that NO ONE CAN PREDICT WITH ACCURACY HOW LONG, then let me share with you some numbers we do have so that we can make some reasonable guesses. Based on some research, and long experience.200 hours is minimum for MEASURABLE IMPROVEMENT. That doesn’t mean 200 hours to “LEVEL 4.” It means that less than 200 hours of language training produces no measurable improvement of language skills.However, these numbers come from immersion (English speaking country) intensive (25 hrs week) academic programs. With reading, writing, and grammar study, in addition to speaking and listening. And they are in teacher-led programs.
15 Canadian Civil Service “True beginner” (ICAO Level 1)ToCanadian Civil Service Level 3(can describe process but not explain process.1300 Hours
16 Aviation English Training 100 hours @ DLI Level Level 2Defense Language Institute, USAImmersionIntensiveTeacher ledSpeaking/Listening OnlyAviation specificAdditional info from Defence Language Institute in USA. LI: 100 hours level 1 to level 2. Level 2 is a low level 4 icao, roughly. Based on this scant research and long experience in language learning, we can generally estimate a minimum of 100 hours for a measurable progress, wbut with much variation according to learner starting point, motivation, teachng method used; in country or in an English speaking country. A broad (wild, really) estimate might be between 800 and 1,200 learner contact hours required from level 0/1 to level 4. Minimum of 6 months full time intensive immersion study for true beginner to reach level 4.CBT for language learning is interesting but not proven and so far not very useful for real communicative language learning.
17 Aviation English training: Best Practice Teacher-ledImmersionIntensiveTESL qualified teacher/material developerCommunicative, interactiveLearner-centeredAviation-specificTeacher led is best.CALL or CBT only as support to real training.CALL or CBT unproven in language learning: research on teacher led. Little research on CALL: 30 minutes max= boring. Not interactive and ll all about interactions. Some assistance, but maybe not all. Children, no effect.
18 Teacher Training Programs STANDARDTOEFLAcademiaTestsPersonnelTeacher Training ProgramsProgramsIn academia, a bottom-up approach to the development of infrastructure to support the STANDARD was applied, an organic approach. First students came to US and UK, universities hired teachers to help them with English, Slowly, materials were developed, eventually published, when the market became evident, teacher training programs were implemented, and tests were developed. At some point, a few certain tests became to be used as the “Standard” the gate keeping tests. Now, accrediting organizations (notably the Commission on English Accreditation in DC) and many teacher training organizations support the academic English market.Materials
21 Threats and Challenges to effective language TEACHING …”If you are a native speaker and can fog a mirror, then you can teach English abroad.”Recruiter for English teachersQuoted in Time Magazine,June 30, 2003To illustrate what I mean,It is just that the stakes are so high in aviation training and testing that the need for efficient training and reliable, valid testing becomes ever so much more critical. A main purpose of the GM is to guide States away from “the business” of English teaching towards “the profession,” to set and keep the standards high, so that the men and women facing whatever testing regimes are developed can as far as possible be assured of receiving the best our field has to offer. For the professional pilots/controllers that is our target audience, the notion that anyone who speaks English can teach English is not good enough.An additional challenge is that the market for aviation English testing is relatively limited, perhaps 100,000 to 150,000 pilots and controllers requiring testing, compared to the 3 to 4 million tests given per year by large testing organizations (TOEFL, for example.) Additionally, aviation English test development requires a more expensive collaboration between Applied linguists and pilots and controllers. The expense compared to the relatively limited market means there may be little interest from large testing agencies.So, the stakes are high. The need is great. How can we respond?
22 Threats and challenges in language testing 06/29/103Threats and challenges in language testingHigh stakes: careers versus safetyRequires specialized knowledge, beyond language teacher knowledge-baseUsers usually lack awareness of professional requirementsTest development is expensive2008 Deadline creates urgencyLarge testing and training marketLanguage testing is unregulatedInto this high stakes arena, lets look at some real threats and challenges. Language testing in the context of aviation faces some unique threats and challenges.Firstly, language testing requires specialized knowledge (linguistic and testing expertise). Language test development, done correctly, is expensive. Requires lengthy academic trialing and analysis. Requires input from not only linguistic test experts, but also from pilots and controllers, from all stakeholders.Yet, in the face of the requirements for high academic validity, the language testing industry is largely unregulated. This leaves the market open to poor quality or, even, potential fraud. Additionally, language test development is costly: it cost the same to develop a test for 50 candidates as it does to develop one for 500 or 50,000 candidates, in terms of development and academic trialing costs.12
23 Threats and challenges to language training 06/29/103Threats and challenges to language trainingPerception that English teaching easyEffective and efficient Aviation English training requires specialized knowledgeMuch training requiredPilots and controllers cannot come off line easilyNot much aviation-specific English materials currently available.Unregulated industryInto this high stakes arena, lets look at some real threats and challenges. Language testing in the context of aviation faces some unique threats and challenges.Firstly, language testing requires specialized knowledge (linguistic and testing expertise). Language test development, done correctly, is expensive. Requires lengthy academic trialing and analysis. Requires input from not only linguistic test experts, but also from pilots and controllers, from all stakeholders.Yet, in the face of the requirements for high academic validity, the language testing industry is largely unregulated. This leaves the market open to poor quality or, even, potential fraud. Additionally, language test development is costly: it cost the same to develop a test for 50 candidates as it does to develop one for 500 or 50,000 candidates, in terms of development and academic trialing costs.12
24 Role of General English Basic LevelsUntil Aviation-specific material more availableWhy aviation English at low levels/InterestMotivationRelevance
25 Practical suggestions Document 9835TESL input especially for test DEVELOPMENT and curriculum developmentStart early. Stay the course.Believe it is possible!If I can leave you with anything today, it is that as your organizations seek to implement training programs in compliance with ICAO language proficiency requirements, you should keep in mind that Language teaching is a specialized activity. It is further distinguished from other teaching activities because of the unique nature of language learning: it is a complex blend of skill, knowledge, and cultural awareness, combining physical components with mental, communicative, and social processes. Professionally prepared language teachers study linguistics, so they can explain how the language works. They study language learning theories, about how humans acquire languages, and they learn specific methods, among other subjects appropriate to language teaching.