1. Training for Proficiency or Training for success in a test? Philip Shawcross ICAEA ICAO LPRI workshop, ENAC, Rome, 4 th March 2010

2. Proficiency or success? Training for Proficiency or Training for success in a test? 1. What are the concerns? 2. Forms of negative washback 3. Different types of assessment 4. How can proficiency be trained for? 5. How do we know we are getting it right? 6. Conclusion

3. What are the concerns? (1)  Wide variety of test types and quality  No international test accreditation process  A test may have too narrow a focus  A test may not adequately assess all skills  A test may not sufficiently simulate operational conditions  Many tests have well-defined & well known tasks  Encourages use of “easiest” test

4. What are the concerns? (2)  Training to a test does not necessarily prepare for the real world  “Taking our eye off the ball”  Detrimental to learners long-term view of language  Targeted proficiency has wider scope than any test  Proficiency to be defined in operational terms  Need for sustainable levels of proficiency

5. Forms of negative washback (1)  Communication content may be limited by test content  Communication functions may be limited by test format  Particular skills (e.g. interaction) may not be sufficiently trained  Tendency to train to narrow focus  Excessively modular approach to language

6. Forms of negative washback (2)  Tendency to train only to minimum level  Training may “ape” test rather than prepare for real world  Tendency to learn “parrot-fashion”  Lack of separation between training and testing processes and staff  Publication of a bank of test questions

7. Different types of assessment  Benchmark / prognostic tests  Placement tests  Entry tests  Progress tests  Exit tests  Proficiency tests (licensing)

8. How can proficiency be trained for? (1) Training should have its eye on:  The “big” picture  The long term  Operational realities  Whole range of communicative functions  The acid test of communicating safely  Ability to manage the unexpected  A safety margin  Sustainability

9. How can proficiency be trained for? (2) How can this be put into practice? Aviation English training should:  Be modelled on operations, not testing  Provide sufficient practice to achieve a solid level of proficiency  Be approved by SMEs  Explore a wide range of situations  Be applicable to operational situations (lexis) and communicative functions (structure)

10. How can proficiency be trained for? (3)  Take students beyond minimum Level 4  Prioritize expression, fluency & interactions (classroom)  Adopt a holistic approach to language  Make language training a corporate project  Incorporate language maintenance strategies & recurrent training  Receive feedback from operations for adjustment

11. How can proficiency be trained for? (4) … and more generally  Foster operational effectiveness rather than linguistic correctness  Develop appropriate cognitive processes  Encourage flexibility & adaptability in language use  Respect the students’ different learning styles & speeds  Build self-confidence & self-esteem

12. How do we know we are getting it right?(1)  Integrate language training as a core institutional activity, not a peripheral activity ---> a closed loop for greater awareness & visibility  Do not only rely on progress & exit tests  --  more holistic assessment  Develop students’ own pedagogical self- awareness  Consult SMEs on students’ performance

13. How do we know we are getting it right?(2)  Use feedback from operational monitoring  Pay particular attention to holistic skills (fluency, interactions & discourse management) as criteria of proficiency  Ask “Would I entrust my life to this person in an emergency?”

14. Conclusions Training  High stakes  Operations-driven  Integrated set of skills  Targets sustainable proficiency  Part of life-long learning

15. Training for Proficiency or Training for success in a test? Thank you very much for listening philip.shawcross@icaea.net ICAO LPRI workshop, ENAC, Rome, 4 th March 2010