1. Weaving Pathways: Interculturalism and Language
TESL Canada Conference 2009
Weaving Pathways: Interculturalism and Language

2. ELTPA Assessment Tool for Placement in ELT Programs
Audrey: Welcome introductions, and agenda
Materials needed: Laptop, LCD, Speakers
Test Kit + video of Speaking Assessment (Zeeshan or Haider)
Carolyn Cohen has worked in the field of ESL for 20 years. She began as an ESL Teacher and Teacher Trainer. She worked as a TSE evaluator and LINC Assessor and became fully engaged in LINC and ESL as a programs and services manager. She managed development of the CLBA/LA, the ELTPA, the CLBA International and the original form of the CLBPT and CLB Online self-assessment in partnership with the CCLB. Carolyn is Senior Project Advisor for the Centre for Education & Training where her work areas include the National ELTPA Support Services, CLB On-Line Self Assessment, and Language Assessment Support Services. In November 2008, Carolyn was recognized by TESL Ontario as one who has demonstrated significant ongoing commitment to our profession.
Audrey Bennett is the Project Manager for the National ELTPA Support Services. She has worked at the Centre for Education & Training since 1992 in assessment, coordination of LINC Assessment Centres, training assessors in the use of CLB-based assessment tools, including CLBA, CLABLA, CLBA-International and ELTPA.
presented by:
Centre for Education and Training
Carolyn Cohen, Sr. Program, Advisor
Audrey Bennett, Project Manager

3. Centre for Education & Training Test Development
CLBA Placement
CLBLA Literacy
CLBPT Streamlined
CLBA –I High stakes
CLB-OSA (English) Online Self- NCLC-AEL (French) Assessment
ELTPA (6-10) Placement
Carolyn
NCLC-AEL (French) = Niveaux de Compétences Linguistiques Canadiens (NCLC) – Autoévaluation en ligne 3

4. ELTPA (6-10) Assessment
referenced to CLB 2000
identifies proficiencies from Bm 6- through Bm 10+
for assessment of clients with prior CLB scored at Bm 6 or higher
designed for placement in ELT programs
workforce-oriented tasks and themes
four language skills and four scores
time-efficient, easy to use and reliable
administered and scored by a trained assessor
Funding: ELT Initiative and National Settlement and Resettlement
Carolyn 4

5. Key Strategies consistency in application of development methodology
consideration for stakeholder issues
ongoing consultation with experts in the field
rigorous validation process
Carolyn:
formulating development methodology and meticulous application
stakeholders: the ELT initiatives and its goals
the ELT providers and their needs
language assessors and administration constraints
consultants: test development
CLB experts
university-based ESL experts
workforce training experts
consistent application of validation methodology

6. Successes
statistical analysis of pilot data confirms the validity of the speaking, listening, reading and writing sub-tests, and reveals reliability results commensurate with or exceeding standards for low-stakes tools
positive feedback from candidates, assessors, and ELT instructors and administrators
Carolyn

7. Recommended Uses for ELTPA Instruments
determine client proficiency for placement in ELT programs
provide instructors and service providers with accurate information on the clients’ proficiency
facilitate program planning and development
provide candidates with a better understanding of their higher-level workforce-context language skills
Audrey
Ontario region has provided ELT service Providers and Assessment Centres with a protocol for use of the ELTPA to place candidates in higher level language training programs.

8. ELTPA (6-10) Speaking and Listening
15 min; interview
Task 1 and 2:
introduce oneself and describe/narrate an experience
Task 3:
conduct an analysis
Task 4:
deliver a presentation
Task 5:
participate in a discussion
Listening
20 min; assessor facilitated
Task 1:
follow a phone message
Task 2:
listen to a semi-formal presentation
Task 3:
listen to an informal conversation
Task 4:
follow a formal discussion
Audrey

9. ELTPA (6-10) Reading and Writing
60 min; multiple choice
Task 1:
read an instructional
Task 2:
read a company policy excerpt
Task 3:
read and compare ideas in two opinion passages
Writing
45 minutes
Task 1:
write a memo/report
Task 2:
write a formal letter
Audrey
Reading
Task 1 – message to someone replacing the writer while on vacation
Task 2 – formal text outlining a company policy – contains acronyms referenced to the text
Task 3 – editorial and response
Writing
Task 1 – regarding a workplace/professional challenge
Task 2 – regarding a personal request and offering congratulation/support

10. 2008-9 Implementation Project Outputs:
Trained 45 new ELTPA assessors and 2 trainers (BC and Manitoba). This is in addition to Ontario (trained under separate agreement).
Audrey
To date there are nearly 100 trained ELTPA Assessors across Canada – federally funded (CIC)

11. Project Outputs – cont’d:
Provided Information Sessions to ELT providers, CIC, community partners - Questions and discussion points from these sessions have led to development of a series of FAQs that will be reviewed by a Discussion Forum as part of the current agreement
Audrey
These sessions were well attended in most jurisdictions and attendees were invited to look at the assessment (including a video of the Speaking assessment), ask questions related to the assessment, their programs, and how ELT programs meet the needs of the community. Programs were given the opportunity to network with each other, the assessment centres, trainers and CIC and community representatives.

12. Project Outputs – cont’d Research Study (2009) Purpose:
Comparison of standardized CLB Assessment results with ELTPA
To address fact that some ELT candidates lack prior CLB-referenced scores (lengthens qualifying process)
To address ELT program eligibility criteria for those lower that CLB 7
Carolyn
Can the ELTPA Speaking Assessment be used as a screener to reduce the number of clients who have to take two complete language assessments?
Can we find a way to predict, based on ELTPA Speaking, whether a client is more likely to achieve a meaningful reading, Writing and Listening outcome on CLBA/PT or ELTPA?

13. Project Outputs cont’d: Comparison Study
To compare ELTPA results with those of CLBA and CLBPT where skills have been evaluated through prior CLB-referenced assessments to be BM 6 or higher
- 152 case samples assessed on ELTPA and CLBA/PT
- range of 29 occupations
- trade certificate to doctoral degree
Carolyn

14. Comparison Study (cont’d)
ELTPA results are consistent with achievement of CLBA and CLBPT Benchmarks Achievement of Benchmarks 9 and 10 cannot be measured using CLBA/PT

15. Potential Screening to ELTPA
Can the ELTPA Speaking be used as a screening tool to reduce the number of clients who have to take two complete assessments?
Could it be a predictor of outcomes in other skills?
 Can we identify Benchmarks 5 and 6 for some ELTPA results?
Carolyn

16. 92% achieved Bm 7 or more in Listening
Potential Screening to ELTPA (Continued)
Sample of 241 cases
92% achieved Bm 7 or more in Listening
96% achieved Bm 7 or more in Reading
Writing less promising – 54%

17. Recommendations
Possible use of ELTPA Speaking as a screener where 2 or more Benchmarks at 7 are sufficient
Expand ELTPA Speaking component to identify Benchmarks 5 and 6 mean fewer candidates require two assessments
Further validation and consultation with Service Providers
Carolyn

18. National ELTPA Support Services 2009-2011
Create a National Discussion Forum to address regional needs, issues, challenges and solutions for use of ELTPA for placement of learners in ELT programs
Train up to 15 new ELTPA assessors per year in the administration of Enhanced Language Training Placement Assessment (ELTPA) tool to newly identified assessors
Offer 1-day recalibration for current ELTPA assessors to maintain consistent scoring practices and CLB results determination
Conduct an inter-rater reliability study on inter-scorer accuracy in administration and scoring of ELTPA to ensure consistency in scoring
Consultations to examine when it is best to use the ELTPA in comparison to other assessments
Audrey
Discussion Forum – to start will have 11 members representing CIC, SPOs, assessment and training
Training – first training under this agreement will be in October (22-23)
Recalibration – anticipated start of this component – November
Inter-rater reliability study – next fiscal year
Consultations – to extend from the National Discussion Forum to promote uniformity of use and access for individuals seeking higher level language training