Presentation on theme: "Weaving Pathways: Interculturalism and Language"— Presentation transcript:
1 Weaving Pathways: Interculturalism and Language TESL Canada Conference 2009Weaving Pathways: Interculturalism and Language
2 ELTPA Assessment Tool for Placement in ELT Programs Audrey: Welcome introductions, and agendaMaterials needed: Laptop, LCD, SpeakersTest Kit + video of Speaking Assessment (Zeeshan or Haider)Carolyn Cohen has worked in the field of ESL for 20 years. She began as an ESL Teacher and Teacher Trainer. She worked as a TSE evaluator and LINC Assessor and became fully engaged in LINC and ESL as a programs and services manager. She managed development of the CLBA/LA, the ELTPA, the CLBA International and the original form of the CLBPT and CLB Online self-assessment in partnership with the CCLB. Carolyn is Senior Project Advisor for the Centre for Education & Training where her work areas include the National ELTPA Support Services, CLB On-Line Self Assessment, and Language Assessment Support Services. In November 2008, Carolyn was recognized by TESL Ontario as one who has demonstrated significant ongoing commitment to our profession.Audrey Bennett is the Project Manager for the National ELTPA Support Services. She has worked at the Centre for Education & Training since 1992 in assessment, coordination of LINC Assessment Centres, training assessors in the use of CLB-based assessment tools, including CLBA, CLABLA, CLBA-International and ELTPA.presented by:Centre for Education and TrainingCarolyn Cohen, Sr. Program, AdvisorAudrey Bennett, Project Manager
3 Centre for Education & Training Test Development CLBA PlacementCLBLA LiteracyCLBPT StreamlinedCLBA –I High stakesCLB-OSA (English) Online Self- NCLC-AEL (French) AssessmentELTPA (6-10) PlacementCarolynNCLC-AEL (French) = Niveaux de Compétences Linguistiques Canadiens (NCLC) – Autoévaluation en ligne3
4 ELTPA (6-10) Assessmentreferenced to CLB 2000identifies proficiencies from Bm 6- through Bm 10+for assessment of clients with prior CLB scored at Bm 6 or higherdesigned for placement in ELT programsworkforce-oriented tasks and themesfour language skills and four scorestime-efficient, easy to use and reliableadministered and scored by a trained assessorFunding: ELT Initiative and National Settlement and ResettlementCarolyn4
5 Key Strategies consistency in application of development methodology consideration for stakeholder issuesongoing consultation with experts in the fieldrigorous validation processCarolyn:formulating development methodology and meticulous applicationstakeholders: the ELT initiatives and its goalsthe ELT providers and their needslanguage assessors and administration constraintsconsultants: test developmentCLB expertsuniversity-based ESL expertsworkforce training expertsconsistent application of validation methodology
6 Successesstatistical analysis of pilot data confirms the validity of the speaking, listening, reading and writing sub-tests, and reveals reliability results commensurate with or exceeding standards for low-stakes toolspositive feedback from candidates, assessors, and ELT instructors and administratorsCarolyn
7 Recommended Uses for ELTPA Instruments determine client proficiency for placement in ELT programsprovide instructors and service providers with accurate information on the clients’ proficiencyfacilitate program planning and developmentprovide candidates with a better understanding of their higher-level workforce-context language skillsAudreyOntario region has provided ELT service Providers and Assessment Centres with a protocol for use of the ELTPA to place candidates in higher level language training programs.
8 ELTPA (6-10) Speaking and Listening 15 min; interviewTask 1 and 2:introduce oneself and describe/narrate an experienceTask 3:conduct an analysisTask 4:deliver a presentationTask 5:participate in a discussionListening20 min; assessor facilitatedTask 1:follow a phone messageTask 2:listen to a semi-formal presentationTask 3:listen to an informal conversationTask 4:follow a formal discussionAudrey
9 ELTPA (6-10) Reading and Writing 60 min; multiple choiceTask 1:read an instructionalTask 2:read a company policy excerptTask 3:read and compare ideas in two opinion passagesWriting45 minutesTask 1:write a memo/reportTask 2:write a formal letterAudreyReadingTask 1 – message to someone replacing the writer while on vacationTask 2 – formal text outlining a company policy – contains acronyms referenced to the textTask 3 – editorial and responseWritingTask 1 – regarding a workplace/professional challengeTask 2 – regarding a personal request and offering congratulation/support
10 2008-9 Implementation Project Outputs: Trained 45 new ELTPA assessors and 2 trainers (BC and Manitoba). This is in addition to Ontario (trained under separate agreement).AudreyTo date there are nearly 100 trained ELTPA Assessors across Canada – federally funded (CIC)
11 Project Outputs – cont’d: Provided Information Sessions to ELT providers, CIC, community partners - Questions and discussion points from these sessions have led to development of a series of FAQs that will be reviewed by a Discussion Forum as part of the current agreementAudreyThese sessions were well attended in most jurisdictions and attendees were invited to look at the assessment (including a video of the Speaking assessment), ask questions related to the assessment, their programs, and how ELT programs meet the needs of the community. Programs were given the opportunity to network with each other, the assessment centres, trainers and CIC and community representatives.
12 Project Outputs – cont’d Research Study (2009) Purpose: Comparison of standardized CLB Assessment results with ELTPATo address fact that some ELT candidates lack prior CLB-referenced scores (lengthens qualifying process)To address ELT program eligibility criteria for those lower that CLB 7CarolynCan the ELTPA Speaking Assessment be used as a screener to reduce the number of clients who have to take two complete language assessments?Can we find a way to predict, based on ELTPA Speaking, whether a client is more likely to achieve a meaningful reading, Writing and Listening outcome on CLBA/PT or ELTPA?
13 Project Outputs cont’d: Comparison Study To compare ELTPA results with those of CLBA and CLBPT where skills have been evaluated through prior CLB-referenced assessments to be BM 6 or higher- 152 case samples assessed on ELTPA and CLBA/PT- range of 29 occupations- trade certificate to doctoral degreeCarolyn
14 Comparison Study (cont’d) ELTPA results are consistent with achievement of CLBA and CLBPT Benchmarks Achievement of Benchmarks 9 and 10 cannot be measured using CLBA/PT
15 Potential Screening to ELTPA Can the ELTPA Speaking be used as a screening tool to reduce the number of clients who have to take two complete assessments?Could it be a predictor of outcomes in other skills? Can we identify Benchmarks 5 and 6 for some ELTPA results?Carolyn
16 92% achieved Bm 7 or more in Listening Potential Screening to ELTPA (Continued)Sample of 241 cases92% achieved Bm 7 or more in Listening96% achieved Bm 7 or more in ReadingWriting less promising – 54%
17 RecommendationsPossible use of ELTPA Speaking as a screener where 2 or more Benchmarks at 7 are sufficientExpand ELTPA Speaking component to identify Benchmarks 5 and 6 mean fewer candidates require two assessmentsFurther validation and consultation with Service ProvidersCarolyn
18 National ELTPA Support Services 2009-2011 Create a National Discussion Forum to address regional needs, issues, challenges and solutions for use of ELTPA for placement of learners in ELT programsTrain up to 15 new ELTPA assessors per year in the administration of Enhanced Language Training Placement Assessment (ELTPA) tool to newly identified assessorsOffer 1-day recalibration for current ELTPA assessors to maintain consistent scoring practices and CLB results determinationConduct an inter-rater reliability study on inter-scorer accuracy in administration and scoring of ELTPA to ensure consistency in scoringConsultations to examine when it is best to use the ELTPA in comparison to other assessmentsAudreyDiscussion Forum – to start will have 11 members representing CIC, SPOs, assessment and trainingTraining – first training under this agreement will be in October (22-23)Recalibration – anticipated start of this component – NovemberInter-rater reliability study – next fiscal yearConsultations – to extend from the National Discussion Forum to promote uniformity of use and access for individuals seeking higher level language training