1. L2 Learner Self-Assessment (SA): Theory and Applications in College ESL Programs Brett Reynolds & Lara McInnis CALL Conference 2010

2. Agenda 1.Background & Overview 2.SA in College ESL/EAP Programs 3.SA in College Programs, Non-ESL 4.SA in ESL/EFL Internationally 5.College ESL Teacher Survey: Why is SA used or avoided? 6.Implementation of SA in ESL Programs, Courses and Lessons 7.Questions & Discussion

3. Background & Overview What Self-Assessment is/isn’t to us: a way of measuring one’s own competency or progress in a particular language introspection with an objective guideline (“can do” statements, performance indicators and exemplars) It isn’t an online quiz What led us to explore self-assessment: Interested in the role of SA in ESL college writing courses Possible development of online self-assessment tool for EAP placement (overseas)

4. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Adapted from Liang (2006)

5. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Promotes learner autonomy and self-sufficiency Adapted from Liang (2006)

6. 1. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Promotes learner autonomy and self-sufficiency Learners can notice gaps in their abilities and self-correct effectively Adapted from Liang (2006)

7. 1. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Promotes learner autonomy and self-sufficiency Learners can notice gaps in their abilities and self-correct effectively Encourages noticing, critical thinking and goal-setting Adapted from Liang (2006)

8. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Promotes learner autonomy and self-sufficiency Learners can notice gaps in their abilities and self-correct effectively Encourages noticing, critical thinking and goal-setting  motivation and active engagement in the language learning process and learner-specific goals Adapted from Liang (2006)

9. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Promotes learner autonomy and self-sufficiency Learners can notice gaps in their abilities and self-correct effectively Encourages noticing, critical thinking and goal-setting  motivation and active engagement in the language learning process and learner-specific goals  workload for instructors Adapted from Liang (2006)

10. Background & Overview Other advantages: Assess affective aspects Simplified test development and administration

11. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Difficult to define parameters Promotes learner autonomy and self-sufficiency Learners can notice gaps in their abilities and self-correct effectively Encourages noticing, critical thinking and goal-setting  motivation and active engagement in the language learning process and learner-specific goals  workload for instructors Adapted from Liang (2006)

12. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Difficult to define parameters Promotes learner autonomy and self-sufficiency Perceived uselessness by some learners Learners can notice gaps in their abilities and self-correct effectively Encourages noticing, critical thinking and goal-setting  motivation and active engagement in the language learning process and learner-specific goals  workload for instructors Adapted from Liang (2006)

13. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Difficult to define parameters Promotes learner autonomy and self-sufficiency Perceived uselessness by some learners Learners can notice gaps in their abilities and self-correct effectively Time consuming: time required for instructor to model and for in-class practice Encourages noticing, critical thinking and goal-setting  motivation and active engagement in the language learning process and learner-specific goals  workload for instructors Adapted from Liang (2006)

14. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Difficult to define parameters Promotes learner autonomy and self-sufficiency Perceived uselessness by some learners Learners can notice gaps in their abilities and self-correct effectively Time consuming: time required for instructor to model and for in-class practice Encourages noticing, critical thinking and goal-setting Formative Assessment: Learners may resist b/c of attitude or de-motivation  motivation and active engagement in the language learning process and learner-specific goals  workload for instructors Adapted from Liang (2006)

15. Background & Overview Advantages of SA Disadvantages of SA Directly involves learner in his/her language learning plan Difficult to define parameters Promotes learner autonomy and self-sufficiency Perceived uselessness by some learners Learners can notice gaps in their abilities and self-correct effectively Time consuming: time required for instructor to model and for in-class practice Encourages noticing, critical thinking and goal-setting Formative Assessment: Learners may resist b/c of attitude or de-motivation  motivation and active engagement in the language learning process and learner-specific goals Summative Assessment: Problems with validity, reliability, objectivity and capacity  workload for instructors Adapted from Liang (2006)

16. Background & Overview Used for Placement Certification Diagnosis Feedback Program Evaluation Learner Attitude Learner Preference (Liang, 2006)

17. Background & Overview Related theories: selective attention, noticing and awareness of language rules and form (see Schmidt, 1990) Schmidt (1990): “When I finally did notice the form, I began to use it.”

18. College ESL Teacher Survey: Why is SA used or avoided? Informal online survey sent to ESL full- and part-time ESL faculty at Humber 11 respondents, 7 questions (multiple choice + short- answer) Sample questions: Have you ever used learner self-assessment practices in your ESL classroom? Have you ever included a self-assessment component in your students' final grades? Would you be interested in learning more about self- assessment practices in the ESL college classroom?

19. College ESL use of SA Self-assessment is generally viewed favourably, but… There is very little use of self assessment for students.

20. Have you ever used learner self-assessment practices in your ESL classroom ?

21. Is Self-Assessment beneficial for teachers and learners?

22. Other findings 0% of respondents incorporate self-assessment grades into students’ final marks Two respondents used the writing course’s standardized rubric/criterion sheet as a guideline for students’ self- assessment A few teachers mentioned “checklists” as effective self- assessment tools in writing courses 82% of the respondents said they were interested in learning more about self-assessment in the L2 writing classroom

23. Sample of comments from ESL instructors “I find that [SA] results are not reliable.” “In general, I don't have a solid theoretical background in assessment, so I would value [access to] more research in this area.” “I could do [SA] for grammar and writing skills. I think I would have more issues with a reading one to gauge level.” “Self-assessment ultimately makes students more mindful about their work.” “It would be interesting to see how teachers' assessments change after they see student self-assessments”

24. Self-Assessment Tool for EAP Speaking/Listening Task (David Cooper) How Did I Do? Answer the following questions about your participation in today’s conversation assignment: Speaker Role 1. With this particular topic, I a) had nothing to say; did not know what to say b) did not need the whole time to say what I wanted to say c) had the right amount of time to say what I wanted to say d) needed more time to speak; could have spoken for longer

25. Self-Assessment Tool for EAP Speaking/Listening Task (David Cooper) 2. While I was speaking, I a) could not think of any words to express my ideas b) used words I usually use to express my ideas c) tried using some new vocabulary from the chapter to express my ideas d) had no problems using both familiar and new words to express my ideas

26. Sample checklist for summary task: advanced ESL writing course

27. Self Assessment in Non-ESL Programs Again, general agreement that self assessment is a Good Thing Again, very few people using it An informal survey of teachers in the school of Liberal Arts & Sciences turned up no users BUT…

28. Self Assessment in Non-ESL Programs Humber College Practical Nursing Program SA used in the practicum component of a first-year course SA used in clinical trials and for individualized learning plans Framework: Quality Assurance (QA) Reflective Practice (RP) and the College of Nurses of Ontario: Standards of Practice Learners participate in QA and RP by completing a self- assessment tool and initiation of a professional portfolio Final course grade is Satisfactory/Unsatisfactory Some students may be asked to revise if not enough information See http://www.cno.org/docs/qa/SAT_Overview.pdf See http://www.cno.org/qa/qa_sa.htm

29. Self-Assessment in ESL/EFL internationally Europe Self assessment is widely used Government support Often mandatory Good tools European Framework: (CEFRL) –Language portfolio »Language passport »Language biography »Language Dossier

30. Self-Assessment in Canada Canadian Language Benchmarks Pre-placement into LINC: How well do you use English? “Can Do” Checklist related to everyday tasks If you check nearly all items, you may be appropriate for that level “Are you interested in assessing yourself? ? If yes, click on Self Assessment. Please remember that this is a self assessment only. Based on 4-skill language functions and CLB performance indicators CELBAN *Not to be confused with the 2008-2009 Research and Development project “CLB Online Self-Assessment” for reading and listening http://clb-osa.ca/

31. 5. How can we implement Self Assessment into our ESL programs, classrooms and lessons? Can be used formatively for all levels, if modelled well, defined clearly, and practiced thoroughly Effective for task-based activities Checklist approaches work well Support from the top Make it mandatory (preferably by mutual agreement) but flexible Portfolios, reflective written pieces (for high levels) “Self-assessment scores in strong average correlations with external assessment on receptive skills but in relative low average correlations with productive skills” (Liang, 2006)

32. Online EAP-Levels Self-Assessment Offshore reality check for prospective Humber EAP students

33. 6. Questions & Discussion

34. References Council of Europe Secretariat. (2005, December). Surveying the use of the Common European Framework of Reference for Languages (CEFR) Draft synthesis of results. Retrieved May 27, 2010, from Council of Europe: www.coe.int/T/DG4/Portfolio/documents/CEFSurveyResults.doc EQUALS-ALTE. (2005). EQUALS-ALTE ePortfolio. Retrieved May 27, 2010, from EQUALS-ALTE ePortfolio: http://www.eelp.org/eportfolio/index.htmlhttp://www.eelp.org/eportfolio/index.html LeBlanc, R. & Painchaud, G. (1985). Self-Assessment as a Second Language Placement Instrument. TESOL Quarterly, Vol. 19, No. 4, pp. 673-687 Liang, J. (2006). Overview of Self-Assessment in the Second Language Writing Classroom. TESOL Conference. Tampa, Florida. Little, D. (2007, February 6-8). From the Common European Framework of Reference to the European Language Portfolio. Retrieved May 27, 2010, from Council of Europe: http://www.coe.int/t/dg4/linguistic/Source/SourceForum07/D- Little_handout_EN.pdf

35. References Oscarson, M. (1989). Self-assessment of language proficiency: Rationale and application. Language Testing, 6, 1-12. Schmidt, R. (1990). The role of consciousness in second language learning. Applied Linguistics, 11, 129-58. University of Ottawa. (2009). Self-Assessment Test on Writing Comprehension: French as a Second Language. Retrieved May 28, 2010, from University of Ottawa: http://www.socialsciences.uottawa.ca/psy/eng/prog2_adm-test.asp