1. Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College

2. What is the first thing you think of when you hear the word “evaluation”? “assessment”? “test”?

8. Example of a Speaking Test One item Single-word repetition Testing a discrete point Stimulus is the prompt, “Say now Shibboleth.” Scoring criteria – right / wrong Consequences – potentially dire!!

9. Purpose of Assessment Why?

10. Formative vs. Summative How will we use the test?

11. Reasons for Testing Placement Tests Diagnostic Tests Progress or Achievement Tests Proficiency Tests Portfolio Assessments

12. What makes a good test? Validity Reliability

13. Test Design A verb tense A word Discrete Use a variety of language and skills for a task Integrative

14. Test Design

15. Question Types - Indirect Multiple Choice

16. Question Types - Indirect Gap-fill and Cloze www.lextutor.ca

17. Question Types - Indirect Transformation Sentence re-ordering Edit Match

18. Question Types - Direct May use items that look like indirect items Others are specific tasks to the skills Replicate real-life interaction – Mirror an authentic purpose

19. Question Types - Direct Remember – Items should look like the kind of tasks students have been practicing in their lessons. – Create a “level playing field”. – Marking direct test items will be more difficult than indirect items.

20. Constructed-response assessments Fill-in assessment

21. Performance-based assessments Simulates real-life language use Offers open-ended tasks Allows for creative and divergent responses Often employs strategies Requires students to synthesize information Promotes recursive learning Evaluates performance based on well-defined critera

22. Testing Reading and Listening Choose the best summary Put pictures in order Complete a phone message form (listening) Fill out a summary form (reading)

23. Testing Reading and Listening Global understanding Details

24. Testing Reading Many skills at different levels of reading from word recognition, reading speed, skimming techniques, etc. Primary evaluation is looking at reading comprehension at every level.

25. Testing Listening Use a recorded audio track to ensure test relability Ensure the recording is clearly and easily audible

26. Testing Writing Produce written texts – Leaflets based on information in a text – Compositions (narrative or discursive essays) – Transactional letters (reply to advertisement)

27. Testing Speaking Seen as the most challenging of exams to prepare, administer and score.

28. Testing Speaking Interview students Pair tasks – Compare two pictures – Solve a problem – Discuss a topic – Role-play a situation

29. Grammar and Vocabulary Knowledge versus Ability Isolated (discrete points) or integrated

30. Marking Backwash – Positive – Negative

31. Marking Writing

32. Are you ever tempted to do this?

33. Marking Writing Holistic Scoring – Rubrics assigning a single score Analytic Scoring – 5 aspects of writing weighted and combined (Jacob) – 7 scales (TEEP by Weir) – 3 scales (Michigan Writing Assessment)

34. Other Applications of Rubrics and Scales Speaking Performance of Tasks Portfolio evaluation

35. 1.List the things you want to test. 2.Decide how important each item is. Give them more time or space on the test Weight the marks to reflect the importance 3.Consider the test situation. 4.Write the test. 5.Consult and share. 6.Test the test. (the trial) 7.Reflect and refine. Steps in Test Design

36. Another list (Brown and Hudson) Require students to perform, create, produce, or do something Use real-world contexts or simulations Allow students to be assessed on what they normally do in class every day Use tasks that present meaningful instructional activities Focus on processes as well as products Tap into higher-level thinking and problem-solving skills Provide information about both the strength and weaknesses of students Are multi-culturally sensitive when properly administered Ensure that people, not machines, do the scoring using human judgment Encourage open disclosure of standards and rating criteria Call upon teachers to perform new instructional and assessment roles