1. Education 325: Assessment for Classroom Teaching G. Galy, PhD Week 5

2. Evaluation: A judgment, the determination of worth Common evaluation tools: Exams, Tests, Quizzes Let’s look at Evaluation: Teacher- Made Tests

3. Teacher-made Tests

4. How do we make our tests valid and reliable? Valid and Reliable

5. Validity and Reliability

6. “Measuring what we set out to test.” “So that we can make sound judgments.” “Validity is never 100% achievable.” Validity:

7. “Getting consistent results” Factors that can impact process validity include: Directions Vocabulary Time Difficulty Clues Scoring (unclear criteria) Arrangement Emotion/Feelings Cheating Process Validity

8. “Alignment” Factors that can impact content validity: Irregular ‘spot check’ Gaps (untested) Alignment (teach it = test it) Appropriate value and weighting Content Validity

9. “Future Performance” Minimal Competencies: Pass? Fail? (Driver’s License) Pass: Mastery? 80% Pattern of achievement and performance? How confidently can you predict success for this student in the next grade? Next course? Predictive Validity

10. “Ability to accurately measure without influence” Test anxiety Competitive stress Too much teaching to the test, rather than learning Consequential Validity

11. “attitudes, values or beliefs” Creativity Team Work Problem-Solving Safety-mindedness Higher- order thinking Does your test REALLY measure this? How do you measure such things? How do you accurately measure a mental process? Construct Validity

12. “produces consistent results…even if given to the same group, a second time” Common characteristics: Re-test reliability Inter rater reliability Reliability

13. Write Valid and Reliable Items Arrange Items in clear, sequential order Writing the Test

14. 1.Bonus points can distort achievement.T F 1.Achievement is distorted by ____________. a) Bonus Points b) Avion Miles A B Alternate Response

15. Pro:Con: Recall level Not easy to make (No ambiguity) Quick to complete Unimportant details tend to be tested Promotes ‘guessing’ Not good with complex material Alternate Response

16. #1. Word statement clearly : Poor Example: “Achievement may be distorted by bonus marks.” Better Example: “Achievement is distorted by bonus marks.” Note: Broad generalizations are rarely wrong. CUE: ‘usually’, ‘generally’ ‘often’ ‘sometimes’ cue TRUE CUE: Absolutes ‘all’ ‘never’ ‘none’ ‘only’ ‘always’ cue FALSE Alternate Response: Guidelines

17. 2. Avoid testing trivial or pointless information : Each item should test important learning outcomes. Ask: Is the answer obvious? If so, then it will undermine the content validity of the test. Alternate Response: Guidelines

18. 3. Avoid the use of negative statements and double negatives. “Neither the bonding nor the adhesive surface should be handled with an ungloved hand. Bold, underline or italicize the word ‘not’. Double negatives (not, no, none, never, neither, nor) are confusing. Alternate Response: Guidelines

19. 4. Keep statements short. Avoid marathon statements. “Large bubbles in local anaesthetic carpules, with or without plungers that extend beyond the end of the carpule, may be formed from dissolved gases in the solution. These bubbles are harmless. TF Alternate Response: Guidelines

20. 5. Avoid double barreled statements. “Parks and protected areas should be maintained for the use and benefit of all British Columbians.” Alternate Response: Guidelines -

21. 6. Avoid statements that contain lists. If you do, you won’t be able to determine if they knew the answer or took a guess. “ Stockton, Jordan and Robinson are in the NBA Hall of Fame.” Alternate Response: Guidelines

22. 7. Avoid using trickery. Don’t insert insignificant detail to deceive the student. “The area of a rectangle 4 m x 3 m equals 12 square centimeters.”TF Alternate Response: Guidelines

23. 8. Have an equal number of True and False items. Why? Students have a tendency to guess True more often than False. Skewing the number will unduly favour or penalize a number of students. Alternate Response: Guidelines

24. 9. Keep True and False statements items similar in length. The longer statement is most often True. Largely because it contains additional detail and is more accurate. If possible, keep all items short and the same length. Alternate Response: Guidelines

25. 1. Blank is located at the end of the sentence. 2. Only important words are omitted. 3. No more than two blanks per statement. 4. Each blank asks for a single idea. “Fill in the Blank” or Short Answer Guidelines

26. 5. Blanks must be the same length. 6. Only an ‘informed’ person should be able to complete the answer. “Fill in the Blank” Guidelines