Education 325: Assessment for Classroom Teaching G. Galy, PhD Week 5

Evaluation: A judgment, the determination of worth Common evaluation tools: Exams, Tests, Quizzes Let’s look at Evaluation: Teacher- Made Tests

Teacher-made Tests

How do we make our tests valid and reliable? Valid and Reliable

Validity and Reliability

“Measuring what we set out to test.” “So that we can make sound judgments.” “Validity is never 100% achievable.” Validity:

“Getting consistent results” Factors that can impact process validity include: Directions Vocabulary Time Difficulty Clues Scoring (unclear criteria) Arrangement Emotion/Feelings Cheating Process Validity

“Alignment” Factors that can impact content validity: Irregular ‘spot check’ Gaps (untested) Alignment (teach it = test it) Appropriate value and weighting Content Validity

“Future Performance” Minimal Competencies: Pass? Fail? (Driver’s License) Pass: Mastery? 80% Pattern of achievement and performance? How confidently can you predict success for this student in the next grade? Next course? Predictive Validity

“Ability to accurately measure without influence” Test anxiety Competitive stress Too much teaching to the test, rather than learning Consequential Validity

“attitudes, values or beliefs” Creativity Team Work Problem-Solving Safety-mindedness Higher- order thinking Does your test REALLY measure this? How do you measure such things? How do you accurately measure a mental process? Construct Validity

“produces consistent results…even if given to the same group, a second time” Common characteristics: Re-test reliability Inter rater reliability Reliability

Write Valid and Reliable Items Arrange Items in clear, sequential order Writing the Test

1.Bonus points can distort achievement.T F 1.Achievement is distorted by ____________. a) Bonus Points b) Avion Miles A B Alternate Response

Pro:Con: Recall level Not easy to make (No ambiguity) Quick to complete Unimportant details tend to be tested Promotes ‘guessing’ Not good with complex material Alternate Response

#1. Word statement clearly : Poor Example: “Achievement may be distorted by bonus marks.” Better Example: “Achievement is distorted by bonus marks.” Note: Broad generalizations are rarely wrong. CUE: ‘usually’, ‘generally’ ‘often’ ‘sometimes’ cue TRUE CUE: Absolutes ‘all’ ‘never’ ‘none’ ‘only’ ‘always’ cue FALSE Alternate Response: Guidelines

2. Avoid testing trivial or pointless information : Each item should test important learning outcomes. Ask: Is the answer obvious? If so, then it will undermine the content validity of the test. Alternate Response: Guidelines

3. Avoid the use of negative statements and double negatives. “Neither the bonding nor the adhesive surface should be handled with an ungloved hand. Bold, underline or italicize the word ‘not’. Double negatives (not, no, none, never, neither, nor) are confusing. Alternate Response: Guidelines

4. Keep statements short. Avoid marathon statements. “Large bubbles in local anaesthetic carpules, with or without plungers that extend beyond the end of the carpule, may be formed from dissolved gases in the solution. These bubbles are harmless. TF Alternate Response: Guidelines

5. Avoid double barreled statements. “Parks and protected areas should be maintained for the use and benefit of all British Columbians.” Alternate Response: Guidelines -

6. Avoid statements that contain lists. If you do, you won’t be able to determine if they knew the answer or took a guess. “ Stockton, Jordan and Robinson are in the NBA Hall of Fame.” Alternate Response: Guidelines

7. Avoid using trickery. Don’t insert insignificant detail to deceive the student. “The area of a rectangle 4 m x 3 m equals 12 square centimeters.”TF Alternate Response: Guidelines

8. Have an equal number of True and False items. Why? Students have a tendency to guess True more often than False. Skewing the number will unduly favour or penalize a number of students. Alternate Response: Guidelines

9. Keep True and False statements items similar in length. The longer statement is most often True. Largely because it contains additional detail and is more accurate. If possible, keep all items short and the same length. Alternate Response: Guidelines

1. Blank is located at the end of the sentence. 2. Only important words are omitted. 3. No more than two blanks per statement. 4. Each blank asks for a single idea. “Fill in the Blank” or Short Answer Guidelines

5. Blanks must be the same length. 6. Only an ‘informed’ person should be able to complete the answer. “Fill in the Blank” Guidelines