1. Grading Principles and Practices James Wollack Department of Educational Psychology Office of Testing & Evaluation Services UW-Madison Teaching Academy September 28, 2012

2. Walvoord & Anderson (1995) Which assessment method… has nearly universal faculty participation enjoys superb student participation is never accused of violating academic freedom provides detailed diagnostic assessment of student learning is tightly linked to teaching has a tight feedback loop into classroom learning and into teacher planning is cheap to implement? Grading, when well done

3. Value of Grading to Instructors To what extent are the students achieving the stated course goals? Are students understanding the concepts? How should class time be allocated for the current topic? Can this topic be introduced more effectively? What parts of this course are students finding most valuable or most difficult? How should this course be altered next time it is taught?

4. Value of Grading to Students Do I know what my instructor thinks is most important? Am I mastering the course content? How can I improve the way I study in this course? What grade am I earning in this course?

5. Value of Grading Assessment and evaluation (i.e., grading) are essential components of effective instruction Research suggests that faculty spend 30% of their planning/instructional time on assessment activities Clearly, we are assessment driven and recognize the value of proper evaluation Why, then, do we not evaluate our own grading practices?

6. Evaluating One’s Grading Should assessments be learning experiences for students? What evidence of student learning should figure into grades? How should these data be combined to form final grades? Ought grades reflect developmental nature of learning? How can my grading criteria remain reasonable and comparable when assessments keep changing?

7. Components of Meaningful Grades Although grades serve multiple purposes, their primary purpose should be to reflect the extent to which a student has mastered the essential learning outcomes. Grades should be reliable Students’ demonstrated mastery should depend little on assessment modality or specific tasks Students’ work should be scored similarly (using same criteria) regardless of who is evaluating or when it is evaluated Grades should be valid Based on assessments closely linked to course objectives and desired learning outcomes Speak to students’ mastery of ELOs

8. #1 Grading Rule Focus on Fairness and Consistency Fairness = points/grades accurately reflect the weighting of course objectives. Consistency = similar scoring/grading for comparable work/depth of understanding.

9. Making Grades Fair and Consistent Consider what you want your students to learn Select assignments/assessments that measure what you value most Course Assessment Map: Description of how various learning goals will be assessed

10. Test Blueprints Develop test blueprints for exams Identifies the objectives/skills to be measured Objectives should be specific, but general enough to allow for multiple questions in that category. Blueprint should specify the relative weight of each category

11. Sample Biology Blueprint General Biology45% Cellular and Molecular Biology Diversity of Life Forms Health Microbiology35% Microorganisms Infectious Diseases & Prevention Microbial Ecology Medical Microbiology Immunity Human Anatomy and Physiology20% Structure Systems

12. Sample U.S. History Blueprint Domestic Affairs25% American political System Major Social Problems Global Affairs20% Civil Rights/Human Rights20% Economics25% Economic Transformation of US Government involvement in economy Culture10%

13. More on Blueprints Develop blueprints before making tests Can build into blueprint a measure of cognitive load Bloom’s taxonomy Overlap with textbook/lecture

14. Grading Subjective Assessments Embrace the grading challenge There is no one right way to score artifacts of student learning. Grading is socially constructed and context-dependent Expert graders will likely differ—sometimes considerably—in their evaluations. Overall harshness Importance of material Magnitude of student errors

15. Develop a Scoring Rubric A scoring rubric identifies the basis for awarding and subtracting points at each phase of each item. Nature of scoring rubric will change depending on whether item is computational or a narrative response.

16. Class Presentation Scoring Rubric (Sum-to-total) Knowledge/Understanding30% Thinking/Inquiry30% Communication20% Use of Visual Aids20% 20-point assignment For each domain, construct a scale, describing characteristics of each scale point (or range of points) 6 points 4 points

17. Needs Work (0-2)Competent (3-4)Excellent (5-6) Knowledge/ Understanding 30% The presentation uses little relevant or accurate information, not even that which was presented in class or in the assigned texts. Little or no research is apparent The presentation uses knowledge which is generally accurate with only minor inaccuracies, and which is generally relevant to the student’s thesis. Research is adequate but does not go much beyond what was presented in class or the assigned texts. The presentation demonstrates a depth of historical understanding by using relevant and accurate detail to support the student’s thesis. Research is through and goes beyond what was presented in class or in the assigned texts. Class Presentation Scoring Rubric (Sum-to-total)

18. Needs Work (0-2)Competent (3-4)Excellent (5-6) Thinking/ Inquiry 30% The presentation shows no analytical structure and no central thesis. The presentation shows an analytical structure and a central thesis, but the analysis is not always fully developed and/or linked to the thesis. The presentation is centered around a thesis which shows a highly developed awareness of historiographic or social issues and a high level of conceptual ability. Class Presentation Scoring Rubric (Sum-to-total)

19. Needs Work (0-1)Competent (2-3)Excellent (4) Communicat’n 20% The presentation fails to capture the interest of the audience and/or is confusing in what is communicated. Presentation techniques used are effective in conveying main ideas, but a bit unimaginative. Some questions from the audience remain unanswered. The presentation is imaginative and effective in conveying ideas to the audience. The presenter responds effectively to audience reactions and questions. Class Presentation Scoring Rubric (Sum-to-total)

20. Needs Work (0-1)Competent (2-3)Excellent (4) Use of Visual Aids 20% The presentation includes no visual aids or visual aids that are inappropriate and/or too small or messy to be understood. The presenter makes no mention of visual aids in the presentation. The presentation includes appropriate visual aids, but they are too few in number and/or in a format that makes them difficult to use or understand. The presentation includes appropriate and easily understood visual aids which the presenter refers to and explains at appropriate moments in the presentation Class Presentation Scoring Rubric (Sum-to-total)

21. Points 14-15 Describes work concisely Relates message to artist’s choice and use of various devices Develops how message affects beholder Considers audience in writing Clearly organized and presented Well-imagined Legible No problems with mechanics, grammar, spelling, or punctuation 11-13 Good description Relates message to artist’s choices and use of various devices Some consideration of effect on beholder Perhaps could be better organized or presented Adequately imagined Legible Few problems with mechanics, grammar, spelling, or punctuation Art History News Report (Holistic)

22. Points 8-10 Adequate description Less thorough analysis of how artist conveys message and devices Audience not necessarily kept in mind Needs significant improvement in organization or presentation Needs better imagination Problems with legibility, mechanics 6-7 Lacking substantially in either description or analysis Problems with audience, organization, presentation, or mechanics interfere with understanding 0-5 Substandard on more than two of these: description, analysis of choices and devices, effects on beholder Major problems with audience, organization, presentation, or mechanics Art History News Report (Holistic)

23. Students are given scores for 12 students on a Physics quiz and asked to compute the mean, median, and standard deviation of quiz scores, showing all work. Mean Correct formula (1 point) correct (1 point) Formula applied correctly (2 points) Correct answer* (1 point) Median Apparent understanding that median refers to middle score (1 point) Scores ranked correctly (2 points) Correct answer (2 points) Standard Deviation Correct formula (1 point) or correct (1 point) Formula applied correctly (2 points). Correct except no square root (1 point) Correct answer* (1 point) Given (possibly incorrect) numbers from previous steps Confusing mean for median (& vice versa) (−3 points) Nonsensical answer without annotation (−2 points) Statistics Problem Scoring Rubric

24. Strategies for Maintaining Consistency Grading should be done anonymously Grade all students’ responses one question at a time. Assign each rater a unique set of problems Multiple raters per item Maintain an error log and corresponding deductions.

25. Assigning Final Grades To curve or not to curve? Should only be considered for large classes (N ≥ 100) Not appropriate if course involves scored group work Runs counter to philosophy that teaching affects learning. Easiest if all assignments are scored numerically Hard to average (or weight) letter grades Assignments’ points may not reflect their relative weighting Problem for norm-referenced grading schemes

26. Assigning Final Grades Test A (15 points) Test B (30 points) Comp. (45 points) Rank A1513281 B1314272 C1115263 D916254 E717245 F518236 G319227 H120218 Mean816.5 St. Dev.4.92.45

27. Assigning Final Grades To curve or not to curve? Should only be considered for large classes (N ≥ 100) Not appropriate if course involves scored group work Runs counter to philosophy that teaching affects learning. Easiest if all assignments are scored numerically Hard to average (or weight) letter grades Assignments’ points may not reflect their relative weighting Problem for norm-referenced grading schemes Unless assessment standard deviations are comparable, weighting will not be as intended

28. Assigning Final Grades

29. Test A (15 ) Test B (30) Z(A)Z(B)15Z(A)30Z(B)Comp.Rank A15131.43−1.4321.43−42.87−21.438 B13141.02−1.0215.31−30.62−15.317 C11150.61−0.619.19−18.37−9.196 D9160.20−0.203.06−6.12−3.065 E717−0.200.20−3.066.123.064 F518−0.610.61−9.1918.379.193 G319−1.021.02−15.3130.6215.312 H120−1.431.43−21.4342.8721.431 Mean816.500000 St. Dev.4.92.45111530

30. Assigning Final Grades What should contribute? Should homework, participation, attendance, etc. count? Perhaps we could adopt a grading strategy that was reflective of this relationship Student Engagement Students’ Mastery of ELOs

31. Noncompensatory Grading Non-assessment activities likely to improve student learning can serve as gate-keepers for grades Composite score based entirely on assessments Can have separate engagement targets that must be met in order to qualify for grades A 90 – 100% + passing score for at least 85% HW AB 85 – 89% + passing score for at least 80% HW B 80 – 84 % + passing score for at least 70% HW

32. What else should contribute? Learning is often cumulative and unscheduled Adopting a developmental grading scheme Exams count more later in semester than earlier Final counts more and gives students chance to demonstrate mastery of concepts assessed earlier in semester Dropping lowest midterm score?

33. How Many Points is an A? No ubiquitous definition Clearly some conventions Like it or not, a B is an average grade Dramatic variations from conventions communicates misinformation to stakeholders (students, industry, grad schools, etc.) Test development is very difficult What if I write an exam that’s too hard? Consider adopting a class-referenced approach

34. Class-Referenced Grading Write exams so that the very best students should be able to attain a perfect or nearly perfect score. Rescale exams (percentage correct scores) so that the top score is treated as the maximum achievable score Highest earned score will always be 100% Students not punished because of an overly (and inadvertently) hard exam Standards in syllabus are conservative Different from norm-referenced in that percentage of students at each grade is not predetermined.

35. Grading Challenges Grading is really tricky Appropriate assignments that cover domain and allow students to demonstrate their knowledge Consistent, fair, and appropriate scores for each Determining what counts and in what combination Obscure task of assigning composite score ranges to ill-defined grade labels. Doing it right, is paramount!

36. Thank You, Teaching Academy Questions? James Wollack Department of Educational Psychology Office of Testing & Evaluation Services jwollack@wisc.edu