Can a grade based approach improve our assessment practice? Andrew Barnes, Susie Crawford, Sarah Churchill, Pete Olusoga, James Rumbold and Louise Turner Vs

UG Sport and Exercise Science The context UG Sport and Exercise Science Modules of up to 200 students. Lectures/practicals/seminars One and two task modules. Large marking teams (3-4).

Assessment instructions (current practice) Assessment brief (mark scheme) Marking grid Assessment screencast In class tasks (e.g. example lab reports L4)

Current feedback Practice In text annotations/Audio file Highlighted marking grid Overall mark (%) based on weighted criteria. Text/audio - how to improve in subsequent work

The rationale for GBA Consistency: Marking teams of 3-4 of staff assessing work over multiple criteria (SSC and NSS feedback) Flawed assumptions: Using an interval scale to represent student achievement that relies on a qualitative judgement has been suggested to be flawed (Dalzeil 1998). Markers judgement: The use a 100 point scale offers too many possible ways of categorising achievement, for example can we adjudge a 65% as categorically different from a 64%? (Hornby 2003) Level specific criteria: Having different marking criteria between modules prevents students from understanding level specific criteria on which they are judged as they progress through their course (Sadler 2005). Compressed range of marks: Feedback from externals suggest we need to make more use of higher end >75%.

GBA approach for selected L4 assessments What we did GBA approach for selected L4 assessments Not used for exams/phase tests Similar instructions to students Generic grading descriptor (no marking grid) Grade score 0-18

Assessment feedback In text annotations/Audio file Highlighted grade descriptor Grade Score 0-18 (whole assessment) How to move up a grade boundary No criteria specific grades

The dreaded conversion table! Degree class Grade Score Numerical equivalent (%) Indicative mark range (%) First Perfect 1st 18 100 Exceptional 1st 17 96 99 - 93 High 1st 16 89 92 - 85 Mid 1st 15 81 84 - 78 Low 1st 14 74 77 - 70 Upper second High 2.1 13 68 69 - 67 Mid 2.1 12 65 66 - 64 Low 2.1 11 62 63 - 60 Lower second High 2.2 10 58 59 - 57 Mid 2.2 9 55 56 - 54 Low 2.2 8 52 53 - 50 Third High 3rd 7 48 49 - 47 Mid 3rd 6 45 46 - 44 Low 3rd 5 42 43 - 40 Fail Marginal fail 4 38 39 - 35 Mid fail 3 32 34 - 30 Low fail 1-2 29 - 1 Zero

A mixed methods approach from multiple perspectives Evaluation A mixed methods approach from multiple perspectives Student attainment Level 4 students - Online survey (assessment instructions/feedback) Level 5 and 6 - Focus group Staff - Focus group

Student Attainment Laboratory Report 15-16 16-17 GBA Mean 50% 48% 52%   15-16 16-17 GBA Mean 50% 48% 52% 58% StDev 17% 20% 10% 12% Min 16% 18% 13% Max 80% 81% 96% Presentation 61% 56% 57%  58%  9%  11%  27%  20% 22%  89% 72%   81%

L4 Student feedback (results based on 95 responses, 49% of the cohort) Assessment instructions Suggested improvements "Be clear on the difference between grades for marking." "Peer assessment seminar was really helpful in writing my work" "more detail on what markers are looking for in scientific writing such as a lab report" "All assessment information seemed detailed and very clear "Could talk about the university marking criteria more, and maybe be more clear on how our overall grade at the end of the year would be calculated"

L4 Student feedback (results based on 95 responses, 49% of the cohort) Assessment feedback Suggested improvements " Feedback was really detailed and explains why I got the grade I did " "A lot of the feedback only applied to that specific assessment so wasn't very helpful for improving in future assessments" "Be more clearer in weaknesses and strengths, try to provide information on how I can move up a grade boundary please" "Thought the feedback was generally very useful although each point could have been in more detail" "Rather than comments down the side of my work, a detailed break down of how to improve the area would be appreciated. "

L5 and 6 feedback "what is the difference between good, very good and excellent?" "Prefer marking grid as it highlights what is required in each section" "I would be less likely to use the generic grade descriptor than marking grid " "Some lecturers are good at saying how we can improve, while others just repeat annotations in text" " if there are 10-12 people marking, the criteria are still open to interpretation" "I like knowing the % for my coursework so I know exactly how well I have to do in the exam" "I like the idea of grades as I relate % back to it anyway" "The more I moved through university the more I have used the marking grid. In first year I didn't know how to use a marking grid" " GBA would make comparisons a lot easier. Some modules/assessments are perceived as being harder than others"

L5 and 6 student feedback

What did module leaders and tutors make of the changes? Staff feedback What did module leaders and tutors make of the changes? "Converting grades to percentages for SI and displaying grades on blackboard was time-consuming and fiddly" "I prefer the old style marking grid as I felt it was easier to mark the work based on each criteria rather than overall" " I found I was always comparing the grades I gave to the percentage equivalent" " Made it easier to offer explicit feedforward in how to move up a grade boundary" " Found it much to quicker to mark assignments" " Moderation was easier due to the reduced scale" " I was more confident in the final grade awarded than when using a marking grid" " At the end of the day, we are all experienced academics that should be able to make an informed judgement about the quality of a piece of work"

Staff (the verdict!) ` BB Grading schemas (time consuming) Converting marks to % for SI Two tasks (aggregated score) Percentage mindset! More confident in the grade awarded Quicker marking process. Moderation was easier More explicit feedforward

We are rolling GBA into L5 for SES Moving forward We are rolling GBA into L5 for SES Ensure assessment instruction clarity (screencast/in class). Ensure every mark scheme clearly outlines expectation (marking grid info). Develop/adapt assessment specific descriptors (e.g presentation/written piece). Make feedforward points explicit (staff training). GBA mindset! (readiness to change).

Acknowledgments Dr James Rumbold Dr Pete Olusoga Dr Sarah Churchill Dr Susie Crawford Dr Louise Turner

References Dalzeil, J. (1998) Using marks to assess student performance: Some problems and alternatives. Assessment and Evaluation in higher Education, 23, 351-366. Hornby, W. (2003). Assessing using grade-related criteria: A single currency for Universities? Assessment and Evaluation in higher Education, 28, 435-454. Sadler, D. (2005). Interpretations of criteria-based assessment and grading in higher education. Assessment & Evaluation in Higher Education, 30, 175–194.