Making the Grade Moving to a GPA-based System in a UK University Dr. Gavin Brown Senior Lecturer in Biochemistry, g.brown@lancaster.ac.uk

Please ask questions as we go along! A plan for the day Presentation – what has Lancaster done and why? Discussion – a role for GPA at Leeds? Break Presentation – impact on classification, effecting change Discussion – how might change happen at Leeds? Final questions Please ask questions as we go along!

The Lancaster GPA System

Drivers for change Simplification Comments from external examiners Limited use of the full marking range Complexity of our regulations Some inconsistency wrt operation of exam boards Concerns from LU staff Burgess Review Ensure regulations are fair, transparent and fit for purpose Possible future changes in the sector

Simplification An example from Lancaster’s old classification system The mark profile consisted of an overall average plus the mark distribution of 16 module marks (15 credits each) To get a first you need: An average of 70%+ OR An average of 68%+ and 8/16 marks at 70%+ OR An average of 68%+ and 6/16 marks at 70%+ OR An average of 68%+ from the best 14 marks and 8/16 at 70%+ OR An average of 68%+from the best 15 marks and 7/16 at 70%+ OR Exit velocity …

External examiner in a humanities subject, 2006 External examiners Common feedback about using the full range of marks in some subjects, but … “I read a project where the supervisor had commented that it was the best dissertation they had read in over 20 years and was of publishable standard. The supervisor had given it 72%. I thought that was rather unfair, I would have given it 78%” External examiner in a humanities subject, 2006

Concerns from LU staff Complexity Treatment of mitigating circumstances Inconsistency between subjects (joint degrees) Special rules Exit velocity

Summary of ‘problems’? Reluctance to use the full range of marks in some subjects Quantitative subjects wanted overall average, qualitative subjects wanted mark distribution Creeping complexity of the regulations Generous condonation rules Exams boards were fair but there were different approaches Inconsistent treatment of mitigating circumstances Lots of ‘special rules’ / system complexity

Marking - what we considered Leaving as was but introducing differential weighting Allowing departmental/faculty variation (use of mean or distribution) Moving to a step marking scale based on percentages Moving purely to a grading scale Mapping percentage ranges to grade points Optimal maximum GPA Number of points on any scale / in any class Linearity of scale …

Timeline Feb 2007 - Project started May 2010 - Approval of principles Nov 2010 - Final approval from Senate Oct 2011 - Introduced for 1st & 2nd years Jun 2013 - First graduating cohort

Lancaster’s new system Qualitative work marked in grades Quantitative work marked in percentages Marks converted to a GP (aggregation score) Classification based on overall GPA Criteria for borderlines Resits mandatory Minimum condonation threshold Set aside rules for mitigating circumstances

Grading criteria (qualitative work)

Grading: the pedagogical change Must not mark in % then ‘convert’ to grade Use objective criteria to assign a grade Initially assign the middle grade (A, B, C, D) then revise up or down using secondary descriptors Use objective criteria for feedback Bespoke marking criteria for the subject, level, assessment type, etc.

Marking in %s (quantitative work) Marking as usual Percentages converted to GP 0% = 0.000, 1% = 0.225, 2% = 0.450, 3% = 0.675, … … 40% = 9.000 … … 98% = 23.700, 99% = 23.850, 100% = 24.000

24 point scale A+ and 100% both equal 24 Grade Point Average 24 point scale A+ and 100% both equal 24 GRADE (qualitative work) OVERALL AGGREGATION SCORE (a GPA) AGGREGATION SCORE 0-24 (a ‘grade point’) PERCENTAGE (quantitative work) CLASSIFICATION

Calculation of module result

Calculation of final result

The Interactive Transcript A big win!!

Classification

Borderlines If a student’s overall GPA falls into a borderline then the higher award is given if: Half or more of the credits are in the higher class or The final year average is in the higher class (exit velocity) Exam boards can make individual cases to Senate

How is the new system better? Grading scheme linked to objective marking criteria Promotes use of the full marking range More straightforward marking Clearer opportunities for feedback Percentages still used for quantitative assessments Classification based on mean only Less condonation and only if a satisfactory attempt has been made Exam boards given greater guidance (and less discretion)

External examiner feedback – positive I am very supportive of the shift away from percentages to A,B,C grading I felt that the new marking system is clear and equitable in ways that our own new 17-point system is not I am happy to note that Lancaster have moved to a conceptual scale of marking (e.g. A+, A, A- etc). A percentage scale, particularly in Arts subjects, discourages marking at the top end, thus this new procedure will hopefully encourage assignment of marks across the entire scale I very much welcome the aggregation initiative with the use of letter grades, which will certainly solve the age-old problem of borderline marking and not using the full range of grades, particularly at the upper end of the scale Uniquely both new and old systems were in operation this year and I can’t help thinking the former will represent an improvement. It will be interesting to see whether the A band gets used more adventurously than the old first class band was, which is surely what it’s designed to do. Greater use of the A band breadth may also have the effect of pulling up B/C/D marks ie. because the dizzying heights of an A+ are a possibility now, that puts lower grades in a revised perspective

External examiner feedback – negative I wonder however if the new assessment system being phased in will be as easy to work with and as transparent to potential employers etc. I found the new marking scheme to be very confusing (as I was looking at 2nd and 3rd year work there were letter grades, marks out of 24 and %s used). I would recommend the University to use either letter grades or %s The “24 point” marking scheme is completely new to me, and is not something I have encountered any other university where I have been external examiner, nor in my own institution. I have not yet seen any meaningful justification why Lancaster University should employ this system, or how it helps students or examiners I am aware that the new assessment regulations are in operation and I broadly welcome its introduction. Despite, its introduction I would still encourage the staff in the department to utilise the full range of marks as my sense of conservatism in marking at the top end still has potential to create problems regarding a student’s classification as the letter grade mark needs to be translated into a numeric mark

A Role for GPA at Leeds?

Discussion Possible questions to consider … Are there any ‘problems’ with the current Leeds system? If yes, how might a GPA system help? If no, could enhancements be made? At what point would GPA be used (component, module, overall)? Would you want a mixed system (%s and grades)? How much subject variation would you want/need? What’s in it for your staff? What’s in it for your students? What might the reception be to change and why?

Lancaster GPA – Impact on Classification

Minimising risk: data modelling Not easy! Changing more than one variable All graduating cohorts from previous 3 years ‘Qualitative subjects’ – percentage ranges mapped to grades ‘Quantitative subjects’ – percentages converted to scores

Distribution of first class averages for final year % of firsts Aggregation Score

Classification results – overall % of cohort Year

Classification results – science and technology % of cohort Year

Classification results – business and management % of cohort Year

Classification results – arts, humanities and social sciences % of cohort Year

Classification conclusions Several factors at work (increasing entry grades, compulsory resits, final year resits, etc) Increase in overall proportion of ‘good degrees’ (GDs) broadly in line with increased intake qualifications Quantitative subjects show more GDs but with variability in firsts Qualitative subjects show slight improvement in GDs but number of firsts improved

Lancaster GPA – Effecting Change

How did we go about making a change? Took 3.5 years Started with a cross-faculty working party (including ‘difficult’ subjects) Consultation with other universities Establish principles Outline planning permission from Senate Several rounds of consultation with amendments and compromises Visits to departments Post-implementation monitoring group

Communication with staff and students Departmental staff meetings Faculty teaching committees Students’ Union Council Student Course Representatives Staff and Student FAQs Flyers + emails Student Newspaper Students’ Union social media

Staff FAQ How will work be marked in the new scheme? What do I put on student work? How are final marks calculated? What happens if a student fails my module? What do I do if a student has mitigating circumstances? What if a student has still failed after resitting? What does a student need to progress between years? What are the penalties for late submission? What will appear on a student’s transcript? …

Lessons learned Tackle the cause, not the symptom Be prepared to compromise Use enthusiasts The lead must invest a lot of time Consult with external examiners and employers Introduce parallel benefits for students Don’t underestimate system implications (student records systems, Turnitin, etc) Piecemeal is problematic!

Discussion Possible questions to consider … What would be the process of change at Leeds? Who would lead it? Who needs to be involved? What role would/might students have? Where would the hurdles be? Can you get the data for modelling? Is it worth it?

Any final questions …?