1. Brunel University Briefing for External Examiners Spring 2013 Derek Milligan Director of Academic Programme Development

2. Topics Panels of Examiners Panels of Examiners Boards of Examiners Boards of Examiners Mitigating Circumstances Panels Mitigating Circumstances Panels Regulations Regulations

3. Panels and Boards are responsible directly to Senate Senate Regulation 4 governs Panels of Examiners, Boards of Examiners and Mitigating Circumstances Panels Senate Regulation 4 governs Panels of Examiners, Boards of Examiners and Mitigating Circumstances Panels

4. Panels and Boards of Examiners Combined Purpose To ensure: Fairness to all students and each student Fairness to all students and each student Rigour of assessment Rigour of assessment Robustness of the results and decisions Robustness of the results and decisions Maintenance of Standards Maintenance of Standards

5. Assessment Waypoints Specify assessment tasks and criteria Specify assessment tasks and criteria Approval of tasks and criteria Approval of tasks and criteria Students take Students take Mitigating circumstances Mitigating circumstances Marking/Grading Marking/Grading Moderation (Internal and External) Moderation (Internal and External) Panels of Examiners Panels of Examiners Boards of Examiners Boards of Examiners

6. Panels of Examiners - Purpose To verify and confirm the marks/grades for each module/block To verify and confirm the marks/grades for each module/block Not to look at individual students (unless error or additional info presented) Not to look at individual students (unless error or additional info presented) May make specific recommendations to Boards concerning particular assessments May make specific recommendations to Boards concerning particular assessments Implement Senate Regulations 4.52-4.66 Implement Senate Regulations 4.52-4.66 Mitigating Circumstances Panels are different Mitigating Circumstances Panels are different – see later

7. Senate Regulation 4.64 A Panel of Examiners shall not confirm grades/marks for an assessment block until it is satisfied with the integrity and fairness of the assessment(s) leading to the grades/marks. Where the Panel of Examiners has insufficient confidence in the integrity and fairness of the outcomes of an assessment, it shall take appropriate action in order to achieve sufficient confidence. The Panel of Examiners may require the reconsideration by assessors of the grades/marks for the complete cohort of students taking an assessment. Only in exceptional circumstances may the Panel directly adjust the grades/marks for a particular assessment and must then must record the justification and rationale for the adjustment. Grades/marks for an individual student may not be adjusted, unless they have been wrongly recorded or additional information is presented. A Panel of Examiners shall not confirm grades/marks for an assessment block until it is satisfied with the integrity and fairness of the assessment(s) leading to the grades/marks. Where the Panel of Examiners has insufficient confidence in the integrity and fairness of the outcomes of an assessment, it shall take appropriate action in order to achieve sufficient confidence. The Panel of Examiners may require the reconsideration by assessors of the grades/marks for the complete cohort of students taking an assessment. Only in exceptional circumstances may the Panel directly adjust the grades/marks for a particular assessment and must then must record the justification and rationale for the adjustment. Grades/marks for an individual student may not be adjusted, unless they have been wrongly recorded or additional information is presented.

8. Role of External Examiners at Panels Full membership of the PoE (including voting) Full membership of the PoE (including voting) Contribute to the collective academic decision-making Contribute to the collective academic decision-making Have a distinct influence on the PoE (through their independence and their overview) Have a distinct influence on the PoE (through their independence and their overview) EEs are not empowered to change marks or grades – all decisions are made collectively EEs are not empowered to change marks or grades – all decisions are made collectively May ask to inspect any documents involved in the assessment process May ask to inspect any documents involved in the assessment process See SR4.122 – 4.132 See SR4.122 – 4.132

9. Issues for Panels Potential Assessment Design Errors Potential Assessment Design Errors Academic Judgment Academic Judgment Moderation effectiveness Moderation effectiveness Extraordinary factors Extraordinary factors Adjustment of Results Adjustment of Results Justification of Actions Justification of Actions Recording of Decisions at all stages! Recording of Decisions at all stages!

10. Design Errors Potential Locations: Learning Outcomes Coverage Learning Outcomes Coverage Element Weightings Element Weightings Assessment Specification Assessment Specification Sub-element combination method Sub-element combination method Assessment Criteria Specification Assessment Criteria Specification Marking/Grading strategy Marking/Grading strategy Moderation process Moderation process

11. Academic Judgement The vast majority of the collective academic judgement is exercised in PANELS, not in Boards, of Examiners The vast majority of the collective academic judgement is exercised in PANELS, not in Boards, of Examiners Boards are more about judgement by academics, rather than academic judgement – but there may be extraordinary academic judgements needed Boards are more about judgement by academics, rather than academic judgement – but there may be extraordinary academic judgements needed

12. Boards of Examiners - Purpose To receive confirmed marks/grades from Panels To receive confirmed marks/grades from Panels To consider individual student profile of achievement To consider individual student profile of achievement To take into account mitigating circumstances and determine appropriate actions To take into account mitigating circumstances and determine appropriate actions To decide on progression and re-assessment To decide on progression and re-assessmentor To recommend to Senate appropriate awards for each student Implement Senate Regulations 4.52-4.58 and 4.67-4.80 Implement Senate Regulations 4.52-4.58 and 4.67-4.80 NB Boards now have few powers of discretion within the regulations (compared with the old regulations) – but all decisions must still be justified

13. Role of External Examiners at Boards Full membership of the BoE (including voting) Full membership of the BoE (including voting) Contribute to the collective academic decision-making Contribute to the collective academic decision-making Have a distinct influence on the BoE (through their independence and their overview) Have a distinct influence on the BoE (through their independence and their overview) EEs are not empowered to change marks or grades/progression/classification unilaterally – all decisions are made collectively EEs are not empowered to change marks or grades/progression/classification unilaterally – all decisions are made collectively Assure themselves that due process followed and appropriate consideration and decisions have been made by the BoE Assure themselves that due process followed and appropriate consideration and decisions have been made by the BoE See Senate Regulations SR4.122 – 4.132 See Senate Regulations SR4.122 – 4.132

14. Issues for Boards Mitigating Circumstances Mitigating Circumstances Academic Judgment Academic Judgment Progression and Re-assessment Progression and Re-assessment Awards Awards Extraordinary Decisions - Discretion Extraordinary Decisions - Discretion Justification of Decisions Justification of Decisions Recording of Decisions at all stages! Recording of Decisions at all stages! Condonation, Set-Aside & Borderlines (UG pre-2009 only) Condonation, Set-Aside & Borderlines (UG pre-2009 only)

15. Board Discretion Potential Locations: Action re MCs Action re MCs Treatment of AP(E)L – NB New processes for 2013/4 will obviate need for BoE discretion Treatment of AP(E)L – NB New processes for 2013/4 will obviate need for BoE discretion Extraordinary situations Extraordinary situations Re-assessment (mainly PG) Re-assessment (mainly PG) Condonation/Set-Aside/Promotion across boundaries/Re-assessment (UG Old SR2 only) Condonation/Set-Aside/Promotion across boundaries/Re-assessment (UG Old SR2 only)

16. Mitigating Circumstances Panels Report to Board of Examiners Report to Board of Examiners Consider MC submissions from students Consider MC submissions from students Have no access to academic results when considering cases Have no access to academic results when considering cases Consider the likely impact on studies/assessment Consider the likely impact on studies/assessment Accept or reject MCs Accept or reject MCs Make decisions directly re coursework submission deadlines with MCs Make decisions directly re coursework submission deadlines with MCs All other accepted MCs -> Board of Examiners All other accepted MCs -> Board of Examiners MC Panel will identify serious cases to the BoE MC Panel will identify serious cases to the BoE SR4 governs treatment of MCs + see Guidance on MCs SR4 governs treatment of MCs + see Guidance on MCs

17. Actions re MCs by Boards See Senate Regulations 4.43 – 4.51 See Senate Regulations 4.43 – 4.51 Essentially, Boards can take any appropriate action it sees fit except: Essentially, Boards can take any appropriate action it sees fit except: The marks/grades for individual assessment elements (e.g. exam script, piece of coursework, etc) may not be changed – i.e., the actual level of performance recorded for an individual assessment (confirmed by a PoE) cannot be altered. The marks/grades for individual assessment elements (e.g. exam script, piece of coursework, etc) may not be changed – i.e., the actual level of performance recorded for an individual assessment (confirmed by a PoE) cannot be altered.

18. Grounds for Appeal (SR6) 6.5 A student may appeal against the decision of a Board of Examiners on any of the following grounds: 6.5 A student may appeal against the decision of a Board of Examiners on any of the following grounds: a that there exist circumstances materially affecting the student's performance which were not known to the Board of Examiners when its decision was taken and which it was not reasonably practicable for the student to make known to the Board beforehand; a that there exist circumstances materially affecting the student's performance which were not known to the Board of Examiners when its decision was taken and which it was not reasonably practicable for the student to make known to the Board beforehand; b that there were procedural irregularities in the conduct of the examinations and/or assessment procedures, including assessment of coursework, of such a nature as to create a reasonable possibility that the result might have been different had they not occurred; b that there were procedural irregularities in the conduct of the examinations and/or assessment procedures, including assessment of coursework, of such a nature as to create a reasonable possibility that the result might have been different had they not occurred; c that there is evidence of prejudice, bias or inadequate assessment on the part of one or more Examiners. c that there is evidence of prejudice, bias or inadequate assessment on the part of one or more Examiners. 6.6 No appeal shall be allowed on the grounds that, although the decision of the Examiners was properly made, the Board of Examiners is alleged to have erred in its judgement of the academic standard achieved by the student. 6.6 No appeal shall be allowed on the grounds that, although the decision of the Examiners was properly made, the Board of Examiners is alleged to have erred in its judgement of the academic standard achieved by the student.

19. Documentation For all summative assessment there should be a complete documented audit trail for all assessments – For all summative assessment there should be a complete documented audit trail for all assessments – all the way from specification of an individual piece of assessment and its checking all the way from specification of an individual piece of assessment and its checking through marking and moderation through marking and moderation to Panel and Board decisions to Panel and Board decisions

20. Senate Regulations Undergraduate Programmes & Awards SR2 (2009 onwards) SR2 (2009 onwards) SR2 (pre-2009 entry, very few students now) SR2 (pre-2009 entry, very few students now) Postgraduate Programmes & Awards SR3 (2006 onwards) SR3 (2006 onwards) SR3 (pre-2006 entry, very few students now) SR3 (pre-2006 entry, very few students now) Assessment in Taught Programmes SR4 (combined) SR4 (combined)

21. Old UG (pre-2009) Regulations Progression and Awards based on % average + gaining of 360 credits Progression and Awards based on % average + gaining of 360 credits Compensatory mechanisms Compensatory mechanisms Condonation (at BoE discretrion) Condonation (at BoE discretrion) Set-aside (at BoE discretion) Set-aside (at BoE discretion) Best 100 credits Best 100 credits Trailing allowed Trailing allowed No limit but much BoE discretion on volume of re-assessment No limit but much BoE discretion on volume of re-assessment

22. Current Regulations (SR2/3) Transparency of Standards of Awards Transparency of Standards of Awards Equity of Treatment Equity of Treatment University Grade Descriptors are Primary Standards Reference University Grade Descriptors are Primary Standards Reference Grades are primary performance measure Grades are primary performance measure Defined minimum grade profile requirements for awards Defined minimum grade profile requirements for awards Credit is not “awarded” – credit is solely a volume/weighting metric Credit is not “awarded” – credit is solely a volume/weighting metric “Core” assessments can be defined (must be passed) “Core” assessments can be defined (must be passed) Reassessment volume right = limit (no UG BoE discretion, but some at PGT currently) Reassessment volume right = limit (no UG BoE discretion, but some at PGT currently)

23. Current UG (2009) Regulations De-modularised Regulations (Assessment Blocks & Study Blocks) De-modularised Regulations (Assessment Blocks & Study Blocks) 17-point grade scale (A*,A+,A,A-,…F) with % mark equivalence defined 17-point grade scale (A*,A+,A,A-,…F) with % mark equivalence defined Progression and (Threshold) Award based on Grade Profile over defined volume of Credit Progression and (Threshold) Award based on Grade Profile over defined volume of Credit Classification based on combination of GPA + volume in class Classification based on combination of GPA + volume in class No compensation/condonation mechanisms No compensation/condonation mechanisms No Trailing allowed (Except for some cases of MCs and with individual Senate approval) No Trailing allowed (Except for some cases of MCs and with individual Senate approval) Integration of Sandwich placements into degree Integration of Sandwich placements into degree

24. UG Integration of Sandwich Placements (SR2 2009-, 2.46-48) Special award name (defined in the Programme Specification) available for sandwich students who have passed (D-) the placement assessment. Special award name (defined in the Programme Specification) available for sandwich students who have passed (D-) the placement assessment. For this special award the placement module grade is included in the Level 2 grade profile for award. For this special award the placement module grade is included in the Level 2 grade profile for award. Percentage contribution to Level 2 of the placement module is defined in the Programme Specification. Percentage contribution to Level 2 of the placement module is defined in the Programme Specification. All students who have failed the placement module (or who choose not to have it included in the award grade profile) are eligible for the standard (non-sandwich) award. All students who have failed the placement module (or who choose not to have it included in the award grade profile) are eligible for the standard (non-sandwich) award. 2012/13 is first year of awards with integrated sandwich placements 2012/13 is first year of awards with integrated sandwich placements

25. UG Grade Scale Indicative Mark Band Degree Class Equivalent GradeGrade Point 90 and above1A*17 80-891A+16 73-791A15 70-721A-14 68-692.1B+13 63-672.1B12 60-622.1B-11 58-592.2C+10 53-572.2C9 50-522.2C-8 48-493D+7 43-473D6 40-423D-5 38-39FailE+4 33-37FailE3 30-32FailE-2 29 and belowFailF1

26. Current PGT (2006) Regulations Progression and (Threshold) Award based on Grade Profile over defined volume of Credit Progression and (Threshold) Award based on Grade Profile over defined volume of Credit Masters Pass, Merit, Distinction Classifications Masters Pass, Merit, Distinction Classifications Classification based on minimum grade profile for class Classification based on minimum grade profile for class No compensation/condonation mechanisms No compensation/condonation mechanisms SR3 for 2013/4 onwards will provide alignment with current SR2 SR3 for 2013/4 onwards will provide alignment with current SR2 De-modularised Regulations (assessment & study blocks) De-modularised Regulations (assessment & study blocks) Finer grade scale Finer grade scale Use of GPA for Classification Use of GPA for Classification

27. PGT Classification Masters Distinction minimum profile: Masters Distinction minimum profile: 120 cr A (incl Dissertation) + 30 cr B + 30 cr C Masters Merit minimum profile : Masters Merit minimum profile : 120 cr B (incl Dissertation) + 60 cr C Masters Pass minimum profile : Masters Pass minimum profile : 120 cr C (incl Dissertation) + 60 cr D PGDip minimum profile : 60 cr C + 60 cr D PGDip minimum profile : 60 cr C + 60 cr D PGCert minimum profile : 30 cr C + 30 cr D PGCert minimum profile : 30 cr C + 30 cr D

28. Old UG (pre-2009) Classification L3:L2 weighting 2:1 L3:L2 weighting 2:1 % Average over best 100 credits at L2 & L3 + 360 credits awarded % Average over best 100 credits at L2 & L3 + 360 credits awarded 3 rd : 40% average 3 rd : 40% average 2.2: 50% average 2.2: 50% average 2.1: 60% average 2.1: 60% average 1 st : 70% average 1 st : 70% average Variable Bordelines Variable Bordelines Condonation and set-aside used to award credit when module not passed – at BoE discretion Condonation and set-aside used to award credit when module not passed – at BoE discretion Please note that very few, if any, of Bachelors awards at 2012/13 BoEs will be under these regulations Please note that very few, if any, of Bachelors awards at 2012/13 BoEs will be under these regulations

29. Current UG (2009-) Classification L3:L2 weighting 2:1 as before L3:L2 weighting 2:1 as before Two-stage classification decision Two-stage classification decision Test 1 – limits of sub-threshold credit volume (determines eligibility for 3 rd ) Test 1 – limits of sub-threshold credit volume (determines eligibility for 3 rd ) Test 2 – Minimum GPA + In-class Volume (determines classification) Test 2 – Minimum GPA + In-class Volume (determines classification) Please note that almost all Bachelors awards at 2012/2013 BoEs will be under these regulations Please note that almost all Bachelors awards at 2012/2013 BoEs will be under these regulations

30. Current UG Classification – Test 1 Maximum credit volume of assessment blocks containing core assessments below D- Maximum (non-weighted) credit volume of non- core Grade Band E (E+,E,E-) Maximum credit volume of non-core Grade F Level 2Level 3Level 2 + 3Level 2Level 3 1 st 000000 2.10200 00 2.2040204000 3 rd 040 6000

31. Current UG Classification – Test 2 Class 1 st 2.12.23rd Volume of grades (weighted) in Class or better Minimum weighted GPA At least 33% of grades (weighted) in Class or better 14.511.58.5NA At least 41% of grades (weighted) in Class or better 13.510.57.5NA At least 50% of grades (weighted) in Class or better 12.59.56.5NA At least 58% of grades (weighted) in Class or better 11.58.55.5NA At least 66% of grades (weighted) in Class or better 10.57.54.5NA At least 72% of grades (weighted) in Class or better NA 3.3

32. Classification Example 1 Level 2 Profile (20 cr blocks): BB - CC + DE (E non- core) Level 2 Profile (20 cr blocks): BB - CC + DE (E non- core) Level 2 GPA: 12+11+9+10+6+3/6= 8.5 Level 2 GPA: 12+11+9+10+6+3/6= 8.5 Level 3 Profile (20 cr blocks): AA - BB - CD Level 3 Profile (20 cr blocks): AA - BB - CD Level 3 GPA: 15+14+12+11+9+6/6= 11.16 Level 3 GPA: 15+14+12+11+9+6/6= 11.16 Test 1: No core credit below D- = 0 No core credit below D- = 0 Grade F credit = 0 Grade F credit = 0 Non-core credit in E band = 20 at Level 2 Non-core credit in E band = 20 at Level 2 => maximum class = 2.1 => maximum class = 2.1 Test 2: Overall weighted GPA = (8.5+(11.16x2)) /3= 10.27 Overall weighted GPA = (8.5+(11.16x2)) /3= 10.27 Consider for 2.1 with weighted GPA = 10.27 Needs 50% in 2.1 Class to get a 2.1 Needs 50% in 2.1 Class to get a 2.1 Proportion of weighted grade B- or better = Proportion of weighted grade B- or better = 1/3 at L2 and 2/3 at L3 = (1/3 +2/3 +2/3)/3 = 55.55% => Classification is 2.1 Maximum credit volume of assessment blocks containing core assessments below D- Maximum (non-weighted) credit volume of non-core Grade Band E (E+,E,E-) Maximum credit volume of non-core Grade F Level 2 Level 3 Level 2 + 3 Level 2 Level 3 1 st 000000 2.10200 00 2.2040204000 3 rd 040 6000 Class 1 st 2.12.23rd Volume of grades (weighted) in Class or better Minimum weighted GPA At least 33% of grades (weighted) in Class or better 14.511.58.5NA At least 41% of grades (weighted) in Class or better 13.510.57.5NA At least 50% of grades (weighted) in Class or better 12.59.56.5NA At least 58% of grades (weighted) in Class or better 11.58.55.5NA At least 66% of grades (weighted) in Class or better 10.57.54.5NA At least 72% of grades (weighted) in Class or better NA 3.3 Gra de Gra de Poin t A*17 A+16 A15 A-14 B+13 B12 B-11 C+10 C9 C-8 D+7 D6 D-5 E+4 E3 E-2 F1

33. Classification Example 2 Level 2 Profile (20 cr blocks): BB - CC + DE (E non- core) Level 2 Profile (20 cr blocks): BB - CC + DE (E non- core) Level 2 GPA: 12+11+9+10+6+3/6= 8.5 Level 2 GPA: 12+11+9+10+6+3/6= 8.5 Level 3 Profile (20 cr blocks): ABCDDD Level 3 Profile (20 cr blocks): ABCDDD Level 3 GPA: 15+12+9+6+6+6/6= 9.0 Level 3 GPA: 15+12+9+6+6+6/6= 9.0 Test 1: No core credit below D- = 0 No core credit below D- = 0 Grade F credit = 0 Grade F credit = 0 Non-core credit in E band = 20 at Level 2 Non-core credit in E band = 20 at Level 2 => maximum class = 2.1 => maximum class = 2.1 Test 2: Overall weighted GPA = (8.5+(9.0x2)) /3= 8.83 Overall weighted GPA = (8.5+(9.0x2)) /3= 8.83 Consider for 2.1 with weighted GPA = 8.83 Needs 58% in 2.1 Class to get a 2.1 Needs 58% in 2.1 Class to get a 2.1 Proportion of weighted grade B- or better = Proportion of weighted grade B- or better = 1/3 at L2 and 1/3 at L3 = (1/3 +1/3 +1/3)/3 = 33.33% => 2.1 not achieved Consider for 2.2 with weighted GPA = 8.83 Needs 33% in 2.2 Class to get a 2.2 Needs 33% in 2.2 Class to get a 2.2 Proportion of weighted grade C- or better = Proportion of weighted grade C- or better = 2/3 at L2 and 1/2 at L3 = (2/3 +(1/2 +1/2))/3 = 55.55% => Classification is 2.2 Maximum credit volume of assessment blocks containing core assessments below D- Maximum (non-weighted) credit volume of non-core Grade Band E (E+,E,E-) Maximum credit volume of non-core Grade F Level 2 Level 3 Level 2 + 3 Level 2 Level 3 1 st 000000 2.10200 00 2.2040204000 3 rd 040 6000 Class 1 st 2.12.23rd Volume of grades (weighted) in Class or better Minimum weighted GPA At least 33% of grades (weighted) in Class or better 14.511.58.5NA At least 41% of grades (weighted) in Class or better 13.510.57.5NA At least 50% of grades (weighted) in Class or better 12.59.56.5NA At least 58% of grades (weighted) in Class or better 11.58.55.5NA At least 66% of grades (weighted) in Class or better 10.57.54.5NA At least 72% of grades (weighted) in Class or better NA 3.3 Gra de Gra de Poin t A*17 A+16 A15 A-14 B+13 B12 B-11 C+10 C9 C-8 D+7 D6 D-5 E+4 E3 E-2 F1