Blind double marking Aim: to raise some issues contingent on the introduction of blind double marking of essay exams.

Slides:

Advertisements

Similar presentations

Pre-entry qualifications – staff perceptions versus reality Sarah Maguire Derry Corey.

Advertisements

How External Examining Works in Practice John Malcolm BA (Oxon) MSc (Econ)

Reliability for Teachers Kansas State Department of Education ASSESSMENT LITERACY PROJECT1 Reliability = Consistency.

Newcastle University School of Chemistry National Student Survey 2013 W hat is the National Student Survey? An annual, anonymous survey of final year undergraduates,

1 NSS Biology Public Assessment (3 rd Consultation) October 2006 HKEAA.

Evaluating tests and examinations What questions to ask to make sure your assessment is the best that can be produced within your context. Dianne Wall.

Update from the UNC General Education Council [presented to the UNC Board of Governors’ Educational Planning, Programs, and Policies Committee on February.

Curriculum Evaluation Reverend Dr David Taylor Director of Quality Enhancement School of Medical Education University of Liverpool.

Assessing Transferable Skills Personal Development Planning David Cole-Hamilton School of Chemistry University of St. Andrews.

BRIEFING NOTES TO THE EDUCATION PORTFOLIO COMMITTEE 15 OCTOBER 2002 WHOLE-SCHOOL EVALUATION POLICY.

Diagnostic Metrics, Part 1 Week 2 Video 2. Different Methods, Different Measures  Today we’ll focus on metrics for classifiers  Later this week we’ll.

TA Marking1 Grading handbook This is an excellent resource from the University of Maryland. Download it from:

ON ORAL EXAMS Susana Tosca (7/12/12). Today´s menu Introduction round Presentation “On Oral Exams” Questions & Answers Video: an examination Discussion.

Marion Webb January  By the end of the session, participants will be able to:  Discuss the role of assessment for learning  Describe key assessment.

Chris Evans, University of Winchester Dr Paul Redford, UWE Chris Evans, University of Winchester Dr Paul Redford, UWE Self-Efficacy and Academic Performance:

The study voucher system. What is the study voucher system? - system implemented by the Flemish government - study vouchers = the total amount of credits.

MEASUREMENT CHARACTERISTICS Error & Confidence Reliability, Validity, & Usability.

A COORDINATOR ’ S YEAR PLAN Written by Rina AKOTONAS, Head of ACCESS Microscholarship Program AMAL NETWORK.

1 An Introduction to Language Testing Fundamentals of Language Testing Fundamentals of Language Testing Dr Abbas Mousavi American Public University.

1 First Briefing HKDSE 2015 Information and Communication Technology School Based Assessment.

Welcome to Higher Administration Mrs Gray. Timetable Monday – double block 1 & 2 Tuesday – block 6 Wednesday – block 3 Friday – block 5.

Our assessment objectives

© 2010 Curriculum Council © 2009 Curriculum Council TRIM: 2010/32879 Music Practical Examination Review Monday 6 December 2010.

FINAL YEAR PROJECT (FYP) ERT 423 SESSION 2010/2011 “INTRODUCTION” MISMISURAYA MEOR AHMAD

Year 10 Coffee morning Sunday 22 nd March 2015 Morning parents, please help yourselves to tea and coffee.

PRESENTATION ON ASSESSMENT Compiled & Presented by: Mrs T.P. Sithole.

Assessment Matters … Monday December 5 th 2011 Student Union Academic Council Andy Lloyd, Assessment Project Manager.

Assessment seminar 2010 Accounting and Finance 2010/05037.

Inter-rater reliability in the KPG exams The Writing Production and Mediation Module.

No tests are dropped No tests are given over No extra time is given Calculators can be used Partial Credit is given Non – calculator No partial credit/multiple.

Final FRCA SOE Course Evaluation Course 1 : June 1 st – 2 nd 2015.

The Longitudinal Student Assessment Project (LSAP)

Wr w i s s e n s c h a f t s r a t the german council of science and humanities provides advice to the german federal government and the state governments.

Donal Crawford International Pacific College July 2010.

Edexcel Certificate in English Literature a brief outline of the course what students will be doing between now and the exam how we can all support them.

Key Stage 3 Assessment and Reporting November 2014 Specialist School in Mathematics and Business St Mary's High School, Newry.

Marking/Second Marking & Feedback Dr Robert Hamilton AD(T) Loughborough University.

Key Stage 4 Assessment and Reporting November 2014 Specialist School in Mathematics and Business St Mary's High School, Newry.

British Society c Exam Technique 1 Understanding the Page Layout Example Question. Assessment Criteria: this is what is being tested in the.

The County High School, Leftwich Year 10 Information Evening.

No tests are dropped No tests are given over No extra time is given Calculators can be used Partial Credit is given Non – calculator No partial credit/multiple.

Different Models for Student Assessment under Egyptian Constraints Dr Robert Schofield Advisor to the Quality Assurance and Accreditation Project Republic.

AMIR ATAEE, SONNIE AVENBUAN, MARVIN BILLINGS, SHARICE CANNADY, TERRANCE HAMILTON, LLIVER JOSE, DAVIS MIRILLA, MARK PISANO, MICHAEL POWELL, EGAL SANCHEZ,

British Society c Exam Technique 1 © Owen Scott Understanding the Page Layout Example Question. Assessment.

The Role of the Internal and External Evaluators in Student Assessment Arthur Brown Advisor to the Quality Assurance and Accreditation Project Republic.

External Examiners’ Workshop The role of the external examiner and its requirements at the University of Brighton Professor Stephen Denyer Pro-Vice-Chancellor.

1 A Framework for Junior Cycle BRIEFING October 2012.

English. As a parent/carer:  To help you to be fully aware of your child’s learning  What do National 4 or 5 look like for your child?  What is the.

Assessment for Learning Centre for Academic Practice Enhancement, Middlesex University.

OBJECTIVE INTRODUCTION Emergency Medicine Milestones: Longitudinal Interrater Agreement EM milestones were developed by EM experts for the Accreditation.

The New Junior Cycle – An Introduction. Starting in September, your son/daughter will do the current Junior Certificate exam in all subjects except English,

Training Week, August 2016 Assessment for

An Introduction to External Examining Procedures at Bangor University

FAEIs Student Contest Jolene Hamm.

An Introduction to External Examining Procedures at Bangor University

Natalie Robinson Centre for Evidence-based Veterinary Medicine

English in the Senior Phase

Conclusions and Next Steps

External Examiner Induction

Implementing the New Qualifications. Session and beyond.

Sociology Outcomes Assessment

Assessment Introduction

Cuyamaca College Library

In-Service Teacher Training

Graphic Design Semester 1 Final Exam

First Year 2019 Subject Choice

Examinations Council of Zambia

HT 1 (3rd September – Friday 25th October, weeks) HT 2 Wk1 Wk2 Wk3 Wk4

Presentation transcript:

Blind double marking Aim: to raise some issues contingent on the introduction of blind double marking of essay exams

BChD ( Bachelor of Dental Surgery ) O 5 year course O non-modular (curriculum largely determined by General Dental Council) O degree exams in 1st, 2nd, 3rd and 5th year plus ‘Progress to Finals’

The written exam in ‘Finals’ O takes place in June of 5th year O = 40% of marks for Finals O 2 three-hour papers O 4 compulsory essay questions in each paper O each essay marked out of 25 O traditionally double-marked O first double-marked ‘blind’ in 1999

Double-marking vs. blind double-marking O traditionally 2nd marker agreed with 1st mark 70% - 80% of time and disagreements nearly always only marks (out of 25) O marking blind, absolute agreement dropped to between 48% and 17% with, in one question, 35% of marks being >2 marks apart

Extent of agreement between markers in 1999 final written exam

Analysis of markers’ consensus process (all questions, 48 students)

Inappropriate use of correlation statistics 5 Correlation statistics show association, not agreement e.g. correlation would be perfect if all marks given by 2nd marker are exactly 10 points higher than those given by 1st marker 4 therefore use kappa, which measures absolute agreement with reference to the frequency with which marks would be expected to agree by chance Note: agreement can be evaluated as: Poor (<0.20), Fair ( ), Moderate ( ), Good ( ) or Very Good ( )

Standard ways of improving inter-marker reliability 4 re-visit assessment criteria/marking scheme 4 have standardisation meetings with markers 4 downgrade importance of essays in assessment and use more objective methods 4 continue to monitor marking S

Action taken between June and November Issues raised and discussed at Undergraduate Dental Education Committee and in Dental School newsletter ‘Teaching Quality Matters’ 4 Assessment criteria reviewed, re-formatted and published in Nathan Bodington 4 Markers for Operative Dentistry prize exam (December 1999) given verbal briefing about assessment criteria S Result = a marked improvement in reliability !! (but….)