Self and Peer Assessment Through DUO Hannah Whaley University of Dundee Hannah Whaley University of Dundee

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Background University of Dundee –Growing use of many forms of self and peer assessment –Paper based systems and ad hoc online approaches –Re-developed a system to integrate with Bb New system –User centered design –Involved academic staff from range of subjects Partnered with Blackboard –Fully integrated with Blackboard –Released in v8.0 so available now

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Background

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Background

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment 2. Deciding Where and When 3. Designing Assessments 4. Running Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment 2. Deciding Where and When 3. Designing Assessments 4. Running Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Important to fully understand the concept –Confusion over terminology –Focus on the real use of the pedagogy –Only then can you realise the full potential for learning 1. Understanding Self and Peer Assessment peer review peer marking groupwork assessment self assessment peer reflection criterion based reference marking marking rubrics critical analysis

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment self and peer assessment not group individual work fixed marking criteria reflection, analysis, evaluation

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment Process –Academics design assessment Includes questions and marking criteria Creates the assessment in Blackboard –Student completes the assessment Could be one or more questions Submits answers in Blackboard –Student marks the assessment Returns to assessment in Blackboard Is given a list of students to mark –Academic moderates results Monitors submissions and marking phases Moderates results before releasing them to students

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment Challenging process for both staff and students Students Reflection Critical Constructive Engage Academics Be creative Be precise Let go! Moderate Sys Admin Understand, support and get excited

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment 2. Deciding Where and When 3. Designing Assessments 4. Running Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Formative or summative? –Formative works particularly well –Summative should include moderation offer Replace old assessment or add new one? –Updating old assessments works well –Chance to add innovative new practice What’s the purpose of the assessment? –Add interaction, reduce marking load, extra practice, new skills –Focus on purpose in assessment design How long should it run for? –2 weeks standard – use the defaults that are given –1 week assessment, 1 week evaluating Supervise it in IT suites or not? –Generally, can be completed entirely online –Makes good use of practical session 2. Deciding Where and When

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 2. Deciding Where and When FundamentalExtendedEmerging AnnouncementePortfolioStudent monitoring Organisational informationContent managementPodcast / Videocast PowerpointsPortalWeb-conference / Web-cast Web linksDiscussion boardBlog / Journal Online assessmentOffice hoursWikis (collaboration) Online tutorialPlagiarism detectionInteractive classroom Online books & journalsSimulationsPuzzles & games Online submissionSynchronous chat (incl. IM)Intelligent searching Online evaluationTimetablingSelf & peer assessment Personal dev. planning Specialist learning programs Gradecenter Mobile learning– SMS, PDA RSS LOW HIGH interactivity

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 2. Deciding Where and When FundamentalExtendedEmerging AnnouncementePortfolioStudent monitoring Organisational informationContent managementPodcast / Videocast PowerPointPortalWeb-conference / Web-cast Web links Discussion BoardsBlog / Journal Online assessmentOffice hoursWikis (collaboration) Online tutorialPlagiarism detectionInteractive classroom Online books & journalsSimulationsPuzzles & games Online submissionSynchronous chat (incl. IM)Intelligent searching Online evaluationTimetablingSelf & peer assessment Personal dev. planning Specialist learning programs Grade center Mobile learning– SMS, PDA RSS LOW HIGH interactivity

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 2. Deciding Where and When FundamentalExtendedEmerging AnnouncementePortfolioStudent monitoring Organisational informationContent managementPodcast / Videocast PowerPointPortalWeb-conference / Web-cast Web linksDiscussion boardBlog / Journal Online assessmentOffice hoursWikis (collaboration) Online tutorialPlagiarism detectionInteractive classroom Online books & journalsSimulationsPuzzles & games Online submissionSynchronous chat (incl. IM)Intelligent searching Online evaluationTimetablingSelf & peer assessment Personal dev. planning Specialist learning programs Grade center Mobile learning– SMS, PDA RSS LOW HIGH interactivity

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 2. Deciding Where and When FundamentalExtendedEmerging AnnouncementePortfolioStudent monitoring Organisational informationContent managementPodcast / Videocast PowerPointPortalWeb-conference / Web-cast Web linksDiscussion boardBlog / Journal Online assessmentOffice hoursWikis (collaboration) Online tutorialPlagiarism detectionInteractive classroom Online books & journalsSimulationsPuzzles & games Online submissionSynchronous chat (incl. IM)Intelligent searching Online evaluationTimetablingSelf & peer assessment Personal dev. planning Specialist learning programs Grade center Mobile learning– SMS, PDA RSS LOW HIGH interactivity

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 2. Deciding Where and When FundamentalExtendedEmerging AnnouncementePortfolioStudent monitoring Organisational informationContent managementPodcast / Videocast PowerPointPortalWeb-conference / Web-cast Web linksDiscussion boardBlog / Journal Online assessmentOffice hoursWikis (collaboration) Online tutorialPlagiarism detectionInteractive classroom Online books & journalsSimulationsPuzzles & games Online submissionSynchronous chat (incl. IM)Intelligent searching Online evaluationTimetabling Self & peer assessment Personal dev. planning Specialist learning programs Grade center Mobile learning– SMS, PDA RSS LOW HIGH interactivity

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 2. Deciding Where and When Traditional Self and Peer Creates Question Creates Criteria Marking Answers Writing Feedback Moderation Formal marks Exercise review Prepares Answer Reviews Feedback Creates Question Creates Criteria Moderation Formal marks Exercise review Prepares Answer Reviews Feedback Marking Answers Writing Feedback

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment 2. Deciding Where and When 3. Designing Assessments 4. Running Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Focus of assessment Learning objectives (primary and secondary) Discipline specific context Flexibility within tool for design 3. Designing Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Assessment Question Criteria Question Criteria - Essay style and exam style assessments are catered for - Submission options include text, html and links - Anonymous or not, change number of peers to mark

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 1 Subject: Life Sciences Motive: Reduce Marking Time Extra Learning: Practice at exam q’s Old or New: Created from old tutorials Exam style 30 Questions 1 criteria each Very specific criteria Model answers

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 1 Subject: Life Sciences Motive: Reduce Marking Time Extra Learning: Practice at exam q’s Old or New: Created from old tutorials Subject specific Text answers File uploads Majority no feedback Marking 2 peers and self

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 1 Subject: Life Sciences Motive: Reduce Marking Time Extra Learning: Practice at exam q’s Old or New: Created from old tutorials Only 1 exercise used per year previously 4 hours moderating 4 exercises Students gain lots of practice Common mistakes, marking scales, model answers

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 2 Subject: Geography Motive: Innovative practical lab Extra Learning: Understand their answers Old or New: New idea Bit of Both 2 Questions Subjective and Specific criteria Granular and expansive marks

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 2 Subject: Geography Motive: Innovative practical lab Extra Learning: Understand their answers Old or New: New idea Deep Learning Text answers Subjective and Specific criteria Marking 3 peers and self

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 2 Subject: Geography Motive: Innovative practical lab Extra Learning: Understand their answers Old or New: New idea Innovative way to introduce students to academic reading Promoting deep learning – synthesis and evaluation Students forced to give opinions and justify them Makes use of flexibility of the system, combining two approaches

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 3 Subject: Law Motive: Improve assessment Extra Learning: Give better feedback Old or New: Added online component Blended style File upload Open Criteria, with guidance Small workload

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 3 Subject: Law Motive: Improve assessment Extra Learning: Give better feedback Old or New: Added online component Quick answers 3 markers Self reflection Emphasis on constructive feedback

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 3. Designing Assessments Example 3 Subject: Law Motive: Improve assessment Extra Learning: Give better feedback Old or New: Added online component Blending online component with existing teaching practices Enhance face to face section, formalise feedback Students get better feedback, from a wider range Understand better and worse presentations clearly

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 1. Understanding Self and Peer Assessment 2. Deciding Where and When 3. Designing Assessments 4. Running Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Flexibility built into system –Timing of assessments –Workload –Publishing Results Motivation Moderation 4. Running Assessments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 4. Running Assessments Motivation –Student understanding of process Importance for their learning –Assignment completed after both parts Marks can be withheld –Actively encourage non completers and remind them Sometimes… –Marks for marking –Deviation from average or tutor mark

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 4. Running Assessments Moderation –Moderator can over ride any average grade –3 key phases each with moderator overview Submission Marking Results

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 4. Running Assessments Moderating Submissions –Encourage –Check submissions for problems –Download submissions

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 4. Running Assessments Moderating Evaluations –Encourage –Check for problems –Download evaluation results

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 4. Running Assessments Moderating Results –Check for problems –Finalise and publish to Grade Center –Any grade can be over-ridden in the Grade Center –Add feedback or grading notes

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE 4. Running Assessments Moderation styles –On request (recommended) –Highs and Lows –Unexpected –Random Sample (recommended) Understand the process –Its student marking –Can’t get a ‘correct’ grade –Accept the average and the learning

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Challenges Fully understanding the potential and where it can be used Designing assessments to suit subjects Supporting good pedagogical use Time consuming to create good assessments Teaching students about effective feedback and reflection Research into best practice

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Common Mistakes Not understanding it Poor criteria Overloading students Obsessive moderating

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Common Mistakes Not using the preview Changing questions and criteria Changing dates back and forth Changing enrolments

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Benefits Re-usable resource – shareable good practice Moved from paper based and ad hoc systems Promoting really deep learning –Comprehension, application, synthesis, evaluation Students learn ‘soft skills’ –Giving effective feedback, analysing, criticising Students gain learning skills –Assessment criteria, marking, answering questions Students can place their work –See work better and worse than their own, monitor their own learning

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Some Ideas… First drafts Review resources Portfolio submission Video Past Papers Research…

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Conclusions Experiences gained using the system for 2 years Flexible, robust and expandable pedagogy Challenge in creating challenging assessments Benefit from experience of moderating –Not always easy –May not get right first time Inspired, motivated, ideas forming?

HANNAH WHALEY - LIBRARY AND LEARNING CENTRE, UNIVERSITY OF DUNDEE Contact Hannah Whaley, University of Dundee, Scotland