Presentation on theme: "Peer review with Aropa in Computing Science"— Presentation transcript:
1 Peer review with Aropa in Computing Science Dr Simon Rogers
2 The course: PSI Professional Skills and Issues: Assignment: L4 and Msc students (>100)Compulsory: Required for BCS accreditationNon-technical: Ethics, IPR, Data protection etcNot the most popular L4 course……Assignment:1,500 word essay.2 stage peer review process.Final mark (from academic) weighted by `engagement factor’
3 AropäPeer review management software developed by John Hamer (Auckland & SCS) and Helen Purchase (SCS).Handles everything:Submission (word/pdf, txt)Assignment to reviewsReview rubric (design your own)Returning reviews to reviewersReviewing of reviewsVery easy to use. Local support available.Willing to expand system to meet new requirements.
4 Peer review process Trial exercise Assessed exercise 500 word essay 2 reviewsNot assessed (almost 100% response)Assessed exerciseStudents submit draft essay on one of 3 topics (0.7).Assigned 3 reviewers from same topic.Reviewers assess work (numerical ratings and text justification) (0.05 x 3).Reviews returned to authors who can use them to improve essay.Each student also reviews 3 other reviews (0.05 x 3).Final marking done by academics, weighted by factors (X)
5 Pros & ConsPros:Aropa very easy to use – assigning reviews, handling deadlines, monitoring submissions etc.High level of engagement (almost all students completed everything).Massive quantity of feedback (not always high quality)Cons:Between cohort tension (L4 v MsC).`Threat’ of plagiarism (by reviewers).Lack of peer review experience – poor quality of feedback (first use of peer review for 85% of class).Anonymity (.doc files).
6 Other feedbackOnly 20% of students would have been happy to be formally assessed by peers.Of those who thought the exercise improved their essay (42%!), 71% felt that the most helpful bit was receiving feedback.