Presentation on theme: "Peer review with Aropa in Computing Science"— Presentation transcript:
1Peer review with Aropa in Computing Science Dr Simon Rogers
2The course: PSI Professional Skills and Issues: Assignment: L4 and Msc students (>100)Compulsory: Required for BCS accreditationNon-technical: Ethics, IPR, Data protection etcNot the most popular L4 course……Assignment:1,500 word essay.2 stage peer review process.Final mark (from academic) weighted by `engagement factor’
3AropäPeer review management software developed by John Hamer (Auckland & SCS) and Helen Purchase (SCS).Handles everything:Submission (word/pdf, txt)Assignment to reviewsReview rubric (design your own)Returning reviews to reviewersReviewing of reviewsVery easy to use. Local support available.Willing to expand system to meet new requirements.
4Peer review process Trial exercise Assessed exercise 500 word essay 2 reviewsNot assessed (almost 100% response)Assessed exerciseStudents submit draft essay on one of 3 topics (0.7).Assigned 3 reviewers from same topic.Reviewers assess work (numerical ratings and text justification) (0.05 x 3).Reviews returned to authors who can use them to improve essay.Each student also reviews 3 other reviews (0.05 x 3).Final marking done by academics, weighted by factors (X)
5Pros & ConsPros:Aropa very easy to use – assigning reviews, handling deadlines, monitoring submissions etc.High level of engagement (almost all students completed everything).Massive quantity of feedback (not always high quality)Cons:Between cohort tension (L4 v MsC).`Threat’ of plagiarism (by reviewers).Lack of peer review experience – poor quality of feedback (first use of peer review for 85% of class).Anonymity (.doc files).
6Other feedbackOnly 20% of students would have been happy to be formally assessed by peers.Of those who thought the exercise improved their essay (42%!), 71% felt that the most helpful bit was receiving feedback.