1. The Evolution of WebPA, an online peer moderated marking system, at Loughborough Melanie Bates, Learning Technology Co- ordinator Loughborough University

2. Background The value of group work Group marks and the need for deriving individual marks from them Peer assessment as a mechanism of rating individual contribution in group work

3. WebPA: the evolution 1 Paper Excel WebPA PASS Paper-based peer assessment A computerised peer assessment system PASS (Online Peer Assessment System) WebPA (Web-based Self and Peer Assessment)

4. Users Developer Lead Academic engCETL WebPA WebPA: the evolution 2

5. Rationale for implementation of WebPA In terms of the Institutional policy: There are the requirements of accrediting bodies which clearly state that students should have had ‘experience of working in groups or teams’. An internal report (Blease, 2006) states that group work takes place in over 500 modules in all departments at Loughborough. This practice clearly suggests the need for a mechanism of deriving individual marks in group work. In terms of online peer assessment and its benefits: Widely recognised benefits of peer assessment The ability to deal with large data and automate data collection and analysis Confidentiality and flexibility Marking not prone to human errors

6. The evaluation of WebPA at Loughborough The following are reported as strengths of the system in internal investigations (Pond et al., 2006; Robinson, 2006; Willmot & Crawford, 2004; 2005): Students perceive the process to be fair Flexible and easy to use Confidentiality The system makes a complicated process very simple to implement Setting up forms and groups takes only a few minutes The results of the assessment are instantly available regardless of the group size A variety of reports can be easily created based on various mark sheets according to parameters. Ease of tracking or monitoring Data can be easily downloaded into Excel Student satisfaction The following are reported as issues with peer assessment (Pond et al., 2006; Robinson, 2006; Willmot & Crawford, 2004; 2005): Possible group collusion Non-participation Full marks in self-assessment Extreme subjectivity

7. The WebPA project The WebPA project was launched in October, 2006. JISC funded, 2 year, research and development project. The project partners - engCETL (Engineering Centre for Excellence - engCETL (Engineering Centre for Excellence in Teaching and Learning), in Teaching and Learning), Loughborough University Loughborough University - The University of Hull - The University of Hull - Higher Education Academy Subject Centres in - Higher Education Academy Subject Centres in Engineering and Physical Sciences Engineering and Physical Sciences

8. Challenges Within the Institution: Increasing the number of users Training issues (students and academics) Produce pedagogic models Outside the institution: Making the system institution-independent Identifying and persuading other institutions to implement WebPA Building up the community of users Support issues - technical support - support for embedding the system in teaching - support for embedding the system in teaching Training issues (students and academics)

9. Future and further software development An improved version of WebPA will become open-source. There is a need for: - building up a community of users - building up a community of users - an effective communication system between users and the project team - an effective communication system between users and the project team - a system using which the users feed into further system development. - a system using which the users feed into further system development. There is a need for sustainability strategies for the software to stay alive after the life of the project. The features WebPA could integrate include the following: Finding a way of making the system institution-independent a demo version for potential users Facility for students to justify why they give the ratings they give to their peers Provide more examples of assessment criteria, scoring ranges etc.

10. References Blease, D. (2006). The organisation and assessment of group work at Loughborough university. Unpublished internal report. Loughborough: Loughborough University. Falchikov, N. (1991). Group process analysis: Self and peer assessment of working together in a group. Self and Peer Assessment,, 63 15-28. Pond, K., Coates, D., & Palermo, O. A. (2006). Student perceptions of peer review marking of team projects Robinson, C. L. (2006). Self and peer assessment in group work. The SEFI MWG Seminar, Kongsberg. Sommervell, H. (1993). Issues in assessment, enterprise and higher education: The case for self-, peer and collaborative assessment. Assessment and Evaluation in Higher Education, 18(3), 221-233. Willmott, P., & Crawford, A. (2004). Online peer assessed marking of team projects. International Conference on Engineering Education, Gainsville, Florida. Willmott, P., & Crawford, A. (2005). Validating the assessment of individuals within undergraduate teams. International Conference on Engineering Education, Gliwice, Poland. Contacts The WebPA project: http://webpaproject/index.php?section=1 http://webpaproject/index.php?section=1 Mailing list: http://www.jiscmail.ac.uk/lists/webpa.html http://www.jiscmail.ac.uk/lists/webpa.html