Using a VLE for Efficient and Effective Feedback Nick Lund Manchester Metropolitan University

Introduction Knight and Yorke (2003) – traditional assessment has more to do with accountability and quality control than with providing feedback for learning. Knight and Yorke (2003) – traditional assessment has more to do with accountability and quality control than with providing feedback for learning. Falchikov (2005) – similarly notes the conflicting demands on assessment. She also points out “there is little other than custom and familiarity to suggest traditional methods have worked ‘well’.” Further she claims it produces passive learners with lower levels of cognitive activities. Falchikov (2005) – similarly notes the conflicting demands on assessment. She also points out “there is little other than custom and familiarity to suggest traditional methods have worked ‘well’.” Further she claims it produces passive learners with lower levels of cognitive activities.

Introduction Bloxham and Boyd (2007) – ‘feedback is the most important aspect of the assessment process for raising achievement, yet currently students express considerable dissatisfaction with much feedback’ Bloxham and Boyd (2007) – ‘feedback is the most important aspect of the assessment process for raising achievement, yet currently students express considerable dissatisfaction with much feedback’ Hulme and Forshaw (2009) - traditional written feedback is often not the most effective for students or efficient for staff. Hulme and Forshaw (2009) - traditional written feedback is often not the most effective for students or efficient for staff. Heinrich et al (2009) – e-tools are used to increase efficiency of marking (with time freed being invested in quality improvements). Heinrich et al (2009) – e-tools are used to increase efficiency of marking (with time freed being invested in quality improvements).

Background HEA Psychology Network workshop HEA Psychology Network workshop The Double EE’s: Giving students feedback effectively and efficiently The Double EE’s: Giving students feedback effectively and efficiently Details at Details at Phil Denton’s electronic feedback software Phil Denton’s electronic feedback software Discussed at the conference Discussed at the conference Used in Psychology at MMU Cheshire for 5 years Used in Psychology at MMU Cheshire for 5 years

Denton's electronic feedback System for producing detailed feedback System for producing detailed feedback Can use different types of comment: Can use different types of comment: general comments (to whole class) general comments (to whole class) standard comments (to a proportion of class) standard comments (to a proportion of class) personal comments (to single person) personal comments (to single person) criterion comments (performance on each assessment criterion) criterion comments (performance on each assessment criterion) All general, standard and criterion comments input to file & then available for use. All general, standard and criterion comments input to file & then available for use. For details see For details see ck_Software_Denton.ppt

Denton's electronic feedback

Problems of software Those of us who use it find the system to be an excellent tool. However there are some problems Those of us who use it find the system to be an excellent tool. However there are some problems Complexity Complexity Some staff find it too complex to set up and to use Some staff find it too complex to set up and to use Set up time Set up time For learning to use the system For learning to use the system For establishing comment bank for grades, specific criteria For establishing comment bank for grades, specific criteria Support Support Produced by an individual not a software company Produced by an individual not a software company Phil Denton always tries to overcome problems but he is not paid Phil Denton always tries to overcome problems but he is not paid Reason this ‘project’ came about was some conflict in the set up on my computer last year Reason this ‘project’ came about was some conflict in the set up on my computer last year

Simple system using WebCT Adapted for an experimental report assignment. Adapted for an experimental report assignment. Took all the descriptors from the electronic marking system & made them available on WebCT. Took all the descriptors from the electronic marking system & made them available on WebCT. Students could see all descriptors of all mark bands and weighting for every part of the report before and after submission. Students could see all descriptors of all mark bands and weighting for every part of the report before and after submission. Feedback consisted of a brief personal comment followed by a mark for each section of the report. Feedback consisted of a brief personal comment followed by a mark for each section of the report. Students told to compare their marks with grade comments and to look at comments of grade/s above. Students told to compare their marks with grade comments and to look at comments of grade/s above. Invited to discuss/challenge marks in individual tutorial slots Invited to discuss/challenge marks in individual tutorial slots

Example of feedback Overall this is a very good report with an excellent method and results section. Your abstract was accurate but not concise, try to keep it brief. The introduction was good but was a little sketchy on the face inversion effect (one of the IVs). Overall this is a very good report with an excellent method and results section. Your abstract was accurate but not concise, try to keep it brief. The introduction was good but was a little sketchy on the face inversion effect (one of the IVs). AbstractIntroductionMethodResultsDiscussionRefsTOTAL

Extract from WebCT information Introduction (20%) Introduction (20%) 0There is no introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment. 0There is no introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment. 20There is only a very superficial introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment. 20There is only a very superficial introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment. 30There is only a superficial introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment. 30There is only a superficial introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment. 40There is a fair introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment. 40There is a fair introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment.

Evaluation Three perspectives Three perspectives Staff Staff External examiner External examiner Students Students Staff – quick and efficient. Staff – quick and efficient. EE – liked the amount of information given to students and the transparency of the system. Made special mention of system in the Examinations Board EE – liked the amount of information given to students and the transparency of the system. Made special mention of system in the Examinations Board

Evaluation Students – 44 on unit Students – 44 on unit Sources of evidence Sources of evidence Questionnaire given after the feedback Questionnaire given after the feedback WebCT records WebCT records Use of consultancy with unit leader Use of consultancy with unit leader

Student questionnaire - results 31 from 44 responded. 31 from 44 responded. 24 favourable 24 favourable System more informative, encouraged them to review work more, more helpful, more transparent. System more informative, encouraged them to review work more, more helpful, more transparent. 7 not entirely favourable 7 not entirely favourable Liked information but found system impersonal Liked information but found system impersonal

Student evaluation WebCT access WebCT access All 39 who collected the assignment viewed the relevant WebCT pages within three days. All 39 who collected the assignment viewed the relevant WebCT pages within three days. Use of consultancy Use of consultancy 15 used the opportunity to discuss their marks 15 used the opportunity to discuss their marks 6 challenged the marking of at least one section 6 challenged the marking of at least one section

Discussion Evidence suggests feedback was Evidence suggests feedback was Efficient – took less staff time than writing long comments Efficient – took less staff time than writing long comments Effective – students understood feedback and were able to act on it Effective – students understood feedback and were able to act on it Engaging – it required active participation. Allowed students to make their own judgements about their work & challenge marks Engaging – it required active participation. Allowed students to make their own judgements about their work & challenge marks Transparent – everything used in marking was available to students & it allowed them to study all mark bands Transparent – everything used in marking was available to students & it allowed them to study all mark bands

Questions Should we encourage students to challenge marks? Should we encourage students to challenge marks? How can it be adapted for other types of assignments? How can it be adapted for other types of assignments? Does it have any advantages over Denton’s more sophisticated system of feedback? Does it have any advantages over Denton’s more sophisticated system of feedback?

References Bloxham, S & Boyd, P (2007). Developing effective assessment in Higher Education. Maidenhead: OU Press Bloxham, S & Boyd, P (2007). Developing effective assessment in Higher Education. Maidenhead: OU Press Denton, P., Roberts, M., Madden, J. & Rowe, P. (2008). Students’ response to traditional and computer assisted formative feedback: a comparative case study. British Journal of Educational Technology, 39, Denton, P., Roberts, M., Madden, J. & Rowe, P. (2008). Students’ response to traditional and computer assisted formative feedback: a comparative case study. British Journal of Educational Technology, 39, Falchicov, N (2005). Improving assessment through student involvement. Abingdon: RoutledgePalmer Falchicov, N (2005). Improving assessment through student involvement. Abingdon: RoutledgePalmer Heinrich, E., Milne, J., Ramsay, A., & Morrison, D. (2009). Recommendations for the use of e-tools for improvements around assignment marking quality. Assessment & Evaluation in Higher Education, 34, Heinrich, E., Milne, J., Ramsay, A., & Morrison, D. (2009). Recommendations for the use of e-tools for improvements around assignment marking quality. Assessment & Evaluation in Higher Education, 34, Hulme, J. & Forshaw, M. (2009). Effectiveness of feedback provision for undergraduate students. Psychology Learning and Teaching, 8, Hulme, J. & Forshaw, M. (2009). Effectiveness of feedback provision for undergraduate students. Psychology Learning and Teaching, 8, Knight, P.T. & Yorke, M. (2003). Assessment, Learning and Employability. Maidenhead: OU Press Knight, P.T. & Yorke, M. (2003). Assessment, Learning and Employability. Maidenhead: OU Press