Presentation on theme: "Ipsative assessment: comparison with past performance"— Presentation transcript:
1Ipsative assessment: comparison with past performance Dr Gwyneth Hughes
2SummaryDefines ipsative assessment and it potential benefits Builds on findings of research on ipsative feedback for distance learners funded by CDEI am senior lecturer at IoE where I do research on learning and teaching in HE and I lead a large JISC funded project Assessment careers Technology enhanced assessment.An initial project looking at types of feedback learners received on a DL MA also CDE funded. This presentation is about a second phase of implementing ipsative feedback on that MA.First a bit more about ipsative assessment
3Ipsative assessment as an alternative to criteria-referenced assessment
4Personal bestAssessment by comparison with yourself rather than in comparison to others or external criteria and standards,–like a personal best in athletics.Assessing progress rather than outcomes.
5What is ipsative assessment ? An ipsative assessment compares current performance with a previous performance.Ipsative feedback on progressIpsative grades awarded for progressSelf-referential assessment not necessarily a self-assessmentHughes, G Aiming for Personal Best: a Case for Introducing Ipsative Assessment in Higher Education Studies in Higher Education 36 (3):Ipsative assessment is not necessarily a self-assessment-it could be –but ipsative feedback and/or grades for progress could be provided by the assessor or peers or self. I term it a self-referential assessment rather than necessarily a self-assessment.
6Possible benefits of ipsative assessment Improves self-esteem and confidence by rewarding progress (particularly for those who do not achieve high grades)It might motivate learners to act on feedbackGives tutor and students a longitudinal view of assessmentAssessing progress makes more sense if a long-term view is taken in multiple linked assessments over time rather than individual modular assessmentsParticularly for distance learners who are very reliant on written feedback and do not have an equivalent for feedback in class
7MA in Applied Educational Leadership and Management Phase 1 of study in showed that ipsative feedback was rareHowever, both students and tutors felt that ipsative feedback would be beneficialIpsative grades were perceived to be problematic and so not developed for the next phase of the study-focus on ipsative feedback.Funded study by CDE for a distance learning programme-Action research – two phases.This distance programme which is delivered jointly by IOE and London University International programmes.
8Ipsative cumulative scheme for 3 sequential assessments Students in the previous study find feedback that indicates what to do to improve or feedforward very helpful but how do they know if they have successfully acted on feedforward (apart from a better mark perhaps?) In this ipsative scheme Feed forward 2 can build on feed forward 1 if remedial action has not been taken.
9Excerpt from the new form Please indicate what feedback you were given for your draft of this assignment, in terms of how you could improve (ifapplicable):Please indicate what feedback you were given, for your last assignment, in terms of how you could improve:Please comment on the extent to which you feel you have responded to feedback:
10MA in Applied Educational Leadership and Management Students submit a draft pieces of work for feedbackStudents fill in a new Assignment Submission form when they submit their work for assessment identifying actions arising from previous feedback (self assessment)Students received ipsative feedback from tutors on how well they have addressed the feedback (self-referential assessment)Students’ previous assignments and feedback were made available in the VLE for consultation.This programme was used to try out ipsative feedback. A meeting with the tutors and programme leader resulted in an agreed intervention.3. Means that tutors do not necessarily have to compare the new piece of work with the previous one, they can use the student self-assessment as a basis for ipsative feedback.However, previous assignments and feedback needed to be easily accessible (if students did not fill in the self-assessment) VLE.
11Evaluation methodsThe written feedback given to 28 students was analysed13 students were interviewed about the usefulness of feedback including the ipsative feedbackThe 3 main tutors were interviewed about the new approach to giving feedback (2 others were part-time and declined to be interviewed)28 students international about half non native English 17 female and 11 male.
12Excerpt from the new form Please indicate what feedback you were given, for your draft of this assignment, in terms of how you could improve (if applicable): - Methodology:clearer statement on exact nature of researchTo identify and use a clearer narrative thread (student 22)Please comment on the extent to which you feel you have responded to feedback: -In the methodology section I have added more detail and have given further thought to sampling and validity of data-tried to use more signposts to direct the reader through the narrative more clearly (student 22)
13Analysis of use of the form and written feedback Not all students used the form – some were unable to access itSome who filled in the form were at the box-ticking end of the spectrum e.g.I have tried to act on all the feedback received (student 11)This is to be expected for an introduction of a new procedure and students will need time to get used it and also to realise that there are benefits to completing the form in that they will get further feedback.
14Ipsative Feedback to students from tutors ‘It is clear that you took careful note of the advice you received on the draft and the previous assignment. You have worked hard to improve the narrative flow and have taken steps to develop your critical engagement with the sources.’ (Tutor A) ‘It is a pity you did not have the chance to revise your assignment on the basis of the feedback provided earlier, as this would probably result in a better grade.’ (Tutor C)There was evidence that tutors provided ipsative feedback –in contrast to the previous study where this was non-existant.
15Analysis of use of the form and written feedback Some tutors used the form to provide ipsative feedbackOne tutor gave ipsative feedback when students did not use the formThe form is not essential for comparing with previous performances as some tutors appeared to have used the records in the VLE to look back to previous comments they had written.Was the form useful?This suggests that having a process such as this helps tutors get a sense of previous feedback and performance but easy access to previous feedback and assessments is needed also to supplement the form –in the VLE.
16Reaction to the new approach to feedback – the students The form helped students to reflect on how and to what extent they had responded to tutor’s comments‘It [the form] allowed me to show how I had attempted to improve on my last assignment and also encouraged me to reflect more deeply on my submission.’ (Student 22)Contributing to motivation and confidence building‘[the comment] gave me confidence and made me feel proud of my work …. It also made me aware of the changes that I made that improved my work so I will know what to do next time.’ (Student 16)
17Reaction to the new approach to feedback – the students ‘My tutor gave particularly good feedback after my draft assignment. It was easy to understand and follow and was very specific so I could see exactly what I needed to improve and how. I think her feedback made a big difference to my final mark.’ (Student 16) ‘…the tutor reminded me of not having addressed an issue raised about my first draft. I did the tutor and highlighted an extract from the final essay where I had covered the suggestion made. I believe the tutor had missed that point and the comment was then removed from my final feedback. I was told, however, that this would not result in a revised mark.’ (Student 13)Focus on short-term grade improvement
18Reaction to the new approach to feedback – the tutors The form helped tutors write feedback‘I was already on the journey to that approach. When providing feedback I tended to approach it in my normal way and then made sure that I adapted things to meet the requirements of the new front sheet.’ (Tutor A)Concern about raising expectations‘I could see from the comments that some students expected their revised draft to lead to an A (grade), and it was subsequently disappointing for these students to gain a sense of progress without an A.’ (Tutor C)Tutors agreed that the form helped them and students and tutors A and C used the form but tutor B less so.Concerned that ipsative feedback might raise expectations.
19The benefits of ipsative feedback The form helped some students reflect on feedbackA structured approach such as including a submission form might help both students and tutors take a longer-term view of assessmentIpsative feedback can be motivational for distance learners
20The challenges for implementing ipsative feedback Ipsative feedback is new and needs time to be acceptedStudents may expect instant higher gradesScaling up needs the practice to be embedded across a programme and tutors may need development timeTutors are not accustomed to viewing each others feedback and records of previous feedback are not routinely stored in VLEsPlans to scale it up and continue on the programme
21Scaling upFunding for a institution-wide project on assessment of £200,000
22ReferencesBroadfoot, P Education, Assessment and Society. Buckingham: Open University Press.Carless, D Differing perceptions in the feedback process. Studies in Higher Education. 3, no. 2:Hughes, G. (2011) Aiming for Personal Best: a Case for Introducing Ipsative Assessment in Higher Education Studies in Higher Education 36 (3): 353 – 367Hughes, G. Wood, E. & Okumoto, K Use of ipsative assessment in distance learning Centre for Distance Education Report, University of London. Nicol, D. & Macfarlane,-Dick, D Formative assessment and self regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31 no. 2:Institute of EducationUniversity of London20 Bedford WayLondon WC1H 0ALTel +44 (0)Fax +44 (0)Web