Presentation on theme: "Assessment and Feedback Principles Theory and Practice"— Presentation transcript:
1Assessment and Feedback Principles Theory and Practice David NicolVisiting Professor in Centre for Higher Education PracticeUniversity of UlsterEmeritus Professor: University of StrathclydeWebsite: reap.ac.ukFestival of Innovative Practice 18 June 2013Magee Campus
2Plan Background: Assessment and Feedback principles Re-engineering Assessment Practices (REAP) project: University of StrathclydeUlster’s principles: research basis and examplesDynamics of ImplementationJISC-funded Viewpoints project (University of Ulster): the A&F design toolkitA principles-based discourse approach to institutional change.
3REAP: Re-engineering Assessment Practices project Scottish Funding Council (£1m):Goals: learning quality and teaching efficiencies3 HEIs (Strathclyde, Glasgow Caledonian Business School, Glasgow University)Large 1st year classes ( students)Range of disciplines (19 modules ~6000 students)Many technologies: online tests, simulations, discussion boards, e-portfolios, e-voting, peer/feedback software, VLE, online-offlineOutputs: new practices, institutional embeddingAssessment for learner self-regulation
4Barriers to transformational change in teaching and learning institution-wide Lack of shared educational frame of reference to guide innovations in practiceDifficulty defining good educational practiceIsolation of academics from educational researchDisciplinary differences in teaching and learningWeak links between local practices and educational policies and strategiesChallenge of getting multi-stakeholder buy-inSee Nicol & Draper (2009)Interest in FASummative A can undermine the value of fa e.g.
5Background (1)Gibbs, G. & Simpson, C (2004)Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, 1, 3-31.See:Formative Assessment in Science Teaching (FAST) project at:
6Gibbs and Simpson (2004) Assessment tasks [Conditions 1-4] Capture enough study time (in and out of class)Are spread out evenly across timeline of studyLead to productive activity (deep vs surface)Communicate clear and high expectationsi.e concern here is with ‘steers’ about how much work to do and the quality of what is done
7Background (2) Literature Review Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1),Nicol, D & Milligan, C. (2006), Rethinking technology-supported assessment practices in relation to the seven principles of good feedback practice. In C. Bryan & K. Clegg, Innovative assessment in higher education, Routledge.BackgroundStudent Enhanced Learning through Effective Feedback [SENLEF] project funded by HE Academy
8Key aspects of research Students are always engaged in self-assessment/ self-regulation of their own learning (Winne, 2005).The act of using teacher feedback implies that self-assessment must be present (Black and Wiliam, 1998).Feedback in HE is being reduced so how are students still learning.The question is: how can we scaffold students’ learning so they become better at self-regulation (Lajoie, 2005)
9Rethinking assessment and feedback 1. Consider self and peers as much as the teacher as sources of assessment and feedbackTap into different qualities than teacher can provideProvides considerable learning benefitsBetter use of teacher time2. Focus on every step of the cycle:Understanding the task criteria (Sadler, 1983)Applying what is learned in action3. Not just written feedback:Also computer, dialogue, formal and informal
10Seven principles of good A&F practice Clarify what good performance is (goals, criteria, standards).Facilitate the development of reflection and self-assessment in learningDeliver high quality feedback to students: that enables them to self-correctEncourage peer and student-teacher and peer dialogue around learningEncourage positive motivational beliefs & self esteem through assessmentProvide opportunities to act on feedbackProvide information to teachers that can be used to help shape their teachingNicol and Macfarlane-Dick (2006)
11Two meta-principlesMeta-principle 1: time and effort on task (in and out of class (engagement) i.e. steers on how much work to do and when – Gibbs and Simpson 4 conditionsMeta-principle 2: developing learner self-regulation (empowerment/self-regulation) i.e steers to encourage ownership of learning – the Nicol and Macfarlane-Dick 7 principles.
12Original Principles of Assessment and Feedback EngagementAssessment tasks capture sufficient study time (in and out of class)Are spread out evenly across timeline of studyLead to productive activity (deep learning)Communicate clear and high expectationsGibbs and Simpson (2004)EmpowermentClarify what good performance is (goals, criteria, standards).Facilitate the development of reflection and self-assessment in learningDeliver high quality feedback to students: that enables them to self-correctEncourage peer and student-teacher and peer dialogue around learningEncourage positive motivational beliefs & self esteem through assessmentProvide opportunities to act on feedbackProvide information to teachers that can be used to help shape their teachingNicol and Macfarlane-Dick (2006)
14Implementation Local redesigns 19 module redesigns: principles guided implementationsSuccess: learning gains in exams (11 out of 19 modules) improved quality w/o extra costs, high levels of student satisfaction, efficiency gains.Institutional developmentsDep. Principal T&L set up working group: new A&F policy (Strathclyde) grounded in principles agreed by SenatePrinciples embedded in QA proceduresMany departmental/university initiatives referencing REAP and using the principlesSharing of good practice using principles as referenceWidespread use of principles:nationally & internationally
15Barriers to transformational change in teaching and learning institution-wide Lack of shared educational frame of reference to guide innovations in practiceDifficulty defining good educational practiceIsolation of academics from educational researchDisciplinary differences in teaching and learningWeak links between local practices and educational policies and strategiesChallenge of getting multi-stakeholder buy-inSee Nicol & Draper (2009)Interest in FASummative A can undermine the value of fa e.g.
16Thinking behind REAPThe vision – Assessment and Feedback should support the development of learner self-regulationA set of assessment & feedback principles based on research – to operationalise this visionPrinciples as translation device – to make the research accessible to busy academicsImplementation strategy: REAP team supported course teams engaged in redesigns (i.e. application of A&F principles in modules and programmes and technologies)Evaluation of innovations to build institutional commitmentInterest in FASummative A can undermine the value of fa e.g.
17Addressing the barriers to change Lack of shared frame of reference to guide innovations: the big idea and principlesDifficulty defining good educational practiceIsolation of academics from educational research: simplifying research into some key ideasDisciplinary differences – tight-loose structure of principlesWeak links between local practices and educational policies and strategies – single set of educational ideas, tying each course design to policy frameworkChallenge of getting multi-stakeholder buy-inInterest in FASummative A can undermine the value of fa e.g.
18Good formative assessment and feedback practices should: Help clarify what good performance is (goals, criteria, standards)Encourage time an effort on challenging learning tasksDeliver high quality feedback information that helps learners self-correctProvide opportunities to act on feedbackEnsure that summative assessment supports formative learning processesEncourage interaction and dialogue around learning (peer, t-st)Facilitate the development of self-assessment and reflection in learningGive choice in the topic, method, criteria, weighting or timing of assessments.Involve students in decision making about assessment policy and practiceSupport the development of learning groups and communitiesEncourage positive motivational beliefs and self-esteemProvide information to teachers that can be used to help shape teaching
19Principles of assessment and feedback for learning Clarify good performance Help to clarify, from the early stages of a programme, what good performance means (goals, criteria, standards);Encourage time and effort on task Encourage 'time and effort' on challenging learning tasks, which recognise the importance of learning from the tasks, not just demonstrating learning through tasks;Deliver timely high quality feedback Deliver timely learner-related feedback information that helps students to self-correct and communicates clear, high, expectations and professionalism;Provide opportunities to act on feedback Provide opportunities for students to act on feedback and close any gap between current and desired performance through complementary and integrated curriculum design and pedagogic practice;Encourage positive motivational beliefs Ensure that all assessment has a beneficial, constructive, impact on student learning, encouraging positive motivational beliefs, confidence and self-esteem;Develop self-assessment and reflection Facilitate the development of self- and peer-assessment skills and reflection on learning, to enable students to progressively take more responsibility for their own learning, and to inspire a lifelong capacity to learn;Encourage interaction and dialogue Encourage interaction and dialogue around learning and professional practice (student-student, lecturer-student and lecturer-lecturer) including supporting the development of student learning groups and peer learning communities.
20Good assessment and feedback practice should Clarify what good performance is (goals, criteria, standards).Teacher provides criteria before task insufficientTeachers provide a range of model answersStudents derive criteria from exemplars before the taskStudents create criteria for new taskStudents compare own work against standards descriptorsStudents rank peer work in order of quality and discussStudents must develop a concept of quality
21Good assessment and feedback should: Encourage time and effort on taskReduce size and increase number of learning tasksGive small number of marks for completionProvide tasks as out-of class homework activities that link to in-class activities (e.g. MCQs)Group tasks are important here (e.g. EVS)Create structures that promote ownership and effort
22Good assessment and feedback practice should: Delver timely high quality feedback: that helps students to self-correctStudents request feedback they wish (cover sheet)Feedback on processes and skills – maximise transferDon’t give feedback – point to resources where answer/issue can be elaboratedFeedback on students’ self-assessments and/or peer reviewsCalibrate students’ ability to make evaluative judgements(see Hattie and Timperley, 2007)
23Good assessment and feedback should: Provide opportunities to act on (respond to) feedbackStudents respond to teacher feedback in writingSequence assignments so feedback is usedProvide feedback as action pointsStudents say how used feedback when submit next assignment [proforma]Reward use of feedback in a new task (Gunn, 2010)Ensures feedback is processed and leads to knowledge building.Key principle if your goal is to enhance NSS results
24Good assessment and feedback should: Encourage positive motivational beliefs and self-esteemEncourage climate of respect and accountabilitySequence tasks for progressive level of difficultyUse real life (authentic) learning tasksGive learners choice in topic, methods, criteriaImplement other principles (group working etc)Balance structure with increasing learner responsibilityAbout giving students a sense of control over their learning
25Good assessment and feedback should: Develop self-assessment (and peer assessment skills) and reflection.Students identify what is strong and weak when they hand in an assignmentProvide an abstract with an essay (reflection)Implement peer review where students’ comment on each other’s work (see Nicol, 2013: Nicol et al, 2013)then review their own workGive students practice in making evaluative judgements
26Good assessment and feedback should: Encourage interaction and dialogue around learning (peer, student-lecturer, lecturer-lecturer)Electronic voting methods: polling and peer discussionDiscussions of feedback in tutorialsPeer review using pairs and groupsCollaborative assignmentsWrap dialogue around all assessment processes (Nicol, 2010)Attenuates teachers’ voice and strengthens students’ voice (shifts responsibility towards students)
27Key focus of current research Developing students’ capacity to make evaluative judgements about own and other’s work (Boud, 2013: Sadler, 2013: Nicol, 2013).Making judgements has not received enough attention in HE (Cowan, 2010)Evaluative capability underpins all graduate attributes (Nicol, 2009)Assessment in HE does not replicate what happens in professional practicePeer review – producing reviews is qualitatively different from receiving reviews (Nicol et al, 2013)
29Bigger picture: learning from REAP Principles-based approach has great potentialPeople talked about and used the principles – an emerging discourseWidespread take-up of principles in UK and internationally – principles legacy more enduring than REAP innovations themselvesQuestion: how to take things further forward so that the ideas and discourse spread more rapidly and deeply across the whole institution.Interest in FASummative A can undermine the value of fa e.g.
30The JISC-funded Viewpoints project Took the REAP assessment and feedback principles and put them ‘in the hands of the user’The principles as a workshop tookit - prompts to think through an assessment and feedback design.The principles as ‘social objects’ to seed and sustain a new educational discourse.Interest in FASummative A can undermine the value of fa e.g.
31Assessment and Feedback Principles REAP (http://www.reap.ac.uk/)Clarify good performance.Encourage time and effort on task.Deliver high quality feedback.Provide opportunities to act on feedback.Encourage interaction and dialogue.Develop self-assessment and reflection.Give assessment choice.Encourage positive motivational beliefs.Inform and shape your teaching.*Implementation ideas for each principle on back of cards.
33The Artefacts:Timeline worksheet & Principles Click to add notes
34Discussing the objective Course team agree the objective for their session and write it at the top of the module worksheet.Click to add notes
35Reading the front of the cards The team read the principles on the front of the cards, choosing ones appropriate to their objective.Click to add notes
36Mapping the cards to the learner timeline The team take their selected cards and map them to the appropriate point on the timeline (e.g. at the induction phase, during first few weeks of course)Click to add notes
37Reading examples on cards Workshop participants turn the cards over and read the examples/ideas on the back.Click to add notes
38Choosing relevant examples The team might select or adapt any examples that would fit with their course objective and their teaching practice.Click to add notes
39Adding in own ideas/comments The team produce their own examples, ideas, comments and use post-it notes develop their own assessment and feedback design.Click to add notes
40Workshop outputsForm basis of an Assessment & Feedback (or other theme) strategyProvide reference for future course team discussions (planning resource)Key information will be transcribed into table (Word or other format)
41Users/teams define the outputs These can be orderly or messy – it’s up to you
42Features of workshop discourse Agenda set by participants (course teams)Peer interaction, discussion and sharing of ideasConcepts/examples on cards inform discussion (i.e. It is research informed)Principles call on each other in useConstruct & co-construct meaning – not about tellingLearner-focused not content focusedProblem-focused yet exploratory and creativeStructured by a timelineSocially engaging- like a board gameIdeas extend beyond the workshop
43Evaluation of REAP, Viewpoints and of other HE institutions that have used educational principles Model of change extrapolatedPrinciples-based discourse modelDiscourse shapes how people think about things and therefore how they act; and how people act and think about things shape their discourses. (Marshak and Grant, 2011)Interest in FASummative A can undermine the value of fa e.g.
44Principles-based discourse model for change Way of addressing change in complex organisationsDiscourse is not just about conversations but also about written texts, official documents, s, memos, stories, narratives, metaphors, slogans etc.Focuses on meaning-making and the social construction of realityAbout back-stage processes as much as front-stage eventsFocus is discourse as a driver for change not just as a symptomThere are already discourses about assessment and feedback but are these educationally informed?Interest in FASummative A can undermine the value of fa e.g.
46Extending the reach: many contexts of application Many reference points and opportunities for discussion in other forums and contextsFacilitated by support services (LLL, TEL, Acad Practice Unit), staff induction, staff development, through revalidation & course review processes, and devolved to departments to meet their needsLinks made when discussing other agendas – graduate attributes, employability etc.Through student initiativesInterest in FASummative A can undermine the value of fa e.g.
47Embedding the discourse in documents and texts In educational policy (Strathclyde, Ulster)In publicity materials for students and staffIn reference documents – QAA procedures, audit, course approval and review, student evaluations, external examiners briefing docs, etc.As framework to comment on other educational innovations – linking languageInterest in FASummative A can undermine the value of fa e.g.
48Some of my Publications Nicol, D., Thomson, A. and Breslin, C (2013) Rethinking feedback practices in higher education: a peer review perspective, Assessment and Evaluation in Higher Education, contact DNNicol, D (2013), Resituating feedback from the reactive to the proactive. In D. Boud and L. Malloy (Eds) Effective Feedback in Higher and Professional Education: understanding it and doing it well, Routledge UKNicol, D (2011) Developing students’ ability to construct feedback, QAA Scotland, Enhancement Themes. Available on Scottish Enhancement Themes websiteNicol, D (2010) The foundation for graduate attributes: developing self-regulation through self and peer assessment, QAA Scotland, Enhancement Themes. Available at: Nicol, D (2010) From monologue to dialogue: improving written feedback in mass higher education, Assessment and Evaluation in Higher Education, 35:5,Nicol, D and Draper, S (2010), A blueprint for transformational organisational change in HE: REAP as a case study (see reap.ac.uk website)Nicol, D (2009), Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, ScotlandNicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3),Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1),Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2),See also for copies.