Presentation on theme: "Enhancing the Education Environment at Queen’s First Annual Conference of the Centre for Educational Development Queen’s University Belfast, 18-19 Sept."— Presentation transcript:
Enhancing the Education Environment at Queen’s First Annual Conference of the Centre for Educational Development Queen’s University Belfast, 18-19 Sept 2006 Enhancing Learning and Teaching: What Role Can Research Evidence Play? Dai Hounsell University of Edinburgh www.ed.ac.uk/etl
BACKGROUND AND INTRODUCTION FEvidence and Practice the lure of evidence-based and evidence-informed practices and policies evidence on teaching-learning and assessment practices –how far can we generalise, and about what? –the challenge of contingency F Feedback and Its Discontents pervasive evidence of variable feedback (e.g. National Student Survey, 2006; QAA Learning from Subject Review, 2003; Krause et al. 2005; Hounsell, 2003; Hounsell et al. 2005; Carless, 2006)
RESEARCH AND FEEDBACK TO STUDENTS F compelling evidence of the role of feedback and formative assessment in facilitating high-quality learning ( see e.g. Black et al. 2003; Nicol and MacFarlane- Dick, 2006) F evolving conceptions of feedback (Sadler, 1998) l what makes for effective feedback –knowledge of results –support and encouragement –grasp of what high-quality achievement entails l closing the loop (waxing and waning) l action taken to close the gap, between desired goal and actual performance
KEYNOTE FOCUS AND AIMS F research findings on guidance and feedback to students F draws on biosciences data from the ETL Project F aims to review: findings from 1 st round of data-gathering, and subsequent action by course team findings from 2 nd round of data-gathering, on impact of measures taken outcomes of subsequent analysis implications for evidence-informed efforts to enhance learning and teaching
Enhancing Teaching-Learning Environments in Undergraduate Courses (ETL Project) RESEARCH DESIGN F Aims to investigate ways of enhancing the quality of undergraduate learning and teaching, in a range of subject areas & settings F Samples and settings first- & final-year modules in three departments F Data-gathering student questionnaires and interviews with students & staff F ‘Enhancement’ focus collection, analysis & joint review of baseline data evidence-based collaborative initiatives
BIOSCIENCE STUDENTS’ PERCEPTIONS OF GUIDANCE AND FEEDBACK F The students’ overall perceptions of their courses were broadly positive across all of six bioscience course units surveyed F Their experiences of the provision of guidance and feedback on assessed work, however, were much more variable F In some units, students reported favourably; in others, there were significant student concerns
BIOSCIENCE STUDENTS’ PERCEPTIONS OF GUIDANCE AND FEEDBACK (Questionnaires) [insert figure 2]
STUDENTS’ CONCERNS ABOUT GUIDANCE AND FEEDBACK (Interviews) F Where guidance and feedback was a significant student concern, it could take various forms: l uncertainty about what staff expected from students in set [i.e. formally required] work l dissatisfaction with the variable quantity and helpfulness of feedback comments from staff l frustration with delays in receiving feedback l (in a small number of instances) uncertainty about the ground-rules for “buttonholing” tutors
STUDENTS’ CONCERNS ABOUT GUIDANCE AND FEEDBACK (Interviews) S5: I got 8 out of 20, and I've got nothing written on my [feedback] sheet at all. S3: Mine's the same. I got 10, and it's got no comments on it whatsoever. S5: And they tell you to do it in double-spacing, so they can write things in, but they never do. S3: I mean, if we're getting half marks, it must have a lot wrong with it.. [S5: Exactly.] But it's not telling us anything. --------------------------------------------------------------------- S1:Sometimes they say ‘Be more concise’ but then another time I thought ‘Well, I’ll try being more concise this time’ and actually I got less for doing that! So then the next time I thought ‘I’ll go back to my other way’ and it worked better! So it’s been confusing.
STUDENTS’ CONCERNS ABOUT GUIDANCE AND FEEDBACK (Interviews) S1:We write the thing, hand it in [S: Yeah] and we get it back with a few comments on … Mainly spelling mistakes. [Laughter][…] S3:It's postgrads [who mark the work], and it's quite, sometimes inconsistent. […] S2:— It's very inconsistent. [S: Yeah]. And also, I don't think that they are marked for us. They are marked for them. [..] I don't think they are writing in the margins so we will know not to do it again. They're writing it in the margins so they will remember that we've done it wrong when they add up the marks, I think. It isn't done as feedback.
TWO CASE STUDIES Case 1 - A Large First-year Course Unit Case 2 - A Small Final-year Honours Module
Case One A LARGE FIRST-YEAR COURSE UNIT F Over 600 students and 25+ staff in varied roles F 50% of overall grade from coursework, incl. a debate, a group poster, an advisory letter to a GP (the ‘pertussis enigma’ exercise) F findings from initial questionnaires and interviews: low questionnaire scores on clarity about assessment and feedback general concern about limitations of pre-assignment guidance and post-assignment feedback particular concern with the ‘pertussis enigma’ exercise
Case One A LARGE FIRST-YEAR COURSE UNIT F The collaborative initiative agreed with the course team to address the concerns identified: strengthened guidance to lab demonstrators about assignments and assessments (incl. the ‘pertussis enigma’ exercise) adoption of a structured marking and feedback proforma for the ‘pertussis enigma’ exercise
Case One: FINDINGS ON IMPACT F Pre-Collaborative Initiative S1: We didn't actually get much feedback on the actual marking of [the pertussis exercise]. Mine had no written comments on it at all and had 10 out of 20 or something, which I wasn't too happy with. I: So you didn't understand why you'd got that mark? S1: Yeah, well no comments were on it at all […] F Collaborative Initiative S: Yeah. [...] I thought [the feedback on the pertussis assignment] was good because it had written comments and how you'd done in each bit. So it wasn't just a mark out of nowhere, you knew where you'd let yourself down, whether it was the presentation,or whether it was the content, or what.
Case One: FINDINGS ON IMPACT [ With apparently highly similar student cohorts ] F More positive perceptions of advance guidance and feedback about the ‘pertussis enigma’ exercise in every interview following the introduction of the initiative F No evidence in the questionnaire data of impact across the module F Suggests difficulty of change across multiple assignments with many staff involved
Case Two A SMALL FINAL-YEAR HONOURS MODULE F A total of 14-15 students and two staff F Took the form of student-led seminars, assessed by oral presentations and essays F Findings from initial questionnaires and interviews: questionnaire scores low on two feedback items interviews indicated, for presentation and essays: uncertainty about assessment criteria relative paucity of feedback
Case Two A SMALL FINAL-YEAR HONOURS MODULE F The collaborative initiative agreed with the course team to address the concerns identified: more guidance about assessment criteria in introductory class briefing handout on assessment criteria for presentations anonymous written peer feedback on presentations private feedback meeting between staff and student-presenters
Case Two: FINDINGS ON IMPACT [ With apparently highly similar student cohorts ] improvement in questionnaire scores on all the ‘teaching-learning environment’ scales largest change on scales relevant to the collaborative initiative similarly very positive comments in the student interviews
Case Two: FINDINGS ON IMPACT (% ‘agree’ or ‘agree somewhat’) clear expectations how to tackle it fdbk for learning staff support fdbk to clarify
Case Two: FINDINGS ON IMPACT F Pre-Collaborative Initiative No, they’re really weird [essay] titles and I’ve just been like, Whoah, where do you start? Like, they’re really bizarre. F Collaborative Initiative S4They have given us good guidance [about the essays] - […] S2Yeah, they did didn’t they? […] S3Yeah, one of them particularly, it’s not really anything we can find references for […] So, it’s something we’ve really got to kind of think about, and draw on our knowledge of what we already know […]
Case Two: FINDINGS ON IMPACT F Collaborative Initiative I:So do you think having feedback from other students [on your presentation] is worthwhile? S1:I think it is, ‘cause then you realise what you did wrong and how you can improve it. It is actually really useful. S2:Especially from people that, you know, if we do something blatantly stupid they’ll tell us. It’s quite good to get opinions from people who’ve been listening to you but not marking.
REVIEW OF CASE FINDINGS (Bearing in mind the need for caution about the scale and limitations of the research) these research findings would seem to indicate that: 1. students’ concerns about the effectiveness of guidance and feedback took various forms 2. areas of particular concern could be pinpointed, and steps taken to try to address these 3. there was follow-up evidence of impact in interviews (in both cases) and in questionnaires (in case 2) 4. findings from these and other cases suggest that enhancing the quality of feedback and guidance may be harder to achieve in larger team-taught courses
A CODA: MODELLING GUIDANCE AND FEEDBACK “Unfinished business” analysis and writing-up of research evidence as ongoing and recursive Remodelling guidance and feedback as an integrated loop
REMODELLING GUIDANCE AND FEEDBACK [Re]modelling guidance and feedback as an integrated whole the guidance-and-feedback loop takes in both coursework and exams illuminates potential troublespots shows how steps can be inter-related F findings as data and evidence in tandem with findings as tools for diagnosis & enhancement
KEY REFERENCES Black, P., Harrison, C., Marshall, L. and Wiliam, D. (2003). Assessment for Learning. Putting It into Practice. Maidenhead: Open University Press. Carless, D. (2006). 'Differing perceptions in the feedback process ', Studies in Higher Education, 31.2, pp. 219-233. Hounsell, D. (2003). 'Student feedback, learning and development'. In: Slowey, M. and Watson, D. ed. Higher Education and the Lifecourse. Maidenhead: SRHE & Open University Press/McGraw-Hill. pp. 67-78. Hounsell, D. [in press]. 'Towards more sustainable feedback to students.' In: Boud, D. and Falchikov, N., eds. Rethinking Assessment for Future Learning. London: Routledge Hounsell, D et al. (2005) Enhancing Teaching-Learning Environments in Undergraduate Courses: End-of-Award Report to ESRC on project L139251099. Universities of Edinburgh, Durham and Coventry: ETL Project. http://www.ed.ac.uk/etl/publications
KEY REFERENCES Hounsell, D., McCune, V., Hounsell, J. and Litjens, J. ‘The quality of guidance and feedback to students’. [Submitted for journal publication, Sept 2006] Krause, K., Hartley, R., James, R. and McInnis, C. (2005). The First Year Experience in Australian Universities: Findings from a Decade of National Studies. Final Report to DEST. Melbourne: University of Melbourne, Centre for the Study of Higher Education. http://www.cshe.unimelb.edu.au/ http://www.cshe.unimelb.edu.au/ McCune, V. and Hounsell, D. (2005). ‘The development of students' ways of thinking and practising in three final-year biology courses’. Higher Education, 49(2), 255-289. Nicol, D. and Macfarlane-Dick, D. (2006). ‘Formative assessment and self-regulated learning: a model and seven principles of good feedback practice’. Studies in Higher Education, 31(2), 199-218 QAA (2003). Learning from Subject Review, 1993-2001: Sharing Good Practice. Gloucester: Quality Assurance Agency for Higher Education. http://www.qaaa.ac.ukhttp://www.qaaa.ac.uk Sadler, D. R. (1998) Formative assessment: revisiting the territory, Assessment in Education 5(1): 77-84.