Presentation is loading. Please wait.

Presentation is loading. Please wait.

Transforming assessment through the tenets Sally Brown and Brenda Smith 23 April 2013.

Similar presentations

Presentation on theme: "Transforming assessment through the tenets Sally Brown and Brenda Smith 23 April 2013."— Presentation transcript:

1 Transforming assessment through the tenets Sally Brown and Brenda Smith 23 April 2013

2 Some thoughts on the nature of assessment as a transformative process to enhance student learning; An overview of the six tenets; A review of how tenets one and two integrate holistically with the other tenets; Some practical suggestions to support your work based on tenets 1 and 2; Opportunities to develop your work and share ideas and issues so you can start to make some meaningful changes at your institutions. This session aims to offer: 2

3 Stems from firm evidence that assessment is notfit for purpose (Race 2010, Brown, 2010); Aims to take a radical approach, recognising that it is time for significant reappraisal of assessment policy and practice (ASKe, Weston Manor Manifesto); Builds on expertise, perspectives, and previous work; Takes an evidence-informed approach; Encourages for assessment to be seen as an integral part of the learning experience. Background to A marked improvement 3

4 QAA subject reviews Burgess Report - system no longer fit for purpose (p5) QAA - …it cannot be assumed students graduating …. will have achieved similar standards (2007) Media accusations of dumbing down and grade inflation National Student Satisfaction Survey There is considerable scope for professional development in the area of assessment (Yorke et al, 2000, p7) Students become more interested in the mark and less interested in the subject over the course of their studies (Newstead 2002, p2). Why change is needed 4

5 The debate on standards needs to focus on how high standards of learning can be achieved through assessment. This requires a greater emphasis on assessment for learning rather than assessment of learning Tenet 1 5

6 Our current systems focused on marks and grades are not working: Belief that it is possible to distinguish the quality of work to a precision of one percentage point (Elander & Hardman, 2002); Belief that double-marking will ensure fairness and reliability (Polani, 1998); Belief that consistency can be achieved through conformity, and simple numerical rules (e.g. level 1 essay 3,000 words, level 3 essay 5,000; or no more than two pieces of assessment per module); The combination of scores, which obscures the different types of learning outcome represented by the separate scores; The distortion of resulting degree classifications by the application of idiosyncratic institutional rules (e.g. Armstrong et al, 1998). Why change is needed 6

7 Students developing as learners Informal feedback Formal feedback Practice, rehearsal Formative and summative Authentic assessment Assessment for learning 7 Adapted from Sambell, McDowell and Montgomery (2012, p5)

8 Even when lecturers say that they want their students to be creative and thoughtful, students often recognise that what is really necessary, or at least what is sufficient, is to memorise (Gibbs, 1992, p10) We need to emphasise for students the importance of spending rich and purposeful time on task, not just going through the motions We need to focus on high standards of achievement and to clarify what we seek 8

9 Are you confident that assessment tasks demand high standards of learning? Is assessment for learning given emphasis in relation to assessment of learning? Do you ensure an appropriate balance between formative and summative assessment? Is assessment and feedback planned within and across a programme to ensure appropriate student preparation and practice before summative assessment takes place? (From A marked improvement) Key questions: To what extent and how do you evidence this? 9

10 When it comes to the assessment of learning, we need to move beyond systems focused on marks and grades towards the valid assessment of the achievement of intended programme outcomes Tenet 2 10

11 Meaningful understanding of standards requires both tacit and explicit knowledge (ODonovan et al. 2004) We can know more than we can tell (Polanyi, reprinted 1998, p.136) Verbal level descriptors are inevitably fuzzy (Sadler 1987) There is a cost (in terms of time and resources) to codifying knowledge which increases the more diverse an audiences experience and language (Snowdon, 2002) Tacit knowledge is experience-based and can only be revealed through the sharing of experience – socialisation processes involving observation, imitation and practice (Nonaka, 1991). Why change is needed 11

12 Is their an emphasis on assessment for learning over systems focused on marks, grades and reliability? Does the assessment design process ensure valid assessment of the intended learning outcomes? Is there a trade off between reliability and validity of assessment? Are assessment decisions in relation to design, development and variety made within a programme context and focused on learning outcomes? (From A marked improvement) Key questions: To what extent and how do you evidence this? 12

13 Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity. There are important benefits of higher education which are not amenable either to the precise specification of standards or to objective assessment (for further discussion at later targeted forums, but here as part of a holistic discussion of the tenets as a whole) Tenet 3 13

14 The most significant factor in student academic success is student involvement fostered by student/staff interactions and student/student interactions (Astin, 1997) The only common factor in a study of departments deemed excellent in both research and learning and teaching is high levels of student involvement (Gibbs, 2007) Participation, as a way of learning, enables the student to both absorb, and be absorbed in the culture of practice (Elwood & Klenowski, 2002, p. 246) Why change is needed 14

15 Assessment standards are socially constructed so there must be a greater emphasis on assessment and feedback processes that actively engage both staff and students in dialogue about standards. It is when learners share an understanding of academic and professional standards in an atmosphere of mutual trust that learning works best (for further discussion at later targeted forums, but here as part of a holistic discussion of the tenets as a whole) Tenet 4 15

16 Important aspects of complex, high-level learning outcomes can only be achieved when students are allowed time to come to know the standards in use by the community: Slowly learnt academic literacies require rehearsal and practice throughout a programme (Knight and Yorke 2004) The achievement of high-level learning requires integrated and coherent progression based on programme outcomes Where there is a greater sense of the holistic programme students are likely to achieve higher standards than on more fragmented programmes (Havnes, p. 2007) Students need to engage as interactive partners in a learning community, relinquishing the passive role of the instructed within processes controlled by academic experts (Gibbs et al, 2004) Why change is needed 16

17 Active engagement with assessment standards needs to be an integral and seamless part of course design and the learning process in order to allow students to develop their own, internalised, conceptions of standards and monitor and supervise their own learning (for further discussion at later targeted forums, but here as part of a holistic discussion of the tenets as a whole) Tenet 5 17

18 Changes in higher education (e.g. massification, reduced unit of resource, expectations of increased productivity in staff) threaten the health of disciplinary communities and their ability to share and exemplify professional judgement. There has been slow progress in the professionalisation of university teachers. There has been limited attention paid to professional assessment practice. Reliance on the external examiner system to mediate standards within the system is misplaced (Newstead and Dennis,1994). If some aspects of high-level learning can only be assessed using professional judgement then we need to ensure that judgement is indeed – professional!! Why change is needed 18

19 Assessment is largely dependent upon professional judgement and confidence in such judgement requires the establishment of appropriate forums for the development and sharing of standards within and between disciplinary and professional communities (for further discussion at later targeted forums, but here as part of a holistic discussion of the tenets as a whole) Tenet 6 19

20 The current approach to assessment standards is inadequate; The quest for reliability is getting in the way of learning; We should be enabling students to achieve high level, complex learning; Active involvement by staff and students in the learning community is essential to reach common understandings of assessment standards; To achieve change there may need to be a review and evaluation of the allocation of time and resources within HE. Why change is needed 20

21 Using the framework on the next slide, focus on tenet one to interrogate your own practices to date and where you can make improvements; Identify between three and five ways in which you would like to make changes in assessment practice to make your assessment focus more on assessment for learning. Prepare some summary bullet points on a flip chart sheet for sharing with other HEI teams. Task: working in institutional teams 21

22 Tenet 1 – Key Points Assessment for learning Learning and assessment should be integrated, fully aligned, challenging & on-going Assessment should build on previous knowledge There should be a balance between formative and summative assessment Students need time for preparation & practice A range of approaches to feedback are needed Students need to give and receive feedback and feed-forward and engage in a dialogue re their learning Tasks should involve the active engagement of students developing the capacity to find things out for themselves & learn independently Are your students enabled to broaden their skills, knowledge and experience? Are a range of assessment methods used and constructively aligned? Is your assessment linked, cumulative and challenging? Are your students given opportunities to prepare and practice? Do your students engage in self and peer assessment & feedback? Do you help students and staff to understand the role of assessment and feedback in learning? Do your assessment tasks encourage independent thinking? Is your assessment spread throughout the academic year? 22

23 Using the framework in the next slide, discuss with colleagues what you consider your top priorities to work on from the questions and suggestions provided there; Identify ways that you would measure and evaluate changes as a result of the actions you plan to take; Ask a spokesperson to be prepared to report back on this in plenary. Task: working in institutional teams 23

24 Tenet 2- Key Points Ensuring assessment is fit for purpose Systems that focus on marks and grades should be reviewed Emphasis should be on assessment validity Students should be offered a choice of assessments suited to their needs Assessments should demonstrate students have achieved the programme learning outcomes Course/programme teams needs to collaborate Assessment methods should be chosen for more than reliability Do your students summative marks reflect the achievement of the programme learning outcomes? Is formative assessment built into your assessment cycle and does it feed forward? Do your colleagues work collaboratively across a programme to ensure coherence and validity? Do your staff ensure assessments are linked to the learning outcomes? Are you confident that students being marked by different people or the same people at different times (inter & intra-tutor reliability) will achieve equivalent marks? Does your University have a strategy for staff engagement and development in assessment and feedback to ensure up-to-date knowledge & experience? 24

25 Programme learning outcomes should reflect what students should achieve Making it clear to students what is expected of them; Making it clear to teachers what students are expected to learn in their own and other modules; Helping teachers to select the most appropriate teaching strategy for the intended learning outcomes e.g. lecture, seminar, tutorial, group work, discussion, student presentation, laboratory work. Helping teachers to select the most appropriate assessment style to assess the achievement of the learning outcomes, e.g. project, essay, performance assessment, multiplechoice questions, exam; Having a focus on programme learning outcomes – staff therefore need time to collaborate; Are you confident that students being marked by different people or the same people at different times (inter and intra-tutor reliability) will achieve equivalent marks? 25

26 Methods of Assessment Assignments and the learning opportunity they offer Communicate information and ideas in other than the written word Become more adept at writing in different forms, formats and genres To understand the benefits and challenges of working collaboratively 26

27 Methods of Assessment Assignments and the learning opportunity they offer Communicate information and ideas in other than the written word Become more adept at writing in different forms, formats and genres To understand the benefits and challenges of working collaboratively Seminar, poster, video, video conferencing or other multimedia Design or proposals; book reviews; case report/study; web pages; journal articles Group problem solving; joint book reviews; team presentations; role plays; group exhibition 27

28 Is the schedule of assignments and the feedback on these assignments made clear to students across the year? Do you provide multiple opportunities for small low stake assessments with feedback? Are your students given opportunities to prepare and practice? Is there an appropriate balance between summative and formative assessment? Is assessment and feedback planned within and across programmes and are staff given time to ensure this happens? Balancing and timing formative and summative assessments 2013 University of Poppleton University of Poppleton 28

29 Total study effort = 68 hours Figure 1: A students estimate of his weekly study effort (in hours) on a programme with no assignments and an exam in week Gibbs (2010)

30 Total study effort = 78 hours Figure 2: A students estimate of her weekly study effort (in hours) on a program with three assignments due in weeks 4, 7 and 10 and an exam in week (Gibbs 2010)

31 Assessment for Learning Do your students always know what the criteria are for all work they are given? How do you help students to understand the criteria? Could the students re-write the criteria in their own words? 31

32 Students cant read our writing There is too much emphasis on grades and marks at the expense of learning The feedback given is often not very useful and comes too late Students are not actively encouraged to self reflect Little or no use is made of peer/self assessment and feedback Little dialogue takes place around feedback Students have little opportunity to collate feedback over time and act upon it Most feedback does not feed forward, it only tell students what they have done that is incorrect Very little use is made of feedback as a normal part of the learning and teaching process Staff vary in their approach to feedback Key issues in feedback – develop the capacity to use feedback effectively 32

33 Some thoughts on promoting the engagement of students by helping them to get confident with assessing Give students in groups a selection of essay or other relevant piece of work Excellent, Good, Pass, Near pass, Fail Let the students decide individually which is which and then compare notes In groups get them to tease out the differences between them and express them as criteria Produce suggestions for writing an effective essay or relevant piece of work Get them to write appropriate feedback comments Exchange the comments with a different group and ask them to comment on the feedback. Is it helpful, would they know what to do differently, does it feed forward etc? What suggestions would they give to the group to enhance the feedback comments? 33

34 How you might help colleagues develop the capacity to use feedback effectively In small groups exchange your assignments and compare the comments received Do you understand the comments, if not discuss what you think they might mean? How would you respond to them? What would you do differently next time if you redid the same assignment and applied the learning to a new and different assignment? Return marked work back to the individual but with no grade or feedback Get them to mark their own work Collect in their grades (Pass a sheet round) 9 out of 10 students will be within 5% Arrange to talk to those where the difference is more than 5% or one grade 34

35 Giving generic feedback within 24 hours on the VLE Offering face- to-face feedback with whole group/small groups Using peer feedback in groups Using technologies including: A sound file – Audacity Dragon software Statement Banks Sounds good project Encouraging students to use self feedback Helping students to recognise that feedback can come from many different sources (including themselves) and given in different ways Saving time – a range of approaches to feedback need to be in place 35

36 Involve the students- Feedback as a dialogue For Student CompletionFor Staff Completion These are areas of my work that I think are good for the following reasons Please comment on the following areas of work What I want to improve or do differently next time The mark I think this piece of work deserves is When I get to sit down with my tutor and discuss my work – thats probably the best form of feedback I can get 36

37 Promoting the engagement of students across modules and programmes Learning Most significant feedback comments What did you understand these to mean? Analysing Things that I did that attracted positive comments Things that I did that attracted critical comments Any recurring trends or comments? Planning and feeding forward Things I can do to build on the positive feedback Things I can do to address the critical feedback (after Race, 2007). 37

38 All the tenets together provide a coherent agenda to bring about a marked improvement in assessment in your HEI; The focus of the rest of the morning is to present your initial plans of action for enhancement with another pilot institution so that ideas and practices can be shared, developed and critiqued. Bringing this thinking together 38

39 1.Using some ideas from the presentation and the publication A Marked Improvement, review how you might work with your colleagues back in your own institution to embed issues arising from Tenet 1 (Assessment for Learning) and Tenet 2 (Ensuring assessment is fit for purpose). 2.Working with another institutional task group, please present your plans to date and share your ideas and any issues that have arisen. 3.Now consider what are the opportunities and challenges that these two tenets offer and how might your plans change as a result of your conversations with colleagues here? Paired activity in your institutional task group 39

40 References and further reading 1 Armstrong, M., Clarkson, P. and Noble, M. (1998), Modularity and credit frameworks: the NUCCAT survey and 1998 conference report, Newcastle-upon- Tyne, Northern Universities Consortiuum for Credit Accumulation and Transfer. ASKe Weston Manor manifesto (accessed April 2012) Assessment Reform Group (1999) Assessment for Learning : Beyond the black box, Cambridge UK, University of Cambridge School of Education. Astin, A. (1997), What matters in college? Four critical years revisited. San Francisco: Jossey Bass. Bloxham, Sue. "Marking and moderation in the UK: false assumptions and wasted resources." Assessment & Evaluation in Higher Education 34.2 (2009): Brown, G. with Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education, London: Routledge. Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, London: Kogan Page. Brown, S. Rust, C. & Gibbs, G. (1994) Strategies for Diversifying Assessment, Oxford: Oxford Centre for Staff Development. Brown, S. and Glasner, A. (eds.) (1999) Assessment Matters in Higher Education, Choosing and Using Diverse Approaches, Maidenhead: Open University Press. Brown, S. and Race, P. (2012) Using effective assessment to promote learning in Hunt, L. and Chambers, D. (2012) University Teaching in Focus, Melbourne: Australian Council for Educational Research( Acer Press). P

41 References and further reading 2 Brown, S. (2011) First class: how assessment can enhance student learning in Blue Skies: new thinking about the future of higher education, London: Pearson. Boud, D. (1995) Enhancing learning through self-assessment, London: Routledge. Burgess, Robert. Beyond the honours degree classification: Burgess Group final report. Universities UK, Elander, J. and Hardman, D. (2002), An application of judgement analysis to examination marking in psychology, British Journal of Psychology 93, pp Elwood, J. and Klenowski, V. (2002), Creating communities of shared practice: the challenges of assessment use in learning and teaching, Assessment and Evaluation in Higher Education, 27, pp Gibbs, G. (1992), Improving the quality of student learning. Bristol: TES Gibbs, G (2007) Departmental leadership of teaching Presented at The Oxford Learning Institute research seminar series, University of Oxford. Hilary term Gibbs, G. 2010, Using assessment to support student learning, Leeds Met Press, Leeds. Gibbs, P., Angelides, P. and Michaelides, P. (2004), Preliminary thoughts on a praxis of higher education teaching, Teaching in Higher Education, 9, pp Havnes, A. (2007), What can feedback practices tell us about variation in grading across fields? Presented at the ASKe Seminar Series, Oxford Brookes University, 19th September. Laming, D. (1990) The reliability of a certain university examination compared with the precision of absolute judgements, Quarterly Journal of Experimental Psychology Section AHuman ExperimentalPsychology, 42 (2), pp. 239–254. Knight, P.and Yorke, M (2004),Learning, Curriculum and Employability in Higher Education. London: Routledge. McDowell, L. and Brown, S. (1998) Assessing students: cheating and plagiarism, Newcastle: Red Guide 10/11 University of Northumbria. 41

42 References and further reading 3 Nicol, D. J. and Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education Vol 31(2), Newstead, S. E. and Dennis, I. (1994), Examiners examined: the reality of exam marking in psychology, The Psychologist, 7, pp Nonaka, I. (1991), The knowledge-creating company, The Harvard Business Review, November- December, pp ODonovan, B., Price, M. and Rust, C. (2004), Know what I mean? Enhancing student understanding of assessment standards and criteria, Teaching in Higher Education, 9, pp Pickford, R. and Brown, S. (2006) Assessing skills and practice, London: Routledge. Race,P., Brown, S. and Smith, B. (2005) 500 Tips on Assessment: 2 nd edition, London: Routledge Polanyi, M. (1998) The tacit dimension, in: L. PRUSAK (Ed.) Knowledge in Organization (Boston, MA,Butterworth Heineman). Race, P. (2010) Making learning happen (chapter 4) Sage, London Race, P. (2001) A Briefing on Self, Peer & Group Assessment, in LTSN Generic Centre Assessment Series No 9 LTSN York. Race P. (2006) The lecturers toolkit (3rd edition), London: Routledge. Race (2007) How to get a good degree, London, Routledge. Rust, C., Price, M. and ODonovan, B. (2003) Improving students learning by developing their understanding of assessment criteria and processes, Assessment and Evaluation in Higher Education. 28 (2),

43 References and further reading 4 Sambell, K McDowell, L and Brown, S 1997, But is it fair? An exploratory study of student perceptions of the consequent validity o assessment, Studies in Educational Evaluation, vol. 23 (4), pp Knight, P. and Yorke, M. (2003) Assessment, learning and employability Maidenhead, UK: SRHE/Open University Press. Sadler, D. R. (1987), Specifying and Promulgating Achievement Standards, Oxford Review of Education, 13, pp. 191–209. Sadler, DR 1989, Formative assessment and the design of instructional systems, Instructional Science, vol. 18, pp Sadler, R. (2008) Assessment of Higher Education, in International Encyclopaedia of Education Yorke, M. (1999) Leaving Early: Undergraduate Non-completion in Higher Education, London: Routledge. Yorke, Mantz, and Bernard Longden. (2007)"The first-year experience in higher education in the UK: Report on Phase 1 of a project funded by the Higher Education Academy." Yorke, M., Bridges, P and Woolf, H. (2000), Mark distributions and marking practices in UK higher education; some challenging issues, Active Learning in Higher Education, vol. 1, no. 1, pp

Download ppt "Transforming assessment through the tenets Sally Brown and Brenda Smith 23 April 2013."

Similar presentations

Ads by Google