Presentation on theme: "Assessment which works: Enhancing student learning without killing the tutors Prof. Sue Bloxham Susan.firstname.lastname@example.org."— Presentation transcript:
1 Assessment which works: Enhancing student learning without killing the tutors Prof. Sue Bloxham
2 pragmatic ways of transforming our assessment practice SomeassessmentchallengesLow satisfaction rates; More students – more marking; Anxiety about innovation in assessment (extra work, QA); Difficulty engaging students in formative assessment, peer assessment.Focus today onpragmatic ways of transforming our assessment practice
3 Certificationto identify and discriminate between different levels of achievement, and between students;providing a license to practice in the case of professional programmes;enabling selection of students for further study and employment.This is assessment of learning.
4 Quality assuranceto provide evidence for relevant stakeholders (for example, external examiners, QAA, professional bodies);to enable them to judge the appropriateness of standards on the programme.This is assessment of learning.
5 Student Learningto promote effective learning; formative and diagnostic; steering students’ approach to studying; giving the tutor useful information to inform changes in teaching strategies.This is assessment for learning.
6 Lifelong learning: sustainable assessment to achieve an understanding of standards;to learn how to make judgments;to be able to use criteria;to be able to tell when you really understand something.This is assessment as learning.
7 The unbalanced purposes of assessment Assessment for and as learningCertification & QA
8 Assessment for certification & QA involves: Formal feedback;2nd marking;Moderation;External examiners;Assessment boards.Slow, costly & high stakesAssessment for and as learning can:Be immediate;Regular;In-class;On-line;Involve students in their own assessment ;Uses peer assessment.Quick, cheap & low stakesLet’s note that staff effort in assessment of learning is often distracted from a focus on how students will learn from doing the assessment. Even the giving of feedback, for example, on coursework may be more driven by QA imperatives (e.g. to give sufficient feedback against the learning outcomes) rather than by what helps students to learn.Where the assessment is primarily for learning (e.g. formative assessment), the issues of reliability and subjectivity in marking, whose work is it, worries about peer judgment can be dispensed with in favour of giving rich and immediate opportunities to learn. It opens up a range of methods such as group, peer and in-class assessment which might well be rejected if they were contributing in significant way to assessment for certification.Such feedback often comes too late to be usefulStudents often don’t get any useful feedback, except an overall mark, from examinations so they have to guess where their weaknesses are.
9 Characteristics of assessment which promote learning and employability Formative, involving dialogue;Demands higher order learning;Learning and assessment are integrated;Students are involved in assessment;It promotes thinking about the learning process;Assessment expectations should be made clear;Involves active engagement of students, developing independent learning;Tasks should be authentic and involve choice;Tasks align with important learning outcomes;Assessment should be used to evaluate teaching.Lets’ look at how assessment can be designed to promote learning – these characteristics are drawn from a range of recent influential studies of higher education assessment.Formative, involving dialogue – so student know that gapDemands higher order learning – so students are developing a picture of quality which matches the way tutors think about quality (more of that later)Learning and assessment are integrated – so students are thinking about what they are learning as they go about itStudents are involved in assessment – so they are developing a better understanding of what the standards are that they should be aiming forIt promotes thinking about the learning process;Assessment expectations should be made clear; - so they know what the standards are – very difficult to do.Involves active engagement of students, developing independent learning;Tasks should be authentic and involve choice;Tasks align with important learning outcomesAssessment should be used to evaluate teaching.We will come back to concrete ideas for assessment methods that meet these criteria in a minute but firstly I’d like to think about what type of learning we are currently assessing.Complex knowledge and skills are being assessedStudents won’t be successful until they know what is expected from them.It is difficult to express the standards, norms and rules of complex achievements, it is ‘tacit’ knowledgeTacit knowledge can only be learnt through participation, imitation, observationStaff develop different views of what they want from students’ workCriteria are an attempt to ‘write down’ tacit knowledge and therefore have limited use if students have not been actively helped to understand what they mean in the specific module context
10 Exams & tests They often come at the end of a course; They are not integrated into the learning;The criteria are oblique;Tend to assess factual & conceptual knowledge only.They rarely result in useful feedback (sometime usefully programmed into a test);When students do badly we tend to blame the students rather than use the results to diagnose problems with our teaching.Lets’ think about how a very common form of assessment, exams and tests, meets the thinking I’ve been describing in terms of both encouraging learning and the range of knowledge that they can assess. There are lots of problems although for certification purposes, they are very pragmatic.
11 Essays Often no formative element; have the potential to meet many characteristics but:Often no formative element;Questions may ask students to ‘evaluate’ or ‘critically assess’ a topic but if students can pass adequately by regurgitating others’ evaluation or criticism (from lectures or reading), they may avoid higher order learning;Students not involved in assessment or thinking about learning;Rarely authentic;Criteria often oblique.
12 DiscussionBriefly discuss current practice with those sitting near you:How well balanced are the different purposes of assessment?How much are students engaged in their learning?How satisfied are students with assessment and feedback?
13 Psychology Redesign 560 students in groups of 7-8; 3 week cycle culminating in word essaye.g.’Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible?Guidance provided for tackling the question and working in a group;Best essays posted on VLE as feedback;Students used familiar language to discuss academic concepts – Dialogue.Nicol 2008
14 Field-based enquiry integrated with the learning encourages independent and active learninginvolves students in the assessment process (avoiding grades in the early stages)higher order skills, complexityauthenticity, choiceincludes formative stages, students can get help and feedback in a low stakes wayExpectations available to students both in written criteria and embedded in feedbackPotential to integrate learning from university with learning from other contexts
15 Peer assessment in lab reports FormativeStudents develop understanding of quality in analysing and reporting scienceHigher order learning, focus on the science rather than a description of the processStudents involved in assessment, developing skills of evaluation
16 Assessment tasks for engagement & independent learning Patchwork texts (students have regular tasks which they bring to class for formative feedback – tutor marks a final, reflective synthesis of these tasks – see P. Ovens for example in Science education)Tutor sets readings with questions each week. Students bring draft answers for discussion with peers to class, groups can ask the tutor if they are stuck, then individuals submit revised answers (4-500 wds). Tutor posts generic feedback on VLE. Portfolio of final answers submitted. Tutor only marks week 4, 7, 9 for feedback & grade (selects the weeks after submission).Students submit draft assignments on line for anonymous peer assessment. Students submit revised assignment with copies of their reviews and statement on how they responded to their reviewers (Liverpool example).Key feature is that the students see this formative work as directly contributing to summative work. Don’t call it formative!!
17 CautionMultiple methods have been associated with negative learning outcomes for studentsExtra workload for students (perhaps this is a good thing?)
18 Engaging students in formative assessment – key requirements It clearly feeds into summative assessment tasks;The students must submit it in some way (bring to class, post on line, hand it in) and action is taken if they don’t;Students receive useful feedback on it;It is not contaminated by summative purposes.
19 Selling peer assessment Evidence shows students find their peers a useful and more approachable source of help with assignments but we need to stress the main value in peer assessment is standing in the shoes of the assessor – not being assessed – because:learning about standards – absolutely crucial to making progress and understanding feedbackSeeing other ways of going about the task – develops strategies for taking actionKey employability skill – being able to judge own performance and assessing and giving feedback to othersMore opportunity for dialogueChance for more formative feedbackPeer assessment needs to become a regular feature of programmes so that it is taken seriously and taken for granted as part of learning at this level.
20 Staff work loadA heavy staff workload in assessment is not necessarily helpful to student learning. Improving assessment needs to be accompanied by a shift in effort from tutors to students:more student activity and engagement (doing tasks, reviewing progress, self-regulation) and less staff activity (marking)Formative assessment has the potential to achieve both of these and is most likely to increase achievement.
21 Tutor friendly formative feedback Tutor posts good examples or model answers on VLE;Students peer assess tasks using assessment criteria;Tutors give feedback on posters, presentations in class;Oral/ on-line feedback to group after marking a sampleTasks done on-line (e.g EMQs, MCQs), auto marked and give immediate feedback;Tutors put main effort into marking drafts (agreed with examiner), just checking for change and putting mark on final piece.
22 Example of module-level approaches: the use of exemplars annotated with feedback toencourage dialogue about assessment criteriaFrom Oxford Brookes FDTL project on feedbackStudents write and submit individual assignm-entsTutor assesses assignm-ents and prepares feedbackOut of class activityTutor leads discussion of exemplars previously marked and annotated with feedbackTutor hands back assignments and leads discussion on feedbackIn-class activity22Module timelineSubmission point
23 Example of module-level approaches: From Oxford Brookes FDTL project on feedbackExample of module-level approaches:Generic (non-personalised) feedback on draftsplus reflective commentary4. Students rewrite and submit assignments with reflective commentary on how they have incorporated the generic feedback1. Students draft & submit individual assignm-ents2. Tutor marks sample of assignments and prepares generic feedback5. Tutor assesses assignm-entsOut of class activity6. Tutor hands back assignments with minimal formative feedback3. In-class discussion of generic feedbackIn-class activity23Module timelineSubmission point
24 ConclusionAddressing student dissatisfaction and staff workloads requires us to:Balance the different purposes of assessment;Encourage and reward student engagement;Ensure that students’ study efforts are directed towards meaningful learning;Provide sufficient low stakes, informal assessment and feedback for students to grasp both the standards required and the strategies needed to achieve them.Use tutor time most effectively for learning and limit the time absorbed by QA to essential summative items;
25 Additional slides which delegates may find useful Prof. Sue Bloxham
26 Dealing with concerns regarding assessment which isn’t so easily quality assured for certification For example, peer, group and self assessment, in class assessment, presentations, feedback at the draft stagePotential ways forwardTake a programme approach to assessment designMake greater use of assessment methods which combine different purposes (e.g. inquiry tasks,)Use peer & self assessment methods in low stakes ways, e.g. for formative assessment and feedback.Make sure you have a robust method for dividing group marks (See University assessment guide)
27 Benefits to students of moving to in-class, on-line, ongoing assessment and feedback Immediate feedbackMore feedbackAssessment & teaching/learning are integratedStudents involved in assessment – gaining better understanding of standards and own performancePotential for greater student engagement throughout modulesMore independent studyintegration with work experienceRaise expectations regarding study workloadQuicker, cheaper and low stakes
28 Feedback from different sources collaborative taskscase discussionsteam assessmentpeer assessmentself feedbackwork-based mentors, etcThese methods allow students to check out their understanding and practice judgements in relative safety. Seeing how other students go about things is a key source of feedback on their own work and helps them find ways to fill the ‘gap’ between current performance/ knowledge and what is expected.Use the examples: Students working as teams in practice, observing each other and giving feedback – started with the tutor modelling feedback but quickly, the tutor found they had very little to do as the students in this situation quickly started thinking about what was important to notice, what was good quality teaching. This is particularly good for helping students grasp the sense of quality that the tutor holds – they have to stand in the shoes of the assessor and have to think about what counts as good work.the carousel approach to students working on tasks or problems in the classroom. In groups of 4, they come up with a proposal or solution. One of them stays on their table to explain their approach, the other 3 disperse to other tables to hear and question (critical friend) a different group’s approach. They then return to reconsider their proposals in the light of what they have heard.However these feedback opportunities are created, it is really important that we explain to students that these are all important sources of feedback that they can learn from.They also all present opportunities to help students understand the complexity (and subjectivity) of professional judgement when many factors have to be taken into account.28
29 FeedbackBe clear about the purpose of the feedback ie what should students expect to get from it in particular instances.Ensure students have sufficient support in understanding criteria and standards to relate to the feedback provided.Require students to self assess against generic/cohort feedback.Model the process of feeding forward for the students.Require students to demonstrate how they have used feedback in subsequent work.
30 Some other assessment methods Evaluation of journal article or other paperFilm or radio programmes about a specific topicResearch Grant applicationsLay commentary on specialist material, e.g. journal articlePoster – presenting information clearly & conciselyPresentation – oral communicationProblems and case study analysisReflective Journals, Diaries & learning logsWriting tasks: newspaper articles, press releases, executive summaries, information sheets.Wikipedia entryShorter pieces can demand just as much student time but involve significantly less staff marking time.