Presentation on theme: "‘Developing & embedding inclusive assessment across Plymouth University’ Professor Pauline Kneale and Jane Collings."— Presentation transcript:
1‘Developing & embedding inclusive assessment across Plymouth University’ Professor Pauline Kneale and Jane Collings
2Challenging myths and changing approaches to assessment Difficult territoryPowerful mythsDisciplinary defencesSpaces of resistanceBroadening possibilitiesEntering ‘brave’ new spacesTransformational learning that is equitable.
4What students say..Sophie – Biomedical Science What are the explicit and implicit messages in the video clip?
5Definition from the literature Inclusive assessment refers to the design and use of fair and effective assessment methods and practices that enable all students to demonstrate to their full potential what they know, understand and can do (Hockings, 2010, p.2)No just a disability issueDiverse student populationIllustrations by Chris Glynn
6Why inclusive assessment ? Through inclusive design wherever possible, and through individual reasonable adjustments wherever required, assessment tasks provide every student with an equal opportunity to demonstrate their achievement. (QAA.2013 UK Quality Code for Higher Education, Ch B6)Traditional assessment practices that were once dominated by the unseen examination and the standard essay…have proved unable to capture the range and nature of the diverse learning outcomes now sought from courses.” (Boud & Falchikov 2006)Work place relevant/authenticPreparation for the real worldAustralia – far advanced in authentic assessment
7Types of Assessment Diagnostic Assessment (Preparing) provides an indicator learner’s existing knowledge and capabilitiesdevelop self regulated learner.identify possible learning support needs .Formative Assessment (Improving)provide learners with timely feedback/forwardintended to have an impact on current learning and ultimately to be connected to improved performanceSummative Assessment (Judging)progression and certification purposesstudents often use performances in summative tasks as a proxy measure of learningIntroduce Assessment for LearningCrisp (2012)
8An inclusive approach Contingent approach Waterfield & West (2006) SPACE ProjectPlymouth UniversityContingent approachoffers provision of special arrangements or adjustments within existing systemsAlternative approachoffers different assessment methods as a bolt-on for a minority of disabled students.Inclusive approachis designed to ensure accessibility for all students and reduces the need for MAPs.This work is not new in SPACE identified the needMAPs – bust the jargon - no room to put it in full
9NSS: Assessment and feedback ( % agree )National Student Survey200920102011201220132014Assessment%Clear marking criteria707172747677Assessment is fair737578FeedbackFeedback is prompt575962656869Provides detailed comments6366Clarifies student queries586067Overall student satisfaction8183848586These NSS figures show the assessment and feedback scores have been really static and difficult to move upwards
10There is some excellent practice…. Student Comments: Source: UPSU Student Voice Report (2014)‘Feedback is both comprehensive and constructive whether formative or summative. I now know exactly where I can improve without ever feeling stupid or a failure.’‘Feedback was so clear and showed me how to develop my work in the future.’‘Feedback is often sent back within a matter of hours after sending work off for formative marking, meaning students can improve and work on skills needed continuouslyFeedback is comprehensive and constructive I now know exactly where I can improve without ever feeling stupid or a failure
11There is more improvement required …… Student Open CommentsSource: (NSS 2014)‘The assessments were vague and unclear leaving me puzzled and confused- I have no idea what makes an A, B or C’.‘The coursework was outdated, and difficult because it lacked industry relevance’‘I was unsure of the assignment requirements/criteria despite attending all tutorials and discussions with the module lead’,‘The disparity in grades and feedback was vast: some being penalised for grammatical, spelling or referencing errors, whilst others were not! ‘‘We need formative assessment at the beginning so we can improve in summative assessments’‘I spent 4 months writing my dissertation project and when I got it back there was five lines of feedback, I felt like I’d wasted my time'‘The coursework guidance was ambiguous’‘Feedback took over 20 days to return and was unclear how my work could be improved’
12Assessment for Learning What are the issues?What are the cultural implications?Why now?What do we need to do ?Comments onto post it notesDiscussion on tables minutesTaking key issues in the plenary
13Developing an inclusive assessment strategy At Plymouth we conducted a comprehensive consultation on this agendaAa a result 15 strategic recommendations went to OVCLeading toInclusive assessment ambitionAssessment Policy 2014
14Inclusive Assessment why now at Plymouth ? The increasing diversity of students in UKAn increase in disabled students & associated modified assessment provision costsStudent voice -low scores in NSS for assessment and feedbackNUS & UPSU focus on assessment and feedbackIn 2012 a review of PU assessment provision indicated inconsistent practicesAn increase in appeals & settlementsA strategic focus on student retention through regular meaningful formative assessment tasks and prompt feedback.Non traditional entrants- mature students- 3rd year direct entry – international studentsStudent voice – NUS charter NUS benchmarking tool ( copies on tables )NSS & SPQ nationally low – at Plymouth we go from excellent to awfulA review of assessment show inconsistency around issues such as academic offences, extenuating circumstances, assessment equivalence, staff having to set 5 different assessments for a one 3 hour examDisabled numbers increasing 2012 –12.6 % % %We spend as much on invigilation of MAPs as all other studentsIncrease in appeals over assessment - poor questions etc
15Plymouth University inclusive assessment ambition: ‘all students will have an equitable, supported assessment experience’.Inclusive assessments will :-Fairly evaluate students’ ability to meet module and programme learning outcomes and academic standardsBe accessible for all studentsProvide every student with an equal opportunity to demonstrate their achievementSupport student engagement, learning, progression, retention and address the needs of our diverse student populationBe authentic and offer students contextualised meaningful tasks that replicate real world challenges through effective programme design.Reduce the need for modified assessment provision)Set the scene – SPACE , CEP , NSS SPQReview of assessment provision in the universityAssessment for Learning project
16PU Assessment Policy: 2014Pre- assessment / feed-in activities & information, discussion, feed-in with clear assessment and marking criteriaAuthentic, MAP free/ simple assessment methods aligned to learning outcomes.Scheduling - throughout the year, normally only 2 summative assignments in 20credit modules.Access to software for electronic submission and originality checking software.Fairly marked, anonymously ( if appropriate) with moderationFeed-forward & feedback with marks (asap) but within 20 daysAfter consultation with over 200 staff and students and outings at many different forums this policy was developed
17Resources to assist programme level review of assessment Staff development through:-workshops on inclusive assessments and examinationsfaculty and school away days,development of resources both paper and comprehensive websiteThese tie in with both Inclusive Assessment and changes to the curriculum for CEP
18Inclusive does not mean ‘easier’ or ‘avoiding things’ Inclusivity can be enhanced through:-Feed-inSet up, briefing, preparation,Practice & rehearse,assessment & marking criteriaFeed-forwardFormative ‘feedback’, cues, discussion, mid-way reviews, peer & self review/feedbackFeedbackend of task, written, verbal, mp3, YouTube,marks/gradesLearningSpaceFeed-in/ briefingFormative assessmentProgressive staging of assessment(building complexity)( Magne 2012)
19Reviewing assessment in a programme Year:1/2/3/4/5MCQ examIPSE / OSCECase study/ lab reportEssayReflective journalWebsite/ wiki /podcastOpen book examPresentation /vivaInformation leafletInclusivity indicator/ risk factorsSkills developmentfeed-in /preparationoffers progressionfeed -forward / feed backModified assessment provisionScheduling bottle- necksWork placed alignedCommunication skillsTeamworkExampleModule 1Module 2Module 3Module 4Module 5Module 6ExerciseUse this gap analysis to review assessment for each year of a programmeOffers risk factors and which modules assess skills developmentRAG Rating - Red – Amber – Green (Collings & Magne 2013)
20NUS Benchmarking assessment and feedback tool 2012 It also offers an excellent example of assessment and marking criteriaFollows the NUS Charter 2010 with the 10 principles of assessment and feedbackNew NUS direction on improved feedback
22Improved design Assignments aligned to learning outcomes & assessment criteriaA range of assessment methods: authentic/work related & few modification implicationsSchedule assessments evenly across the yearOffer students detailed pre-assessment activitiesOffer progression and opportunities to practiceDesign inclusive examinations with 50% max weightingWell written and clearly structured questionsConsider a choice of assessment methods (maximum of 2)
23Design inclusive assessment methods Example of a staff development activity:Using the assessment methods hand-out on the tablesMethods need to be : -Reduce the number of complex modified assessmentsAuthentic and offer real work related challengesAssessment methods hand-out
24Traditional Assessment a) Introduction to Marketing(Principles, overview & apply concepts)Current assessment methods4 x multiple choice question tests - 30% weighting3 hour unseen exam –70% weightingb) Introduction to Animal Feed Management(Principles, & feed management plan)Current assessment methods3 short tests % weighting3 hour unseen exam - 80% weightingExerciseTo discuss more inclusive and authentic methods of assessmentDiscuss weighting inequality of 70:30 and 80:20Advantaging students who have good memory recall and are good at exams and testsWhat are the other assessment method options ?
25Examples of inclusive practice at Plymouth University Education: offers a choice of two assessment methods Law: fewer traditional exams, now ‘open book’ & ‘seen’ exams, more formative short answer & MCQ tests. Accounting & Finance : 100% coursework assessment for modules without professional body exemptions and formative assessment opportunities replace tests. History: work-facing assessments Marine Biology: formative assessment via peer review Marketing: no traditional exams, authentic assessments (e.g. group reports, marketing plans and presentations) Navigation & Maritime Science: assessment briefings, revision sessions, 8 hour assignment with a 27 hour time limit, 1.5 hour class tests with the room invigilated for 3 hours.EducationIn some modules students develop the assessment criteria and agree the methodsLawThe professional body the Solicitors regulation authority (SRA) removed unseen exams - Law developed authentic assessmente.g. assessment brief at 3pm – open book at 10 am – matches workplace practiceAccounting and FinanceOverhaul of all assessment – move towards authentic inclusive assessment unless professional bodies stipulateHistoryA more curatorial approach is being taken in a number of assessmentsStudents have to go to the museum and chose an artefact and then write an essay based on the artefact in a historical contextMarine BiologyIntroduction of formative assessment through peer review has raised performanceMarketingUsed to have exams and tests now work place aligned assessmentsNavigation and Maritime ScienceComprehensive assessment briefing, 27 hour accident investigation – each student receives a different accident scenario.
26Impact of inclusive intervention at Plymouth University Improvements in student satisfaction scoresAssessmentFeedbackAccounting and Finance+20%+16%Business Administration+36%+14%Marketing+31%-4%School of Tourism and Hospitality+19%+17%PhysiotherapyDental Surgery+23%Engineering+32%+22%Marketing - To early for impact on feedback ( As it is in minus )Lower scores in feedback reflect where has been less intervention and changeNational Student Survey: 2014.
27Challenging myths and changing approaches to assessment Difficult territoryPowerful mythsDisciplinary defencesSpaces of resistanceBroadening possibilitiesEntering ‘brave’ new spacesTransformational learning that is equitable.
28ReferencesBoud, D. & Falchikov, N. (2006) Aligning assessment with long term learning. Assessment & Evaluation in HE. vol31.no4.pCrisp, G. (2012) Integrative assessment: Reframing assessment practice for current and future learning. Assessment and Evaluation in Higher Education, vol. 37, no. 1, ppHockings, C. (2010) Inclusive learning and teaching in higher education: a synthesis of research. Higher Education Academy resource.Mogey, N., Purcell, M., Pateson, J., Burke. J .; ( 2013) Handwriting or typing exams – can we give students the choice?O’Neill, G. (2011). A Practitioner’s Guide to Choice of Assessment Methods within a Module: Case Studies for University College Dublin. DublinQAA: (2013) UK Quality Code for HE. Chapter B6. Assessment of students and recognition of prior learningWaterfield, J., & West, B.,(2006) Inclusive Assessment in Higher Education: A Resource for Change. Plymouth University
29Comprehensive staff resource. All resources on these pages
31Conducting inclusive exams Weighting of exams– no more than 50%Use a range of exam methods (e.g seen, open book, take home etc.)Signpost support for exam technique sessionsPrepare students for exams -feed-in / feed-forwardTaking the anxiety out of exams through opportunities to practice and rehearseEnsure students receive feedback after each examOffer students the opportunity to type exams and use assistive software ( Mogey 2013)Discuss weighting inequality of 70:30 and 80:20Advantaging students who have good memory recall and are good at exams and testsLink back to the Sophie video clip