‘Developing & embedding inclusive assessment across Plymouth University’ Professor Pauline Kneale and Jane Collings

Challenging myths and changing approaches to assessment Difficult territory Powerful myths Disciplinary defences Spaces of resistance Broadening possibilities Entering ‘brave’ new spaces Transformational learning that is equitable.

What students say.. Sophie – Biomedical Science What are the explicit and implicit messages in the video clip?

Definition from the literature Inclusive assessment refers to the design and use of fair and effective assessment methods and practices that enable all students to demonstrate to their full potential what they know, understand and can do (Hockings, 2010, p.2) No just a disability issue Diverse student population Illustrations by Chris Glynn

Why inclusive assessment ? Through inclusive design wherever possible, and through individual reasonable adjustments wherever required, assessment tasks provide every student with an equal opportunity to demonstrate their achievement. (QAA.2013 UK Quality Code for Higher Education, Ch B6) Traditional assessment practices that were once dominated by the unseen examination and the standard essay…have proved unable to capture the range and nature of the diverse learning outcomes now sought from courses.” (Boud & Falchikov 2006) Work place relevant/authentic Preparation for the real world Australia – far advanced in authentic assessment

Types of Assessment Diagnostic Assessment (Preparing) provides an indicator learner’s existing knowledge and capabilities develop self regulated learner. identify possible learning support needs . Formative Assessment (Improving) provide learners with timely feedback/forward intended to have an impact on current learning and ultimately to be connected to improved performance Summative Assessment (Judging) progression and certification purposes students often use performances in summative tasks as a proxy measure of learning Introduce Assessment for Learning Crisp (2012)

An inclusive approach Contingent approach Waterfield & West (2006) SPACE Project Plymouth University Contingent approach offers provision of special arrangements or adjustments within existing systems Alternative approach offers different assessment methods as a bolt-on for a minority of disabled students. Inclusive approach is designed to ensure accessibility for all students and reduces the need for MAPs. This work is not new 2006 in SPACE identified the need MAPs – bust the jargon - no room to put it in full

NSS: Assessment and feedback ( % agree ) National Student Survey 2009 2010 2011 2012 2013 2014 Assessment % Clear marking criteria 70 71 72 74 76 77 Assessment is fair 73 75 78 Feedback Feedback is prompt 57 59 62 65 68 69 Provides detailed comments 63 66 Clarifies student queries 58 60 67 Overall student satisfaction 81 83 84 85 86 These NSS figures show the assessment and feedback scores have been really static and difficult to move upwards

There is some excellent practice…. Student Comments: Source: UPSU Student Voice Report (2014) ‘Feedback is both comprehensive and constructive whether formative or summative. I now know exactly where I can improve without ever feeling stupid or a failure.’ ‘Feedback was so clear and showed me how to develop my work in the future.’ ‘Feedback is often sent back within a matter of hours after sending work off for formative marking, meaning students can improve and work on skills needed continuously Feedback is comprehensive and constructive I now know exactly where I can improve without ever feeling stupid or a failure

There is more improvement required …… Student Open Comments Source: (NSS 2014) ‘The assessments were vague and unclear leaving me puzzled and confused- I have no idea what makes an A, B or C’. ‘The coursework was outdated, and difficult because it lacked industry relevance’ ‘I was unsure of the assignment requirements/criteria despite attending all tutorials and discussions with the module lead’, ‘The disparity in grades and feedback was vast: some being penalised for grammatical, spelling or referencing errors, whilst others were not! ‘ ‘We need formative assessment at the beginning so we can improve in summative assessments’ ‘I spent 4 months writing my dissertation project and when I got it back there was five lines of feedback, I felt like I’d wasted my time' ‘The coursework guidance was ambiguous’ ‘Feedback took over 20 days to return and was unclear how my work could be improved’

Assessment for Learning What are the issues? What are the cultural implications? Why now? What do we need to do ? Comments onto post it notes Discussion on tables - 15 minutes Taking key issues in the plenary

Developing an inclusive assessment strategy At Plymouth we conducted a comprehensive consultation on this agenda Aa a result 15 strategic recommendations went to OVC Leading to Inclusive assessment ambition Assessment Policy 2014

Inclusive Assessment why now at Plymouth ? The increasing diversity of students in UK An increase in disabled students & associated modified assessment provision costs Student voice -low scores in NSS for assessment and feedback NUS & UPSU focus on assessment and feedback In 2012 a review of PU assessment provision indicated inconsistent practices An increase in appeals & settlements A strategic focus on student retention through regular meaningful formative assessment tasks and prompt feedback. Non traditional entrants- mature students- 3rd year direct entry – international students Student voice – NUS charter 2010 - NUS benchmarking tool 2013 - ( copies on tables ) NSS & SPQ nationally low – at Plymouth we go from excellent to awful A review of assessment show inconsistency around issues such as academic offences, extenuating circumstances, assessment equivalence, staff having to set 5 different assessments for a one 3 hour exam Disabled numbers increasing 2012 –12.6 % 2013- 8% 2014 13% We spend as much on invigilation of MAPs as all other students Increase in appeals over assessment - poor questions etc

Plymouth University inclusive assessment ambition: ‘all students will have an equitable, supported assessment experience’. Inclusive assessments will :- Fairly evaluate students’ ability to meet module and programme learning outcomes and academic standards Be accessible for all students Provide every student with an equal opportunity to demonstrate their achievement Support student engagement, learning, progression, retention and address the needs of our diverse student population Be authentic and offer students contextualised meaningful tasks that replicate real world challenges through effective programme design. Reduce the need for modified assessment provision ) Set the scene – SPACE , CEP , NSS SPQ Review of assessment provision in the university Assessment for Learning project

PU Assessment Policy: 2014 Pre- assessment / feed-in activities & information, discussion, feed-in with clear assessment and marking criteria Authentic, MAP free/ simple assessment methods aligned to learning outcomes. Scheduling - throughout the year, normally only 2 summative assignments in 20credit modules. Access to software for electronic submission and originality checking software. Fairly marked, anonymously ( if appropriate) with moderation Feed-forward & feedback with marks (asap) but within 20 days After consultation with over 200 staff and students and outings at many different forums this policy was developed

Resources to assist programme level review of assessment Staff development through:- workshops on inclusive assessments and examinations faculty and school away days, development of resources both paper and comprehensive website These tie in with both Inclusive Assessment and changes to the curriculum for 2015 CEP

Inclusive does not mean ‘easier’ or ‘avoiding things’ Inclusivity can be enhanced through:- Feed-in Set up, briefing, preparation, Practice & rehearse, assessment & marking criteria Feed-forward Formative ‘feedback’, cues, discussion, mid-way reviews, peer & self review/feedback Feedback end of task, written, verbal, mp3, YouTube, marks/grades Learning Space Feed-in/ briefing Formative assessment Progressive staging of assessment (building complexity) ( Magne 2012)

Reviewing assessment in a programme Year: 1/2/3/4/5 MCQ exam IPSE / OSCE Case study/ lab report Essay Reflective journal Website/ wiki /podcast Open book exam Presentation /viva Information leaflet Inclusivity indicator/ risk factors Skills development feed-in /preparation offers progression feed -forward / feed back Modified assessment provision Scheduling bottle- necks Work placed aligned Communication skills Teamwork Example    Module 1 Module 2 Module 3 Module 4 Module 5 Module 6 Exercise Use this gap analysis to review assessment for each year of a programme Offers risk factors and which modules assess skills development RAG Rating - Red – Amber – Green (Collings & Magne 2013)

NUS Benchmarking assessment and feedback tool 2012 It also offers an excellent example of assessment and marking criteria Follows the NUS Charter 2010 with the 10 principles of assessment and feedback New NUS direction on improved feedback

Designing and Developing inclusive assessment

Improved design Assignments aligned to learning outcomes & assessment criteria A range of assessment methods: authentic/work related & few modification implications Schedule assessments evenly across the year Offer students detailed pre-assessment activities Offer progression and opportunities to practice Design inclusive examinations with 50% max weighting Well written and clearly structured questions Consider a choice of assessment methods (maximum of 2)

Design inclusive assessment methods Example of a staff development activity: Using the assessment methods hand-out on the tables Methods need to be : - Reduce the number of complex modified assessments Authentic and offer real work related challenges Assessment methods hand-out

Traditional Assessment a) Introduction to Marketing (Principles, overview & apply concepts) Current assessment methods 4 x multiple choice question tests - 30% weighting 3 hour unseen exam –70% weighting b) Introduction to Animal Feed Management (Principles, & feed management plan) Current assessment methods 3 short tests - 20% weighting 3 hour unseen exam - 80% weighting Exercise To discuss more inclusive and authentic methods of assessment Discuss weighting inequality of 70:30 and 80:20 Advantaging students who have good memory recall and are good at exams and tests What are the other assessment method options ?

Examples of inclusive practice at Plymouth University Education: offers a choice of two assessment methods Law: fewer traditional exams, now ‘open book’ & ‘seen’ exams, more formative short answer & MCQ tests. Accounting & Finance : 100% coursework assessment for modules without professional body exemptions and formative assessment opportunities replace tests. History: work-facing assessments Marine Biology: formative assessment via peer review Marketing: no traditional exams, authentic assessments (e.g. group reports, marketing plans and presentations) Navigation & Maritime Science: assessment briefings, revision sessions, 8 hour assignment with a 27 hour time limit, 1.5 hour class tests with the room invigilated for 3 hours. Education In some modules students develop the assessment criteria and agree the methods Law The professional body the Solicitors regulation authority (SRA) removed unseen exams - Law developed authentic assessment e.g. assessment brief at 3pm – open book at 10 am – matches workplace practice Accounting and Finance Overhaul of all assessment – move towards authentic inclusive assessment unless professional bodies stipulate History A more curatorial approach is being taken in a number of assessments Students have to go to the museum and chose an artefact and then write an essay based on the artefact in a historical context Marine Biology Introduction of formative assessment through peer review has raised performance Marketing Used to have exams and tests now work place aligned assessments Navigation and Maritime Science Comprehensive assessment briefing, 27 hour accident investigation – each student receives a different accident scenario.

Impact of inclusive intervention at Plymouth University Improvements in student satisfaction scores Assessment Feedback Accounting and Finance +20% +16% Business Administration +36% +14% Marketing +31% -4% School of Tourism and Hospitality +19% +17% Physiotherapy Dental Surgery +23% Engineering +32% +22% Marketing - To early for impact on feedback ( As it is in minus ) Lower scores in feedback reflect where has been less intervention and change National Student Survey: 2014.

Challenging myths and changing approaches to assessment Difficult territory Powerful myths Disciplinary defences Spaces of resistance Broadening possibilities Entering ‘brave’ new spaces Transformational learning that is equitable.

References Boud, D. & Falchikov, N. (2006) Aligning assessment with long term learning. Assessment & Evaluation in HE. vol31.no4.p399-413 Crisp, G. (2012) Integrative assessment: Reframing assessment practice for current and future learning. Assessment and Evaluation in Higher Education, vol. 37, no. 1, pp. 33-43. Hockings, C. (2010) Inclusive learning and teaching in higher education: a synthesis of research. Higher Education Academy resource. Mogey, N., Purcell, M., Pateson, J., Burke. J .; ( 2013) Handwriting or typing exams – can we give students the choice? O’Neill, G. (2011). A Practitioner’s Guide to Choice of Assessment Methods within a Module: Case Studies for University College Dublin. Dublin QAA: (2013) UK Quality Code for HE. Chapter B6. Assessment of students and recognition of prior learning Waterfield, J., & West, B.,(2006) Inclusive Assessment in Higher Education: A Resource for Change. Plymouth University

Comprehensive staff resource. All resources on these pages http://www1.plymouth.ac.uk/ouruniversity/teachlearn/guidanceresources/Pages/Assessment.aspx

Conducting inclusive exams Weighting of exams– no more than 50% Use a range of exam methods (e.g seen, open book, take home etc.) Signpost support for exam technique sessions Prepare students for exams -feed-in / feed-forward Taking the anxiety out of exams through opportunities to practice and rehearse Ensure students receive feedback after each exam Offer students the opportunity to type exams and use assistive software ( Mogey 2013) Discuss weighting inequality of 70:30 and 80:20 Advantaging students who have good memory recall and are good at exams and tests Link back to the Sophie video clip