1. Looking at assessment dilemmas
What factors do we need to consider in assessment design?
Professor Margaret Price NTF
Dr Erica J. Morris PFHEA
ARU TeachMeet January 2017: 25 January, Cambridge; 27 January, Chelmsford

2. Overview Why assessment dilemmas? Four key dilemmas
On balance considerations
Conclusions
ARU’s Assessment Protocol

3. Why assessment dilemmas?
The detail beyond principles
Assessment design considerations
Subject, discipline or professional context
Your students
ARU Assessment Protocol
Example principles:
Assessment and feedback are integral to learning as part of personal and professional development
Assessment is a complex process and, consequently, the development of assessment literacy is essential for students and staff
Moving from principles to decisions
Assessment design considerations:
Purpose of assessment
Validity
Reliability
Transparency
Overall assessment strategy for module and course (programme)
e.g. practical implications, such as scheduling
Image from: not-taken/

4. Assessment dilemma one: enabling
Independent learning
Scaffolding for learners
Independent learning
Arguments in favour:
More challenging for students
Encourages creativity and originality
Aligned with nature of higher education
Makes marking more interesting
Arguments against:
Too risky for students
Might not meet ‘service’ expectations – students expect a ‘good service’
Does not necessarily support students’ need to have an understanding of elements of quality
Scaffolding for learners
Models approach to assessment for students (illustrates to students how to prepare for an assignment)
Develop students’ learning techniques
Marking can be more straightforward
Convergence of student work (guided therefore could be similar work)
Minimises element of discovery for students
Increases student dependency on staff
Photos: Erica Morris

5. On balance: independence vs scaffolding
The decision for assessment design depends on:
the place in the course – considering student transition, level of study, expected standard of work
learning outcomes relating to expectation of independence
extent of student preparation for the assessment method
students’ understanding of the purpose, nature and techniques of assessment (assessment literacy)
Practice and rehearsal of assessment tasks (including opportunities to develop skills) are
emphasised by assessment for learning perspectives (e.g. Sambell et al, 2013)
a key consideration in terms of employing inclusive assessment practices
Extent of student preparation - have students already completed this type of assessment before? Consideration of assessment strategy of the programme (course) or module
Standard of work – e.g. dissertation work is guided as unlikely student would reach required standards without guidance
See the Anglia assessment album: snapshots on Developing assessment literacy

6. Assessment dilemma two: employing
Conventional method
Innovative assessment
Conventional assessment method
Arguments in favour:
Reliability
Can be resource efficient
Meet student expectations (‘collusion’ with staff)
Fits with the discipline (e.g. lab reports)
Arguments against:
No evidence of greater reliability
Often not designed for learning
Do not cover range of learning outcomes
Some methods may ‘favour’ some groups of students (links with inclusive assessment practices)
Innovative assessment
More likely to be authentic assessment
Interests and motivates students
Can align fully with learning outcomes
More interesting for staff
Can produce efficiency gains – e.g. includes peer assessment
More variety is not always a good thing (Gibbs and Dunbar-Goddet, 2007) (‘over diversifying’ assessment)
Can increase student stress if lack of preparation opportunities (Gibbs and Dunbar-Goddet, 2007)
Up front development and persuading colleagues can take time
Image from: to-focus-listen-attentively-to-college-lectures/

7. On balance: conventional vs innovative methods
Each method needs to be evaluated in relation to:
learning outcomes
resources available
other methods in use as part of the course
time and opportunities for student preparation
‘fit’ with the discipline (although this may need to be challenged)
student preference for authentic assessment
Examples of (more) innovative assessments
writing a journal, blog or reflective diary
producing a briefing paper, newspaper article or action plan
working on a group presentation or conference poster
undertaking a case study analysis
organising and giving student-led seminars
developing information leaflets or learning packages (e.g. for students in schools or related to health promotion)
From: snapshot on Inclusive assessment
See the Anglia assessment album: snapshot on Inclusive assessment

8. Assessment dilemma three: using
Expert judgement
Expert judgement
Arguments in favour:
Meets (traditional) student expectations
Fits easily with quality systems
Staff are experienced markers and know their subjects
Staff teams can compare marks
Arguments against:
Little evidence of expert judgment being more reliable
Does not encourage self regulation
Encourages dependence on staff
Self- and peer-assessment
Develops students’ understanding of standards
Develops students’ assessment literacy
Encourages development of independent learning skills
Can reduce staff marking time
Develops graduate skills (including communicating and working with others, and evaluation of self and others)
Requires time for necessary preparation and set up
Students can be worried about showing their work to other students
Students may perceive use as staff ‘laziness’
Students may think peer-assessment is ‘unfair’
Self- and peer- assessment
Image from: library-and-study-space/
Image from:

9. On balance: expert judgement vs self- and peer-assessment
Consideration needs to be given to:
students’ concerns in using self- and peer-assessment through briefing opportunities with discussion
preparation of students to enable them to undertake self- and peer- assessment well and with confidence
emphasising to students the benefits of self- and peer-assessment (e.g. in terms of graduate attributes and employability)
The assessment strategy with regard to self- and peer- assessment opportunities on the course
Designing authentic or work integrated assessment that involves varied audiences (see Collaborate project): peers, clients, experts, employers
See the Anglia assessment album: snapshot on Assessing groupwork

10. Assessment dilemma four: assessment criteria
Providing developed criteria
Students creating criteria
Providing developed criteria
Arguments in favour:
Aims to provide clarity and transparency of assessment and marking for students (e.g. see NUS, n.d.)
Belief that helps to improve consistency in marking process
Can be used as a basis for discussion with students about what is expected in their work
Arguments against:
It is difficult to fully specify assessment standards (i.e. in terms of student attainment)
Students may become over-reliant on marking guidance and criteria, which may have implications for developing as independent learners
Providing detailed assessment criteria do not reflect practices in the real world or work place (relates to designing authentic assessment)
Students creating criteria
Engages students as they are motivated do well on assessments
Enables students to see more fully the issues related to achieving consistency in marking (and the limitations of using criteria)
Helps students to understand what is expected of them in terms of quality work (can use criteria with exemplars, for instance) (relates to assessment literacy)
Could create confusion for some students who may not see that it is their role to be involved in creating criteria (as they are not the expert)
Unlikely to be effective if students are not prepared
Students may become ‘bogged down’ in the detail of devising criteria (as a ‘science’)
Photo from assessment workshop at ARU: Erica Morris

11. On balance: assessment criteria
Carefully consider when students might have opportunities to be involved in creating criteria through working with other students in class
To fully understand what is expected students need opportunities for activities involving the use of assessment criteria
Is there sufficient opportunity for staff and students to share and understand the criteria?
See the Anglia assessment album: snapshots on Developing assessment literacy and Marking matters

12. Conclusions There is no one answer to suit all contexts
Take an evidence-informed approach for complex decision making in assessment design
Once informed, discuss your ideas with colleagues
Validity matters: learning outcomes are the key guide to resolving dilemmas

13. References
Collaborate Project on designing work integrated assessment: Gibbs, G. and Dunbar-Goddet, H. (2007) The effect of programme assessment environments on student Learning. York: Higher Education Academy. Higher Education Funding Council for England (HEFCE) (2016) A new National Student Survey for Annex A: Final list of NSS 2017 questions. National Union of Students (NUS) (n.d.) Assessment and feedback benchmarking tool. Price, M., Rust, C., O’Donovan, B., Handley, K. and Bryant, R. (2012) Assessment Literacy: The Foundation for Improving Student Learning. Assessment Standards Knowledge exchange (ASKe), Oxford Centre for Staff and Learning Development, Oxford Brookes University. Sambell, K., McDowell, L. and Montgomery, C. (2013) Assessment for Learning in Higher Education. Abingdon, Oxon, UK: Routledge.