1. Assessment in Higher Education
A free, open, online course and community
While we are waiting to start please take the opportunity to run through the Audio Setup Wizard (one last time )
Click (top left)
Select the Audio Setup Wizard option
Follow on-screen instructions
Let us know if you are having a problem by typing “Help” into the Chat box and we’ll see what we can do
Important Note
At this point you may be able to hear and see Rod, Rachel and Hayley in the video pod but not any of the other participants or yourself.
We have done this on purpose .
Please bear with us while we get organised and take the time to run the audio set up wizard again.
Tuesday, April 09, 2019

2. We are about to start recording the webinar.
Please note that we will make all recordings available to participants only via the Assessment in HE Wordpress site.

3. Outline Review of Task 4 Introduction to Topic 5 and Task 5
Topic: What is feedback for?
Task: Analysing your written feedback
Guest activity from Gwyneth Hughes and Holly Smith, UCL Institute of Education, London
Tuesday, April 09, 2019
Event Name and Venue

4. Assessment in HE Webinar 5

5. Assessment for learning
A bit of revision
What is a learning outcome ?
The essential measurable learning or behaviour demonstrated by a student after a specific period of study
What is formative assessment?
“Formative assessment is designed to provide learners with feedback on progress and inform development, but does not contribute to the overall assessment.”
QAA 2000
What is summative assessment ?
The process by which we measure the extent to which a student can demonstrate specific learning outcomes
Assessment for learning
Assessment of learning

6. Impact of feedback
“Feedback is arguably the most important aspect of the assessment process in raising achievement”
(Bloxham and Boyd, 2007)
“Arguably the most powerful enhancement to learning is feedback during learning”
(Biggs and Tang, 2007)
… however
“academics frequently report frustration that students fail to act on feedback or to collect it at all”
(Jollands et al. 2009, Bloxham and Boyd, 2007)

7. Typical Assessment Scenario
In the final year of his Business Management degree George is set the task of producing an individual 1500 word management report based on case study materials that he has been studying with his tutorial group. The report constitutes 50% of the final unit grade the other 50% is assessed by a 3 hour examination at the end of the semester. The hand in date for the essay is 6 weeks before the examination. The work is marked within 3 weeks and returned to George with his mark and a detailed written feedback proforma. The assignment is also discussed in a feedback tutorial during the week the work is returned. The report tests two of the 5 learning outcomes for the unit. These learning outcomes are therefore not tested in examination (which tests the remaining three learning outcomes). George sits the examination for the unit which is marked in time for exam board. George is given the mark awarded for the examination but receives no verbal or written feedback on his performance in the exam.
How do we anticipate that George will use his feedback?
Will it help me to do better in the exam?
Will this feedback help me get a better mark for the report?

8. Characteristics of feedback

9. Overview of literature
Brown, E, Glover, C, Freake, S and Stevens, V (2004)
proposed three levels or degrees of explanation in feedback:
Indication that there is an error/weakness or omission (Level 1)
Provides correction or appropriate response (Level 2)
Provides explanation as to why the student’s response was incorrect or inappropriate or why suggestion was preferable (Level 3)
Glover and Brown (2006) extended this classification and categorisation to the identification (Level 1) and explanation (Levels 2 and 3) of strengths in the work.
Orsmond and Merry (2011) analysed tutors’ feedback using the following feedback categories based on Brown et al. (2003)
JISC projects Assessment Careers (IoE) and InterAct (Medical Education, Dundee) have used amended versions to audit feedback in their institutions to increase dialogue and increase amount and type of feedback provided. The eFEP (OU) is looking at aligning tutor and student use of spoken and written feedback in a language programme.
The e-Feedback Evaluation Project (eFEP) to examine the ways in which students and tutors actually use spoken and written e‑feedback when these have become standard practice across an entire programme;
to evaluate the perceptions and preferences of tutors and students in relation to spoken and written e‑feedback in those learning contexts;
to investigate the ways in which students engage with the written and spoken e‑feedback that they receive;
to evaluate the extent to which the findings from the above evaluations apply to non‑distance learning institutions in which e-feedback is an emerging practice;
to identify the areas in which further support is needed in order to maximise the benefits of spoken and written e-feedback;
to produce a set of training and support tools for students and tutors

10. Timing of feedback?

11. Opportunistic Action Research
Three Case Studies
Tropical Land Use and Conservation (20 credits)
2010/11 Level 5 cohort of 40 students
Written (2000 word) assignment mid-way through the autumn term
Dr Francis Brearley
Accounting, Finance & Economics (30 Credits)
2011/12 Level 6 cohort of 251 students
Written (2000 word) assignment examining theory in relation to real world mergers and acquisitions
Wendy and Peter Wild
Employment and People Management (30 credits)
2011/12 Level 5 cohort of 98 students
A written (2000 word) reflective essay
Ed Bielinski
All practice development projects/dissertation topics PG Cert/MA Academic Practice
Opportunistic Action Research

12. Targeting formative feedback
“Summative
Feedback”
or “Feedforward” to inform future work
Directly linked
Indirectly linked
1. Set Assignment
6. Summative Assessment
2. Draft
Submission
(Optional)
5. Complete Submission
3. Formative
Assessment
“Formative
Feedback” directly linked to current task
Not linked
4. Reflection and Reworking

13. Chris Beaumont, Michelle O’Doherty & Lee Shannon (2011): Reconceptualising assessment feedback: a key to improving student learning?, Studies in Higher Education, 36:6, :

14. Course
% Non-Sub of Draft
Tropical Land Use and Conservation (20 credits)
2010/11 Level 5 cohort of 40 students
37%
Accounting, Finance & Economics
2011/12 Level 6 cohort of 251 students
40%
Employment and People Management (30 credits) 2011/12 Level 5 cohort of 98 students
50%

15. Targeting formative feedback
Consistently higher attainment for those who submitted a draft – 1 grade band

16. Reiteration & Evidence
Submitted 25 (63%)
Non-submitted 15 (37%)
Complete
4 (16%)
Work in Progress
15 (60%)
Outline
6 (24%)
Tropical Land Use and Conservation Brearley F.Q. and Cullen W.R. (2013) Providing formative audio feedback on a written assignment. Bioscience Education eJournal Vol 20, pp
60.8%
52.6%
Summative grade
69.8%
61.6%
51.8%
66.6%
61.3%
46.2%
46.2%
Hand-in
Feedback length
Feedback content
Fine tuning
Fine tuning & Content
Reiteration & Evidence

17. Reiteration & Evidence
Submitted 25 (63%)
Non-submitted 15 (37%)
Complete
4 (16%)
Work in Progress
15 (60%)
Outline
6 (24%) 1
60.8%
52.6%
Summative grade
69.8%
61.6%
51.8% 3 5
66.6%
61.3%
46.2%
46.2%
Hand-in 2
Time on task
Better organised/more strategic
Improved performance
More feedback, better understood, easier to respond to
Is this simply making better students better?
Feedback length 4
Feedback content
Fine tuning
Fine tuning & Content
Reiteration & Evidence

18. Accounting, Finance and Economics

19. Take home messages More attention on learning how to learn
Quality of submission influences the quality of the feedback
The “timing” of the feedback is maybe more important than the “speed of the feedback” and “method” of feedback
Where participation is optional the effect may be to make the “best” students better rather than helping those who most need it academically
More attention on learning how to learn

20. From the literature
Biggs and Tang, 2007 do not regard the feedback provided on end of unit summative assessment as formative as the feedback is provided when the unit is effectively finished and students rarely pay attention to comments provided at the end of a course.
“feedback provided by tutors focused on performance on the assignment being assessed”
Orsmond et al, 2011
“students often struggle to transfer learning from one unit to another”
Orsmond et al. 2011
Glover and Brown (2006) comment that in terms of written feedback students receive plenty of it, but that it is often misunderstood in relation to assessment criteria.
Orsmond et al. (2005) found that a majority of students preferred verbal feedback from tutors as it enabled questioning and discussion.

21. Engaging Students with feedback
What approaches would you suggest to get students valuing and utilizing the feedback that they receive?

22. This week Question: What is feedback for?
Task: Analysing your written feedback
Guest activity from Gwyneth Hughes and Holly Smith, Institute of Education, London
See Wordpress site for details
Submission to the Google discussion area or directly to Rachel, Rod and Hayley via

23. Thoughts? Comments? Questions?

24. Just in case section

25. Involving students - How?
Discussing
Criteria
Choosing
Topics
Self Assessment
Assignment
Briefings
Assessment
Tutorials
Setting
Criteria
Staged
Assessments
Peer Assessment
Feedback
Tutorials
Linked to:
Predefined outcomes,
Knowledge and Process
Learners need direction to carry out procedures and meet outcomes
Able to discuss learning, plan and organise work
Decide when to work alone collaboratively, seek advice
Self managed + directed
Formulate own learning objectives
Select learning strategies
Reflect on progress and decide next step
Autonomy
Procedural
Personal
Critical
McAtominey & Cullen 2002, Ecclestone 2000

26. e.g. PBL, Collaborative learning, Reflective practice, etc
EQAL
Constructive Alignment
Mode and Method of Assessment 2
(ECF) MMU Graduate Outcomes
Requirements of Prof Bodies
Employers requirements
Subject Benchmarks
Programme Learning Outcomes
EQAL Requirements
MMU Assessment Framework
Diagnostic
Formative
Summative
MMU Threshold Standards
Exam, Essay, Report, Portfolio, Poster, Reflection, Presentation, MCQs, etc
Assessment Criteria
Marking Process
Feedback Strategy
Aims and Intended Learning Outcomes 1
Blooms Taxonomy
Methods and Delivery of Teaching and Learning 3
F2F
Online
Blended
Underlying Pedagogy
e.g. PBL, Collaborative learning, Reflective practice, etc
Lecture, Seminar, Practical, Fieldwork, Lab work

27. Online unit proforma

28. Why are they important to us?
Fundamental principle of outcomes-based education (Constructive alignment)
Advocated since Dearing (1997) by the QAA
Entrenched in Bologna process (1999)
Statutory requirement for programme validation (programme and unit/module level)
Way we communicate to students what they are expected to do and achieve
Metrics against which we are required to assess students
Underpin our learning, teaching and assessment strategies
EQAL is very prescriptive

29. MMU Assessment Framework
Principles - Assessment should be:
Valid: the specified assessment should test specific learning outcomes
Reliable: the specified assessment should be able to measure student achievement against learning outcomes accurately and consistently
Achievable: assessments and assessment regimes should be designed so that the threshold requirements are achievable by any student admitted on to the programme of study
Gradable : it should usually be possible to grade student

30. MMU Assessment Framework
Assessment should be integrated into all curriculum planning and to be directly aligned with intended learning outcomes
Assessment should provide evidence of achievement to agreed standards
Assessment should recognise student progress
Assessment should promote learning, facilitate the identification of areas for development and provide motivation for further achievement

31. Checking Alignment - Assessment Strategy Matrix
Intended Learning Outcomes
Learning Activities
Assessment
Paper Based
Online
F2F
Formative
Summative
Describe and explain…
Lecture 1,2
Tutorial – Intro to Assign 1
Lecture 1,2 Overview
Assign 1 Practical write-up
Exam
Self test/ Quiz
Assess the validity …
Lecture 8
Tutorial – Intro to Assign 2
Lecture 8 Overview
Assign 2 Group Poster
Self Test /Quiz

32. Attendance and submission 2008-2010
Cullen W. R., (2011)
Percentage
MA Summative Assessment
1. Reasonably high overall attendance
2. Generally high (70% +) submission rates for formative tasks
3. Use of the feedback is embedded in the online prep and F2F activities
3. Consistently lower submission rates in Week 4

33. Acceptable assignments?
Data Analysis
Computer based assessment
Essay
A Performance
Roll play
Written Report
Event
MCQs
Critique
Photo Essay
Practice Observation
Oral presentation
Poster
Magazine article
Web site
“Dragon’s Den™”
Wiki contribution
Blogg/learning log
Artefact
Job Application
Research Article
Case Study
“The Apprentice”
Newsletter
Reflective account
Group work
Dissertation
Video report
Well with those rules in mind, I’ve made a list of the different types of assignment that might be acceptable at MMU. Do you need me to explain any of these cursory headings? An attempt at brevity may have led to loss of clarity so shout out if anything is unclear.
Are all of these allowed? What are the barriers to implementing any of these?
Just in your head, count up the number of different types of assignment you currently use, either formatively or summatively.
Whisper the number to your neighbour.
Does anyone here use more than 10 different types of assignment?
(if so…well done, special prize coming your way) 6? 3?
Don’t you get bored? Do you think that your students might get bored? Are the assignments that you use really the best ways for students to demonstrate their achievements?
You may have noticed the inspiration of a couple of TV programmes there. You do need to be careful. Would the X factor be appropriate?
Annotated Map
Audio report
Interview
Lab Practical
Annotated Reference List
Portfilio
Competition entry
Translation
Problems to solve
Unseen Written exam
Seen written exam
Work based assessment

34. References
Boud, D. (1995). Assessment and Learning: contradictory or complementary? Assessment for Learning in Higher Education. P. Knight. London, Kogan Page: Biggs, J. B. and Tang, C. (2007). Teaching for quality learning at university. Open University Press/Mc Graw-Hill Education. Bloxham, S. and Boyd, P. (2007). Developing Effective Assessment in Higher Education: A Practical Guide. Berkshire, UK. Open University Press. Cullen W. R., (2011) A multi-technology formative assessment strategy, Media-Enhanced Feedback case studies and methods, Proceedings of the Media-Enhanced Feedback event, Sheffield, 27 October 2010 pp Ecclestone, K. (2000) Assessment and Critical Autonomy in Post Compulsory Education in the UK, in, Journal of Education and Work, Vol. 13, No. 2. Glover, C. and Brown, E. (2006) Written Feedback for Students: too much, too detailed or too incomprehensible to be effective? Bioscience Education Journal (7) Jollands, M., McCallum, N., Bondy, J. (2009) If students want feedback why don’t they collect their assignments? Proceedings of 20th Australasian Association for Engineering Education Conference University of Adelaide, 6-9th December 2009 McAtominey, D. & Cullen, W.R. (2002) Effective e-Learning with VLE’s, Netskills Workshop Materials Orsmond, Paul , Merry, Stephen and Reiling, Kevin(2005) 'Biology students' utilization of tutors' formative feedback: a qualitative interview study', Assessment & Evaluation in Higher Education, 30: 4, pp Orsmond, Paul and Merry, Stephen(2011) 'Feedback alignment: effective and ineffective links between tutors' and students' understanding of coursework feedback', Assessment & Evaluation in Higher Education, 36: 2, pp