1. Matching student expectations and staff practices in producing and engaging with high quality feedback
Jason Bohan, Maxine Swingler, Heather Woods
School of Psychology, University of Glasgow

2. Introduction
Feedback promotes student engagement with deeper learning and positively impacts on future learning (Biggs, 1999)
Effective feedback aids transition from supported school/college environments to independent learning in HE (Poulous & Mahoney, 2008)
UK NSS scores reveal low satisfaction for feedback
Staff perception is that they provide good quality feedback, but student’s fail to utilise this (Careless, 2006)
Q. Given the importance of feedback, why is there a difference between student and staff experiences of the feedback process and can we align the process more tightly?

3. Method Data Collection – 3 Phases Student Survey Student Survey
Focus groups
Staff Survey
59 (12%) Level 1 psychology students responded to a survey posted on our VLE (Moodle). Low response but representative of the class:
Average age = 21 years
78% Female
Honours students 79%
New to psychology - 59%
Native language – 63%
Compared to Bohan & Stack survey – numbers similar. (proportionally higher honours and age here)
26% of the class:
Average age = 19
76% females
Honours 62%
New to psychology 61%
Native language 63%

4. Method Data Collection – 3 Phases Student Survey – 6 themes
Focus groups
Staff Survey
What is feedback?
What is good feedback?
Expectations of assessments and experience of feedback
How do you use feedback?
How do you want to receive feedback?
How does the feedback you receive differ from before university?

5. Method Data Collection – 3 Phases Focus Groups Student Survey
Staff Survey
Honours student volunteers ran one focus group
4 students participated

6. Method Data Collection – 3 Phases Staff Survey Student Survey
Focus groups
Staff Survey
10 staff members all involved in assessing Level 1 feedback were asked:
What they believed was important in coursework
What type of comments they provided on coursework
What type of additional feedback students requested
How they liked to give feedback

7. 1. What is feedback? 10 sources of feedback to be rated:
Written comments
Talking to teachers
Exam feedback
Feedback sheets – written comments
Feedback sheets – generic comments
from tutors
Lab quiz (automated)
VLE (Moodle) comments from staff
VLE (Moodle) comments from student
Peer discussion

8. 1. What is feedback?
Written comments on coursework and feedback sheets rated top- 95% and 93% agreed
Peer communication rated lowest- 19% rated Moodle comments from other students as feedback
69% of students agreed that staff comments on Moodle were feedback

9. 1. What is feedback?
Compared what students considered to be feedback with what they found helpful
3 significant differences (Mann-Whitney) between….
Student comments on Moodle (p<0.001)
19% agreed as feedback, 47% rated as helpful
Comments written on coursework (p<0.001)
95% agreed as feedback, 76% rated as helpful
Automated feedback from lab quizzes (p<0.05)
74% agreed as feedback, 61% rated as helpful

10. 1. What is feedback?
So, differences between what students see as feedback and what they find helpful.
Peer communication is useful but not viewed as feedback – possibly because less reliable information source?
Written comments seen as feedback, but many don’t find helpful
Automated feedback (shown correct answer and given score) 39% do not find helpful
24% don’t rate written comments as helpful

11. What is Feedback? Digging Deeper…
Student led focus group discussion (Level 1 psychology students).
What is Feedback?
Written comments
yeah, it’s I think what we draw most information from in terms of where to improve and where we’re doing quite well.”
Staff-student Forums
“..good idea but people didn’t really think about things before they posted their questions!
Student discussion on Forums
“how would they have more of an idea than I do?”
Participants tended to see written comments about coursework as the most important type of feedback. Comments were reported as the most detailed source of feedback they received on strengths and areas for improvement. Reflect the survey findings of written comments being the top rated source of feedback..
Students had opportunity to ask questions on forums about coursework in general and in the run up to the hand in of their lab reports (forums were particularly busy then). Positive comments about the staff responding quickly and found answers helpful, but the number of posts was overwhelming. Moodle system means everytime a Q is asked or responded to –students get an -often similar questions were posted repeatedly which led to confusion –as 1 P said “ I wish they would just read!”
Student posts on forums were seen as helpful in the survey, but not classed as feedback- and this was reflected in the FG. Students liked discussion with peers, but were not confident in using this information as the basis for their coursework.

12. 2. What is good feedback? Improvements for next piece of coursework
Comments on structure
Comments on argument
Comments on the content
Comments on critical analysis
Comments that provide encouragement
Clearly relates to grade
Acknowledges effort
Further resources
Markers overall impression
Points out errors, e.g. spelling

13. 2. What is good feedback? Everything!
Direct comments on coursework and suggestions on improvement rated as most important- 98%
Points on errors e.g. spelling 14% rated as unimportant but 70% still rated as important
Comments on effort and providing encouragement also important
How important is it to you for the marker to provide feedback on the following points? 10 points listed and students to respond as unimportant, unsure or important.
Re suggestions for improvement- interesting when tied in with how use feedback- discussed later- 66% highlighted that that helped later when understaood and 37% didn’t help later
Nothing lower than 69%
Interesting that students are unsure whether feedback on critical analysis, further resources, markers overall impression- focus on what I did and tell me how to do it better?

14. What is Good Feedback? Timely Break down of assesment grades
“Psychology was in the end the subject where I got most of the feedback from.”
It’s very transparent […] that’s definitely something that I really appreciate!”
Timely
Break down of assesment grades
“I think feedback should not be only negative points but also showing you ‘this was well done, if you keep that going, it’s a good thing for the future.’”
Positive comments
“explained in a way that you can read it and go ‘OK, I know what I’m supposed to be doing next time to avoid this happening again’”
How to improve

15. 3. Expectations of assessments and experience of feedback
Research skills
Knowledge/comprehension
Structure
Critical analysis
Writing style
What should be included in a good answer and what they have received feedback on.
Compared with matched question in staff survey
Research skills, knowledge/comprehension, structure, critical analysis, writing style

16. 3. Expectations of assessments and experience of feedback
No respondents, staff or student, disagreed that these factors should be in a essay/report
Different when asked if the students have received feedback on these factors or staff had provided feedback….

17. 3. Expectations of assessments and experience of feedback
Knowledge and comprehension
100% of staff provided this feedback
32% of students report not getting it
Writing style
80% of staff commented on this
46% of students did not receive it
100% of staff agreed that they had provided feedback on knowledge and comprehension
68% of students agreed that they had received feedback
80% of staff agree that they had provided feedback on writing style vs 54% of students agreeing they had received it

18. 4. How do you use feedback? Feedback is useful
Clarification for understanding
Grade focus
Pre-submission support and advice
Reflection when new assessment issued
Reaction to poor grade
Marker meeting for clarification
Reaction to negative comments
Grade and feedback connection
Remarking
Marking criteria
Understanding comments
Detailed comments

19. 4. How do you use feedback?
97% of students rated the feedback received as useful
36% only look at the grade
54% look back at comments with new coursework (15% unsure)
49% are unsure how their feedback and grade are related
53% disagreed that feedback helped in understanding
22% agreed that they did not understand the comments on their work (17% unsure)

20. Barriers to understanding
How do you use Feedback?
Consistency
My friend will be marked up for something and it’s something I’ll get marked down for and it’s just really inconsistent.”
Clarity
“Sometimes they only write so short bullet points”
One to one
“She was able to see the person who marked it and get..direct feedback from that.”
Barriers to understanding
Previous exam feedback
“..they published the feedback from last year’s exam….and I did look at that and it helped me I think..”
Criteria in advance
“‘see, this is what first year was, but those are the points we’re looking for now in second year’.”
Feedforward
FG discussion provides some clues as to what they don’t understand and why..perceived inconsistency in marking-actually the grading process and criteria are the same-but perhaps the markers comments are different? Students see a comment as a direct reflection of the grade-whereas a C grade or a B grade could both receive a comment on how to improve on structure-important for reflecting on our own assessment practice.
Sometimes difficult to read handwriting or understand short comments. Many students unaware they can meet the marker if they want further clarification of comments. Possibly reading comments outwith the context of the feedback sheet (which provides further explanation).
Students found the general feedback from the previous exam to be very useful (points on what was good, what to improve) and wanted something similar for their coursework-and the criteria to be clear in advance of handing in the coursework-this is available as a report/essay writing guide on moodle-but students seem to be unaware of it.

21. 5. How do you want to receive feedback?
83% students want comments on coursework but 55% have difficulty using and understanding them
73% of students said they would be happy with online feedback only
Staff are split, 50% staff said they would prefer online only and 50% combination of written comments on scripts and structured feedback
Staff rated an online feedback sheet as 1st choice for provision of feedback

22. How do you want to receive feedback?
Structured
“….you could just take the feedback from from first semester and see ‘oh those are the points they’re looking for. Those are the points I need to work on.’”
Direct comments on coursework
“Particular comments towards a particular paragraph or sentence or whatever but then also the kind of an overview.”
Highlights another contradiction-students want a structured breakdown of their feedback-what was good, what could be improved etc which is useful when looking back at previous feedback. This suggests a structured feedback sheet, yet they also want direct comments on the text of the course work itself (not on a feedback sheet!)
How do we reconcile these 2 demands? –second quote sums this up nicely.

23. 6. How does the feedback you receive differ from before university?
When I understood the feedback it helped with later work
I paid close attention to the feedback I received
I received feedback on drafts
Feedback was provided fast enough to be useful
I used feedback from my friends to improve my work
The feedback for one assignment didn’t help later ones
The feedback I received didn’t help me much

24. 6. How does the feedback you receive differ from before university?
Significant differences found between pre-university and university experiences of feedback
90% agreed that understanding feedback helped with later work at school- 66% at uni
68% agreed that they received drafts at school compared to 15% at uni
83% agreed that feedback was provided fast enough to be useful compared to 44% at uni
More uncertainty around paying close attention to feedback at uni compared to school
Level of agreement decreases in uni with feedback helping later work and feedback being returned fast enough
More uncertainty regarding paying close attention to feedback- school more explicit about how they use feedback?
Receiving feedback on drafts- median increases by 1 point for uni- PAL?

25. Differences between school and university
Class size
“Obviously the whole situation at university, there’s more people, it’s more anonymous in a way…that’s something that maybe stands in the way of individual feedback….”
Formative feedback
…opportunity to write an essay that is not graded but where you can get feedback…Like on a voluntary basis.”
Students expectations of feedback at university seem different, given the large class sizes, and the feeling of anonymity.
They would like more opportunities for formative feedback on drafts of essays/reports.

26. Conclusions & Moving Forward
Broadly large agreement between staff and students on what constitutes feedback, differences in experiences
Many students say they don’t receive comments on content (knowledge and comp)
Style of writing
Match structured staff feedback to student expectations more explicitly
Marking criteria should clearly define the differing assessment criteria (knowledge, critical, writing style etc) that markers use

27. Conclusions & Moving Forward
Students view written comments as important feedback but many do not find useful, e.g. don’t understand the comments and/or how they relate to the overall grade
Many students don’t view peer-interaction as feedback but report finding it useful
Help students use feedback more when attempting future assignments, e.g. reflective exercises requiring students to return to prior feedback before attempting current assignments, peer discussions…

28. Conclusions & Moving Forward
University feedback (compared to school/college) is viewed as:
Less helpful for future work
Slower (mirrors NSS results)
More anonymous (i.e. less connection with staff)
Ways of delivering feedback
Students like written comments but are agreeable to online comments
Staff are split between preferences for written and online delivery
Online feedback could improve the speed of delivery. But, can we provide high-quality feedback without written comments on coursework?
Technology like teleform would allow us to produce structure feedback sheet sent directly to student without comments on scripts. Would substantially reduce admin time, but the danger is that by reducing the time between submission and feedback we degrade the quality of the feedback we provide.

29. Thank you for listening!