1. 8 th Programming Workshop Adjusting students' attitudes so that they benefit from feedback Quintin Cutts, Peter Saffrey Dept of Computing Science, Glasgow Uni Steve Draper Dept of Psychology, Glasgow Uni Emily Durrant Centre for Confidence and Well-being, Glasgow Contact: quintin@dcs.gla.ac.uk

2. 8 th Programming Workshop Carol Dweck - Mindset Two opposing attitudes to one's ability to learnTwo opposing attitudes to one's ability to learn Fixed mindsetFixed mindset –Ability is a fixed commodity –Performance level demonstrates ability level –All feedback interpreted as a measure of ability –Failures hit motivation and self beliefs hard Growth mindsetGrowth mindset –Ability can be grown through effort –Feedback interpreted as information to assist growth –Failure is an opportunity to gain feedback & to learn Experiments show that Growth M/S can be cultivatedExperiments show that Growth M/S can be cultivated

3. 8 th Programming Workshop Messages about learning make a difference Mueller and Dweck 98Mueller and Dweck 98 –"You must be smart at those problems" –"You must have worked hard at those problems" Simpson, Open University 07Simpson, Open University 07 –Multiple short tutoring phone calls –Strongly emphasised growth mindset and personal motivation in every call Dar-Nimrod and Heine 06Dar-Nimrod and Heine 06 –Exposure to experience-based learning theory affected female performance in maths Aronson, Fried & Good 01Aronson, Fried & Good 01 –Writing exercise telling others about the growth mindset

4. 8 th Programming Workshop Mindsets & Programming SuggestionSuggestion –Learning to program at university can easily generate a fixed M/S Very high number of failure messagesVery high number of failure messages School & university programming widely differentSchool & university programming widely different Very wide ability range in Yr 1 classesVery wide ability range in Yr 1 classes ParadoxicallyParadoxically –Experienced programmers must have a growth m/s –but a fixed mindset towards learners– "programmers born not made" –(Heslin, Wandewalle & Latham – Managers mindset affects quality of coaching)

5. 8 th Programming Workshop 3 Implementations of Dweck Explicit Mindset Training - (M)Explicit Mindset Training - (M) –4 x 10-minute teaching sessions, led by tutors –Blackwell, Trzesniewski & Dweck (2007) Crib sheet / Wiki - (W)Crib sheet / Wiki - (W) –originated from McCartney et al (2007) – "35 ways that students get unstuck" Feedback 'inducement' - (F)Feedback 'inducement' - (F) –Message on feedback sheet encouraging engagement – –Remember, learning to program can take a surprising amount of time & effort – students may get there at different rates, but almost all students who put in the time & effort get there eventually. Making good use of the feedback on this sheet is an essential part of this process.

6. 8 th Programming Workshop Experiment ContextContext –Level 1, 2 semester (24 wk) course –2 lectures per week + 2hr lab/tutorial - 12 tutorial groups –Wk 6, Wk 13 class tests and Wk 12 lab exam Interventions spread across groupsInterventions spread across groups –2 Control, 2M, 2W, 2F, 1MW, 1MF, 1WF, 1WMF Tutor training sessions and deliveryTutor training sessions and delivery –1 for Wiki – used from Wk 2 –4 for Mindsets – delivered in Wks 2-5 –no training for F – sheet given in Wks 3, 5, 7, 9 MeasuresMeasures –Wk 1 and Wk 7 measures of mindset, self-efficacy, hope, +/- affect –Class tests and lab exam –Self-reported measure of previous experience

7. 8 th Programming WorkshopResults Week 6 – first Class Test (CT1)Week 6 – first Class Test (CT1) –W, F main effects against CT1, p < 0.05 Week 13 – second Class Test (CT2)Week 13 – second Class Test (CT2) –F main effect against CT2, p < 0.005 Could this be due to imperfect random assignment of conditions?Could this be due to imperfect random assignment of conditions? PREV[0,1,2] against PREV[3]PREV[0,1,2] against PREV[3] –p < 0.0007, effect size 0.78, mean diff 1.5 coarse grades Uneven distribution of PREV[3] among groupsUneven distribution of PREV[3] among groups Test effect of F against PREV[0,1,2] and PREV[3] separatelyTest effect of F against PREV[0,1,2] and PREV[3] separately –adjusted effect size 0.36, mean diff 6%, or 0.6 of a coarse grade Time of day not significant against FTime of day not significant against F –but independently significant, effect size 0.5, or 0.9 of a coarse grade

8. 8 th Programming Workshop What to make of it… M and W imperfectly appliedM and W imperfectly applied –so cannot draw any strong conclusions from results F definitely delivering a result – Why?F definitely delivering a result – Why?

9. 8 th Programming Workshop Interpretation by student of feedback Dweck suggests one very clear interpretation:Dweck suggests one very clear interpretation: –"This is saying something about my ability" but experience regularly shows strategic interpretationbut experience regularly shows strategic interpretation –"It's saying something about whether I've put enough effort in" We provide f/b hoping to enhance deep learning…We provide f/b hoping to enhance deep learning… –"It's advising me how to improve my subject skills/knowledge" PDP training, and evident student behaviour, suggestsPDP training, and evident student behaviour, suggests –"It's showing me how my study skills could be improved" Other interpretations are:Other interpretations are: –"The marker's an idiot, the feedback is wrong" –"This has nothing to do with me, it's random"

10. 8 th Programming Workshop Action follows interpretation AbilityAbility –fixed mindset – avoidance, drop-out etc Amount of effortAmount of effort –if high enough, move on to something else, given my overfull life Improve subject skills/knowledgeImprove subject skills/knowledge –read feedback closely, incorporate into my understanding, via a range of practices Improve study skillsImprove study skills –reflect on activity, identify and carry out changes Mark/feedback is wrongMark/feedback is wrong –initiate the complaints procedure RandomRandom –Do nothing

11. 8 th Programming Workshop

12. Sheet is directive and informative… … right at the point of receipt of the feedback… right at the point of receipt of the feedback We state the interpretation we think is most importantWe state the interpretation we think is most important –Learning to program takes time. You'll get there with sufficient effort. It's worth making use of this feedback. [and we almost hide the 'mark'] We give appropriate actionsWe give appropriate actions –"Look at items…" –"Look at items…" –"F/b points – mostly worth reading, even if not identified for you" …and the feedback is plentiful…and the feedback is plentiful –far more than any tutor could hand write –Whole sentences, real explanations, with references Feedback sheets become a potentially major resourceFeedback sheets become a potentially major resource –Not cheap to develop, but speeds up tutor marking time

13. 8 th Programming Workshop Web questionnaire – 35 responses 70% found the sheets (v.) helpful70% found the sheets (v.) helpful 50% motivated to work harder50% motivated to work harder 55% find sheets helped later55% find sheets helped later CommentsComments –Although the program may have worked, the sheet provided comments on how you could have improved the program… –… gave me the confidence and then the joy to continue to work hard… –I was getting problems outlined - I started working harder on next programs, so I could sort these problems before submission. I have started meeting deadlines much more efficiently. –Plenty of good common sense pointers are even now continuing to be helpful. Especially on structure. –Feedback in this course is better than that in other courses. =].

14. 8 th Programming Workshop Conclusions Students amenable to subtle messagesStudents amenable to subtle messages –and in their absence, apply their own interpretations/actions We need to promote our attitude towards learningWe need to promote our attitude towards learning –in this case, on the values we place on feedback –and on the actions we expect them to take We're now applying the rubric across the classWe're now applying the rubric across the class –and will look for any effects in summer diet We'll be exploring the other two interventions furtherWe'll be exploring the other two interventions further –direct teaching about mindsets clearly valuable in other contexts –giving students strategies for solving problems is a key part of the growth mindset