1. Efficiency of assessment: initial thinking from the ITEAM project Mark Russell, Liz Gormley-Fleming and Karen Robins The ITEAM project. University of Hertfordshire

2. The ITEAM project Strand A project A (reasonably) tightly defined focus – EVS – QMP – On line assessment and feedback system – Dashboard

3. A matrix Educationally effective Resource efficiency or (1/t)

4. A matrix Educationally effective Resource efficiency or (1/t)

5. What about you? Think about one of your assessments … 1. Where would you locate it on the grid? 2. How might you nudge it to be more educationally effective and more resource efficient ? …Now, next year, the year after … Thoughts in the chat box - please

6. The ESCAPE assessment for learning themes … Good assessment for learning: …Engages students with assessment criteria …Supports personalised learning …Focuses on student development …Ensures feedback leads to improvement …Stimulates dialogue

7. The ESCAPE assessment for learning themes … Good assessment for learning: …Engages students with assessment criteria …Supports personalised learning …Focuses on student development …Ensures feedback leads to improvement …Stimulates dialogue …Considers staff and student effort

8. What about some sample assessments?

9. Essay - Summative (2000 words) Students Activity Number ActivitiesTime (minutes)2060120 1Module aims and learning outcomes10 2Writing assignment brief & marking criteria 50 3Essay title.30 4Team agreement20 5External examiner comments20 6Re drafting assignment.10 7Student preparation (specific)60 8Tutorial support-drafts & feedback*20 per student.40012002400 9Submission-setting up site, guidance20 10Marking*60 per 3 students40012002400 11Feedback*30 per 3 students.2006001200 12Module report30 13Moderation*60 per 3 students.40012002400 14External examiner30 15Assessment office*-inputting results2 per student.40120240 16Exam Board10 17Student tutorial post assessment process* 15 per student.3009001800 Total time in minutes 2030551010730 Time in minutes per student 101.591.889.4

11. EVS test - 1 hour MCQs, 25 questions Students Activity Number ActivityTime (minutes)2060120 1Module aims and LO’s10 2Create and type up 25MCQ questions15 per question375 3Internal Moderation20 4External moderation20 5Update questions20 6Create practice test375 7Student preparation120 8Preparing environment for assessment*15 9Run MCQ test60 10Marking - creating EVS report10 11Assessment office and inputting of grades*1 per student2060120 12Providing feedback*15 13Module report30 14External examiner moderation30 15Module board20 16Post assessment process tutorial support*15 per student3009001800 17Student tutorial post assessment process*120 Total in minutes 156022003160 Time in minutes per student 783726

12. OSCE-summative. Students Activity Number ActivitiesTime (minutes)2060120 1Module aims and LO’s5555 2Case scenario’s120 / scenario120 3Team meeting120 4External examiner20 5Uploading assessment information30 6Student preparation120 7Environment*450 8OSCE* (role of Academic) - 6hrs*10 academics 3600 1080021600 9Marking & feedback*10 per student2006001200 10Moderation*5 per student100300600 11Module report30 12External examiner20 13Assessment office*5 per student.100300600 14Module board10 15Post assessment process feedback*15 per student.3009001800 Total time in minutes 52251382526725 Time in minutes per student 261.3230.4222.7

15. And you …? What sort of things are you doing / (need to do) to help your staff make informed decisions about choosing an assessment type for effectiveness and efficiency ? Thoughts in the chat box - please

16. Developing a tool

17. Plotting the data Numbers of students Time (minutes)

18. Diverging time scales Numbers of students Time (minutes) Assessment type A Assessment type B

19. Overlapping time scales

20. Not prescriptive!

22. And so … In what ways might these tools (and approaches) help or hinder your work? Thoughts in the chat box - please

23. Does (say) 20 hours of our effort yield the same learning payoff?

24. An example with EVS

28. The ESACPE assessment for learning themes … Good assessment for learning: …Engages students with assessment criteria …Supports personalised learning …Focuses on student development …Ensures feedback leads to improvement …Stimulates dialogue …Considers staff and student effort

29. So let’s always use the least time consuming?

30. Alas, it’s a little (a lot) more complex and messy than that.

31. The purposes of assessment are important too! 31 Summative High Stakes Measure learning Formative No / Low Stakes Motivate learning Educative Auditing

32. The purposes of assessment are important too! 32 Summative High Stakes Measure learning Formative No / Low Stakes Motivate learning Feedback rich? Linked (in-process) tasks? Regular tasks? Error does not lead to punishment Need for feedback is questionable (MBR) One-off tasks End of process tasks? Educative Auditing

33. Assessment patterns Beyond the single assessment task Beyond the single assessment task

34. Some alternatives…

35. And many more …

36. Programme view of assessment

39. Far reaching implications Our work (as teachers) extends beyond our classroom. How might we measure this? Performance on singular assessment? Success in Module? Success in Year? Success in Programme? Institutional reputation?

40. Conclusion – to date Numbers ‘might’ be countable Numbers ‘might’ be meaningful (process improvement, stimulating dialogue, moving in the ‘right direction’)

41. A possible conclusion – to date Numbers ‘might’ be countable Numbers ‘might’ be meaningful (process improvement, stimulating dialogue, moving in the ‘right direction’) But…

42. A possible conclusion – to date Numbers ‘might’ be countable Numbers ‘might’ be meaningful (process improvement, stimulating dialogue, moving in the ‘right direction’) But…

43. Some questions for you to ponder… What is the difference in learning payoff between a 4000 word and a 2000 word essay ? Is a single 4000 word essay more beneficial than (say) two 1500 word essays? What are the resource implications of the above variants? What is the difference in learning payoff between a 4000 word and a 2000 word essay ? Is a single 4000 word essay more beneficial than (say) two 1500 word essays? What are the resource implications of the above variants? What are the various processes of assessment doing? Where might you best place your efforts to reduce the resource burden while keeping the effectiveness of the assessment? What are the various processes of assessment doing? Where might you best place your efforts to reduce the resource burden while keeping the effectiveness of the assessment? In what ways are you facilitating meaningful discussions about both resource efficiency and educational effectiveness?