1. Marking and Assessing Andy Wilson Loughborough University

2. Purpose To help you to… mark assess provide feedback …in line with the expectations of a British university.

3. Why do we assess? Classify or grade students Enable student progression Guide improvement Indicate what the module is about Inform students’ choice of options Provide feedback on our teaching Motivate students Provide statistics for the programme/course Provide evidence for quality purposes Etc.

4. Why do we assess? 2 Summative ~ summary measure of performance Formative ~ feedback to help students improve Informative ~ guidance on what is expected.

5. What do we assess? How well the students have met the requirements of the module.

6. A MORAL model… MethodsTeaching techniques OutcomesILOs ResourcesFacilities, equipment, etc AssessmentTechniques of assessment LearnersStudents’ characteristics An iterative design process. All these must be consistent. If you don’t assess it then students won’t see it as important.

7. Assessment assumptions Students can fail The good students should have the chance to show that they are good There aren’t any trick questions You mark what you see, not what you hope to see One person doesn’t decide their mark You should mark using the full range Students know how they are going to be assessed The assessment criteria are clear to all Abilities – and marks – are distributed reasonably normally How can we make these happen?

8. “assessment criteria are clear” Imagine these questions from a student… “What will get me a mark of 90%?” “What will get me a mark of 70%?” “What will get me a mark of 50%?” “What will get me a mark of 30%?” How would you answer them?

9. “reasonably normally”

10. Distributions compared The following slides show BUE and Loughborough mark distributions The right-hand columns show the Programme marks.

11. Loughborough engineering

23. Comment The BUE programme distributions are sometimes very different from those at Loughborough…

24. Comment 2 These are, of course, the aggregate of module marks In a British university the module marks would show a somewhat similar distribution In the BUE the module marks are markedly different These very different module marks can cancel each other out to produce less extreme programme distributions.

25. Module marking “Firstly, a large number of modules, particularly at Year 1 level, had very low averages (with some below credit level) and large numbers of students failing to achieve credit.” “Secondly, a large number of modules, particularly at Year 3 level, had very high averages (above 70 or even 80%) and disproportionate numbers of students gaining grade A.” “In some cases the same group of students was achieving averages below 40% or above 70% in different modules.”

26. Lessons from this BUE marking tends to be: Inconsistent across modules Within too narrow a range Not sufficiently challenging Not sufficiently discriminating Not sufficiently scrutinised Critical to this is the nature of the questions asked Consider this hierarchy…

27. After Bloom Create Evaluate Analyse Apply Understand Remember

28. Blooming obvious The good students will be able to operate at these higher levels The weaker students will not So we should ask some of our questions at this level… …with more of these higher level questions in the later years.

29. Learning verbs (Hall, 1976) Define Describe Identify List Match Name Order Recognize Recall Appraise Argue Assess Compare Contrast Defend Discriminate Evaluate Explain Justify Predict Summarize

30. Loughborough exam papers See the examples of module specs and exam papers.

31. Feedback Performance without feedback misses opportunities and reduces motivation It makes it a game of Battleships Compare… “This is wrong.” “This has some strengths, such as xxxx and yyyy. What it needs to get a better mark is…”

32. Delivering the assumptions Assess against your ILOs Include some higher level questions Be clear about your criteria… …and share these with your students Show them some typical questions beforehand and discuss them in class Use a wide range of marks to discriminate between students… …which may mean that you have to go back and adjust your marks.

33. Marking practice Mark this essay giving it a percentage mark.

34. Resolutions?