1. Retrospective review of e-Assessment data to enhance student feedback Dr D. Rhys Gwynllyw Rhys.Gwynllyw@uwe.ac.uk Dr Karen L. Henderson Karen.Henderson@uwe.ac.uk www.cems.uwe.ac.uk/dewis Senior Lecturers & UWE L&T Fellows Dept. of Engineering Design & Mathematics University of the West of England, Bristol

2. Retrospective marking A second stage of marking at the end of the assessment period. Marks typically not altered. Facilitates: –Detection of common student errors (CSEs). –Improved targeted feedback to the students, –Evaluation of the e-Assessment, –Improvements to the next run.

3. Example of a common student error (mal-rule)

4. Common Student Error Analysis Analysis of loss-less data to look for possible CSEs Retrospective marking then searches the whole data for all instances of this potential CSE. If triggered, this detection is reported to the student and explains why their approach is incorrect.

5. Overview of the e-Assessment 30 credit module: Stress & Dynamics (250 first year students) Stress: four e-Assessments worth a total of 12.5% Concentrate on the first question of the first e- Assessment here

6. Randomly generated parameters Two inputs (closed form decimals)

7. Work through of the question

8. Feedback to Question 1: Current feedback gives a work through of the question Can we point out where the student has gone wrong?

9. Searching for CSEs from the e-Assessment data

10. DEWIS Loss-less data e-Assessment system Extensive reporting

11. Performance flags -1: not answered 0: incorrect 1: correct

12. DEWIS reporter (Question 1) Flag value = 1, question correct Flag value = 0, question incorrect First input incorrect, Second input correct Flag value = -1, question not answered Total of 360 attempts Only a selection shown here

13. Values for R A and R D entered the wrong way round 35 students answered both parts of Qn 1 incorrectly. Only a selection shown here. Performance Flag search

14. Common student error 1 Values for R A and R D entered the wrong way round. Introduce a 3 rd flag for this question: = 1 if made this mistake = 0 did not make this mistake Re-mark for all students’ attempts

15. 10 students made this error If ( flag3 == 1 ) then “We notice that you have entered your reaction forces the wrong way round. Remember that taking moments about A allows you to find the reaction force at D”

16. Common student error 2 7 students made this error

17. Award continuation marks? The student has R A + R D = F 1 + F 2 Is this worth 1 mark? True for 29 out of 35 students

18. E-Assessment Evaluation/Results (Question 1) Overview of errorNumbers 1.Interchange R A and R D 10 students 2.Incorrect distance for AC used7 students 3.Used/muddled first attempt data4 students 4.Wrong value of L used2 students 5.Rounding error1 student 6.Forgot to divide by L1 student 7.Unexplained10 students

19. Conclusions Loss-less data, algorithmic e-Assessment system facilitates the process. –DEWIS: www.cems.uwe.ac.uk/dewis Enhanced feedback is beneficial for all future students taking the tests.

20. Future work Extend to other e-Assessments Student views