The Mechanical Vibrations Quiz An example of formative assessment Ines Lopez Arteaga 19th March 2012

Overview Background Goals Approach Results Conclusions

Course information 2nd year Mechanical Engineering Number of students +/- 200 Course organization: 4h/week Lectures 2h/week Guided self-studies Assessment: written examination with book Material: course book and handouts 2

Situation 2008 Average first-time pass rate : 45% 3

Improvements : New course slides 2007: New examination format including theory 2008: Use of blackboard instead of slides Exercises and exams previous years with solutions available in Studyweb 4

Situation 2008 Lecture attendance dropped from about 130 to 70 during the course Only 30% of the students attend guided self-study Students do not study course book Consequence: Students know how to solve the exercises … … but do not understand the concepts behind 5

Questions How to stimulate the students to be active with the course book and other material? How to repeat/review important concepts without being “repetitive”? How to give them feedback on their progress during the course? …. And all this at a “low cost” in terms of structural time investment. 6

Situation 2012 Average first-time pass rate : 45% Average first-time pass rate 2009&2010: 70% 7

Goals Activate students Give feedback Monitor student progress Review concepts

Alternatives 9

10

Approach Ask short questions during lectures. Use sharepoint applications in Studyweb to collect answers and show statistics. Give the students a prize in the form of 0.5 extra point. Add a competitive element to make it more fun for the best students. The questions should not be too difficult. The goal is to help them learn and not to let them fail.

Course design: weekly structure 12 Day 1Day 2Day 3 Lecture (45min)Lecture (1 hour)Guided self-study (45 min) Lecture (45min)Guided self-study (45 min) QUIZ (30 min)

Quiz design During lecture Students login to Studyweb Lecturer displays 6 to 8 (multiple choice) questions on the screen Students answer to survey on Studyweb and send their answers after the last question Lecturer shows the students’ answers on the screen and gives feedback on the correct answers 13

Quiz design After lecture Lecturer exports individual answers to Excel and calculates individual scores Lecturer publishes weekly scores in Studyweb Week 8: Final Quiz Best students get a prize Every student with a final score >55% gets 0.5 point extra 14

15

Main ingredients of quiz Peer-instruction: Students discuss with peers during quiz Formative assessment: Inmediate feedback during lecture Long term feedback in the form of published individual scores Reward: Additional 0.5 point on top of examination mark 16

The Mechanical Vibrations Quiz Test 1 Week 1

Question 1 How many degrees of freedom (DOF) has this system? a)1 b)2 c)3 d)0

Question 2 How many degrees of freedom (DOF) has this system? a)1 b)2 c)3 d)0

Studyweb surveys 20

Inmediate feedback 21

Individual results 22

Publish weekly scores 23

Final Quiz

Situation 2012 Average first-time pass rate : 45% Average first-time pass rate 2009&2010: 70% 25

Conclusions student survey Most students are positive about the quiz. Most students say that the extra 0.5 point motivates them. The opinions about the competitive component and DAF sponsoring are very diverse. Some students indicate that discussing with their peers helps them understand better.

Conclusions student survey 56% respondents didn’t start studying earlier than normal, 21% did. 20% prepared better for the lectures, 58% didn’t. Only 13% prepared better for the guided self-study, whereas 57% didn’t. 54% thinks the feedback after the tests was a good indicator of the aspects they need to improve, 22% does not.

Facts Lecture attendance significantly higher than in previous years. Guided self-study attendance significantly higher than in previous years. Only 8 students passed in both 2009 and 2010 due to the extra 0.5 point. Students make less conceptual mistakes in their written exams.

Conclusions Most students say they did not change their studying attitude. But the exam results are significantly better than in previous years. Students are “forced” to think about a number of concepts themselves during the tests, which helps them understand better without giving them the impression they are working harder. Discussing with their peers helps them settle their knowledge better. 29

The Mechanical Vibrations Quiz An example of formative assessment Ines Lopez Arteaga 19th March 2012

Results 31 Average first-time pass rate 2009&2010: 70%