1. 1 Continuous course improvement through feedback - experiences and challenges Karin Väyrynen Tietojenkäsittelytieteiden laitos Business Process Modeling

2. Background of Business Process Modeling First implementation of the course Each lecture contained at least 2 activating lecture tasks Mandatory for all 2nd year students at TOL No experience on actual workload of exercise tasks 130 enroled 101 started to participate actively 97 finished exercises Opintori 2012, Karin Väyrynen 2 -> continuous course improvement by collecting feedback throughout the course

3. Feedback collection in the BPM course 3 L1L2 L4 L3L5 L7 L6 L8L9 E1 E2E4E3 Discussion List in Optima Anonymous Feedback Box in Optima Feedback concerning a certain lecture/exercise Feedback concerning course in general Feedback throughout the course Weboodi written feedback oral feedback … and the quality of the assignments! Opintori 2012, Karin Väyrynen

4. Lecture task Discuss for 1 min in teams of 2-3 What challenges have you encountered when collecting feedback? Opintori 2012, Karin Väyrynen 4

5. Challenges Motivating students to give feedback Explain why you collect feedback Whether and how to react to offensive or unjustified feedback? Conflicting feedback When and how to collect feedback During or after the lecture? Anonymous or with names? Keeping the big picture in mind Opintori 2012, Karin Väyrynen 5

6. Lessons learned Emphasize students responsibility Explain how different feedback channels should be used Reserve enough time for collecting feedback responding to feedback Continuous feedback cycle 6 Opintori 2012, Karin Väyrynen Collect feedback Evaluate feedback Make changes based on feedback visible Students see their opinion matters Students are willing to give feedback again

7. Collecting continuous feedback can be frustrating … …and students will ALWAYS find something to complain about … 7 Opintori 2012, Karin Väyrynen

8. … but it really seems to be worth the effort! Kiitos! Pidin kurssista :) Ja näit todella paljon vaivaa kurssin eteen. Great Job!! I think this was one of the best courses I have attended. The way the course was held helped me to learn things better. For once a teacher who actually seemed to like what she was doing. Your enthusiasm towards the topic was inspiring. Keep up the good work and continue smiling :) I hope that there will be more courses which will be held like this. 8 Opintori 2012, Karin Väyrynen

9. 9 Summary of feedback collection methods in course Business Process Modeling and Examples of Feedback I received

10. Summary: Feedback collection in the course Feedback concerning a certain lecture Feedback forms after lecture concerning that lecture (3 times) Feedback brainstorming session (students discuss a topic in small teams) Oral feedback after lecture / exercise meeting Feedback possibility during whole duration of the course Discussion list in Optima Anonymous feedback box in Optima Overall course feedback Feedback form after final lecture Same feedback form in Optima open also after the final lecture Feedback in WebOodi Other ways of getting feedback The models the students created in the exercises Each student will return an analysis of his/her own work- and learning- process in the course (feedback on whether my time estimates were OK) 10

11. Examples of feedback received in lecture feedback form after L1 – things that hindered learning In the way the lecture was held The no laptop rule Some questions I didnt hear or they came so fast A bit slower please, taking notes slows me down a bit. Also a few more examples would be nice Too much arguing of how to organize this course Some questions I didnt hear or they came so fast Muuten hyvin, mutta itse olen tottunut hakemaan/katsomaan luentokalvot tietokoneelta. Toivoisin, että yli 20-vuotiaat yliopisto- opiskelijat saisivat itse päättää miten/miksi opiskelevat In your own behavior Forgot to take a notebook for paper notes (not usually needed at TOL) Being tired Could have slept more last night Well… my attitude about the laptop use Sometimes my thoughts wander I was too anxious to answer to any of the question, even though I might have known the answer Hard to maintain 100% concentration on less than 8 hours of sleep Coming late 11

12. Examples of feedback received in lecture feedback form after L1 – things that supported your learning In the way the lecture was held Ive never realized how distracting other peoples laptops really are The lecture itself seems really good It helps that the language is English Energinen luennoitsija joka kiinnostunut aiheesta, youtube video The whole time lecture was engaging. It felt like there was interaction with the audience Discussions and interactivity In some way, the strictness (kuri) of the lecture is quite welcomed. It keeps me focused Hyvät, havainnolliset esimerkit Good presentation In your own behavior Concentrating on what the lecturer actually says I always keep an open mind Writing notes I focused more to lesson than usual because couldnt use my laptop I was listening all the time. Maybe because my laptop was in my backpack Being generally acceptive to studying Putting away my laptop High level of interest Listened carefully, paid attention A cup of coffee 12

13. Examples of feedback received in anonymous feedback box I just want to thank you for the enthusiastic attitude and the very informative emails you are sending. You are doing very good job at informing students. But, I was wondering could it be possible to include like TL;DR(Too long, Didn't read) in your emails? Since some people are really busy and they don't have time to read long emails. They just want to get the information fast and short. It would be really helpful Too much group work.. Impossible to find time to work as a group and separating into subteams does not help, we still have know how to use the other tool and compare them and write the analysis.. Possible and fair within these time limits? Don't think so. Considering the laptop ban...I'm sure many will write about this, but i will too. I, for example, read all the lecture material from my laptop, since i can't quite see all the way to the screen. And no, i can not come closer to the screen, because i get headaches from staring upwards for 2 hours. Not that it matters, but the lecturer can't even really ban laptops anyway, all it does is simply makes one look bad in the eyes of students. There should be some kind of list of points in Optima, where you could check out your own point sum! Also, If you have up-to-date list of lecture participation, it would be great to have it here in Optima also.. This kind of arrangement was used in 'Johdatus tutkimustyöhön' -course, and it was a great way to follow your own progress during the course. 13

14. Examples of changes made based on feedback Up-to-date list of lecture attendances in Optima Instructions in pdf format in Optima instead of doc Decreased length requirements for one exercise analysis report (the students really had too little time to do the report) Creating workspace for student teams in Optima Adding a brief summary of the most important things in my information emails to the students (my emails are too long and contain too much information) Give additional instructions about exercise tasks BPM 2012, Karin Väyrynen 14