1. Transforming tertiary education: overview of an AKO Aotearoa project involving a large 100-level Cell Biology class. Rosie Bradshaw, Maggie Hartnett, Gemma Cartwright, Natalie Burr, Ewen Cameron, Ben Kennedy & Zoe Jordens Applying the Carl Wieman Science Education Initiative (CWSEI) Model within the New Zealand Tertiary Environment

2. Students have diverse aims, background and abilities

3. The team Six science classes: four in Geology at Canterbury and two in biology at Massey Mixed team- Education specialists, motivated science lecturers and research assistants. The aim Improve student engagement and learning by: - implementing ‘interventions’ - measuring the effects on engagement & learning based on

4. The method Interventions: Improved learning goals and alignment Emphasis on relevance Interaction in lectures: activities and ‘clickers’ Measuring effects: classroom observations engagement attitude survey ‘knowledge’ survey learning

5. The cell shown here is a human diploid cell destined to undergo meiosis to make an egg cell. The cell is heterozygous for genes A and D. Assuming no crossing over occurs, choose the diagram that best represents what this cell would look like in metaphase of Meiosis I. DD dd AA aa DdAa B Dd Aa C ad A AD DDdd AA aa D E

6. The method 162101 Biology of Cells PRE POST semester 1 2011 1 2 3 surveys 162212 Microbial World PRE POST semester 2 2010semester 2 2011 1 2 surveys 1 2 surveys

7. Results: Classroom observations Observe lecture material and teaching style Measure students engagement through observation

8. Observe lecture material and teaching style Measure students engagement through observation Results: Classroom observations

9. Skimmed content Overview Movie near end of lecture enthusiasm Time (mins)

10. Results: Classroom observations Lecture Link to lab

11. Results: Attitude survey Significant shifts compared to ‘expert-like attitudes ’ (for all of the example Qs ‘strongly agree’ is expert-like and therefore favourable) Category (4 of 7 shown) FavourableUnfavourableExample of question (8 – 10 Qs in each category) Real World Connection ns To understand biology, I sometimes think about my personal experiences and relate them to the topic being analyzed. Enjoymentns If I had plenty of time, I would take a biology class outside of my major requirements just for fun. Problem- solving effort ns I actively try to relate what is presented in biology to what I have learnt in other courses. Reasoningns When not pressed for time, I will continue to work on a biology problem until I understand why something works the way it does. shifts in attitude over the whole semester : ns not significant; got worse (less ‘favourable’); got better (less ‘unfavourable’)

12. Results: Knowledge survey Questions based on: Survey 1Survey 2Survey 3Mean gain PRE 10 Qs 21.4 ± 14.332.7 ± 16.039.3 ± 18.511.3 ± 16.1 POST 10 Qs 20.9 ± 15.227.1 ± 14.641.6 ± 19.114.5 ± 19.8 PRE POST semester 1 2011 1 2 3 Values are % test scores: mean ± SD for 161 matched surveys (questions were designed to be difficult!) Two-tailed paired T-test of significant difference between pre and post mean gains shows no significant difference (P = 0.103)

13. Results: Knowledge survey Values are means of matched % test scores Questions based on: Mean % gain final exam >80% Mean % gain final exam 60-79% Mean % gain final exam 40-59% Mean % gain final exam 20-39% Mean % gain final exam <20% Mean % gain ALL students PRE 10 Qs 20.015.88.61.7-1.711.3 POST 10 Qs 32.915.913.62.21.714.5 P (paired T-test) (n) 0.007 (14) 0.964 (64) 0.424 (59) 0.941 (18) 0.788 (6) 0.103 (161) Conclusions: Most of the ‘post’ (interventions) gain was in the top student group More needs to be done to facilitate learning in other groups! BUT - there are many limitations to this survey. - deleting toughest 2 Qs from each survey only 40-59% show sig gain

14. To assist my learning in lectures, I would prefer: A) 'Traditional style' information delivery lectures B) Mixture of clickers/interactive questions and 'traditional style', with some recommended reading prior to lecture. C) Completely interactive (clickers/activities/discussion) with mandatory pre-reading and on-line pre-lecture quiz. Results: A 5%, B 70%, C 25% (n=207) But what do students think? Clicker-vote at end of 162101 2011:

15. Conclusions No increase shown in student engagement BUT - engagement declines over the semester - positive feedback on use of clickers Significant increase in knowledge gain after interventions BUT - only with top performing students (final exam >80%) - mid students (40-59% FE) gained when toughest Qs removed. Mixture of attitude shifts BUT - some attitudes change over the semester anyway (eg. enjoyment) So – was it worth it? Overall I think my teaching has improved -Better learning goals, etc -Improved style of interactive questions & many ideas of how to improve more…

16. Acknowledgements Zoe Jordens – Massey Project Leader Maggie Hartnett – Education Consultant Gemma Cartwright, Natalie Burr – Research Assistants Ewen Cameron – T&L director, MUHEC approval Ben Kennedy - Canterbury Uni. and overall project leader Applying the Carl Wieman Science Education Initiative (CWSEI) Model within the New Zealand Tertiary Environment