1. John Ogier Making summative unit surveys totally summative - risks or opportunities

2. Survey Timing Collings AHEEF 2007 I misheard Davids method!! What are the implications of a long survey window? Into and after exams? Post grades? Response demographics?? Immediate response Post exams Post grades General - gender, grades etc Online Survey Experiment Wanted a wide range of large units Divide each randomly into 4 groups But high female response rate AUSSE 2007 - so maintain gender balance Got small selection of units Group 1 - normal pre-exams 15 days Group 2 - day after exam 15 days Group 3 - day after grades 13 days Group 4 - open entire 7 weeks Day 1 - personalised invitation email Week before close - reminder email

3. Also some other units - MATH200, ECON200 and 300

4. Responses

5. Variables C_FdBck I received helpful feedback on my progress C_Asmt The assessments in this course measured my learning effectively C_OAll Overall, this was a good quality course T_OAll Overall, the lecturer is an effective teacher Gender F, M Group 1, 2, 3, 4 (Survey Group) NumGrade -1 to 9 (A+ = 9 through to E = -1) PostGrades PG (responded post grades/mid-year test return), NA (prior) Week W1 (responded Week 1), Rem (post-reminder email), NA (between) Also knew: - Ethnicity, International Student status, Citizenship, Age, intended Qualification, Overall GPA, Response Day & Time and Time taken

6. First analysis

7. What about grades? Figure 4: ANOVA – C_FdBck, C_Asmt, C_OAll and T_OAll by Group

8. The dangers of means Dommeyer, Baum, Hanna & Chapman, 2004 - online surveys do not produce significantly different mean evaluation scores from paper UC Paper and Online survey history support this Not just means - also Std Deviations But are the responses representative of the whole group?? (Nulty, 2008)

9. Gender - a response bias? No significant diff @ 95% CI in Likert means or across Groups by Gender

10. What about grades? Across the main units analysed: Bivariate fits Correlations for Overall Qtns Course R 2 = 0.047 Teaching R 2 = 0.0045 A+ is 9 ………… E is -1

11. Broken into Groups Survey timing now apparent! Group 2 - post exams Group 3 - post grades

12. Why not the full picture?

13. Where to from here? Oct 6 - online surveys - 40 units Group 2 runs until 15th Dec. Grades released 5th Dec