1. Supplementing Classroom Instruction with On-line Learning Opportunities David Reuman Report on ITEC Course Development Grant Spring 2014

2. Course Context PSYC 221 Research Design and Analysis in Psychology Foundation course for Psychology majors Meets Numerical and Symbolic Reasoning distribution requirement Taught every semester with 25-35 students Class meets 3 times per week

3. Goals for Enhancing Student Learning through Technology By supplementing classroom instruction with on- line learning opportunities, students may Re-listen to portions of lectures that were initially unclear Re-integrate concepts after hearing initial lecture, completing assigned reading or problem set Encounter course content from a new perspective Process additional examples

4. Examples of On-line Supplements Narrated slide-shows “Survey Sampling: How Representative is that Sample?” http://www.youtube.com/playlist?list=PLEE4D4FFF2F319216 “Wealth Inequality: Importance of the Shape of a Distribution” http://www.upworthy.com/9-out-of-10-americans-are-completely-wrong- about-this-mind-blowing-fact-2 http://www.upworthy.com/9-out-of-10-americans-are-completely-wrong- about-this-mind-blowing-fact-2 Modules from Carnegie Mellon “Open Learning Initiative” for a course on statistical reasoning http://oli.cmu.edu

5. Specific OLI Modules Used 1: Measures of central tendency and variability 6: Normal distributions 7: Sampling distributions and the central limit theorem 9: Confidence intervals for a population mean 10: Hypothesis testing – Type I and Type II errors 10: Hypothesis testing for the population mean 11: Hypothesis testing – Two independent means and matched pairs 11: 1-Factor ANOVA 2: Correlation (Linear relationships)

6. Did The OLI Supplement Make a Difference to Students? “How often this semester did you visit the Open Learning Initiative (OLI) website for “Statistical Reasoning”? 26 percent (6/23) never visited 74 percent (17/23) visited, ranging from once to more than 5 times

7. Did The OLI Supplement Make a Difference to Students? “How often did you answer questions posted on the OLI website and get online feedback”? 48 percent (11/23) never sought feedback 52 percent (12/23) did, ranging from once to more than 5 times

8. Did The OLI Supplement Make a Difference to Students? “If you visited the OLI website at least once, how helpful did you find the OLI modules”? Not helpful at all6 percent Minimally helpful18 percent Somewhat helpful47 percent Very helpful29 percent Extremely helpful0 percent

9. Did OLI Modules Increase Absenteeism? Apparently not. Absenteeism was not significantly different in Spring 2013 semester (without OLI) compared to Fall 2013 semester (with OLI). Number of visits to OLI website was not significantly correlated with (self-reported) number of missed class meetings.

10. Did OLI Use Improve Student Performance? Unclear. Number of visits to OLI website, whether a student used the OLI website to get feedback, and student evaluations of the helpfulness of the OLI website were all uncorrelated with students’ expected grades.

11. Next Steps Repeat this trial. Develop original online modules for course topics that students find difficult, but for which no modules currently exist. Involve teaching assistants as module developers.

12. Acknowledgments Thanks to Lisa Dierker and Wesleyan’s summer 2011 workshop on “Inquiry-Based Approaches to Statistics”. Thanks to ITEC for a course development grant. Thanks to Jean-Pierre Haeberly for adding special items to the online course evaluation for PSYC 221.