1. Innovative Teaching: A Test of the Impact of the Khan Academy in Business Statistics Courses Dr. Füsun F. Gönül Associate Professor of Marketing, School of Business, Slippery Rock University Slippery Rock, PA, fusun.gonul@sru.edufusun.gonul@sru.edu Dr. Roger A. Solano Associate Professor of Management, School of Business, Slippery Rock University Slippery Rock, PA roger.solano@sru.eduroger.solano@sru.edu ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

2. Research Objectives To test the impact of Khan Academy, on student performance, in Business Statistics To glean insights on teaching statistics (impact of Khan Academy on Faculty) To assess interactive learning via YouTube videos In current times, when higher education institutions face a reduction in public funds, it becomes increasingly important to exploit such free resources, in order to produce graduates with sufficient quantitative/analytical skills to help them succeed in the workplace. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

3. Motivation Literature documents that math background improves grades in statistics courses. However, currently, SRU does not have a Math prerequisite for the Business Statistics course. If Khan improves grades, it can be made to replace the prerequisite. We test to find out if it does improve grades. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

4. Literature Review Dickson, G. W. and A. Segars (1999), "Redefining the High-Technology Classroom," Journal of Education for Business, 74 (3), pp. 152-156. Dijksman, J. A., and S. Khan (2011). “Khan Academy: the world's free virtual school.” Paper presented at the APS March Meeting 2011, Dallas, Texas. Gonul, F. F. (2005), “Degrees of Education: Demand is Up, and Online Universities’ Increased Offerings Raise Questions,” Pittsburgh Post- Gazette, August 9, p. E2. Green, J. J., C. C. Stone, C. C., A. Zegeye and T. A. Charles (2009). "How Much Math Do Students Need to Succeed in Business and Economics Statistics? An Ordered Probit Analysis," Journal of Statistics Education, 17 (3). Holt, J. (1964), How Children Fail, New York: G. P. Pitman Publishing Corporation.How Children Fail, New York: G. P. Pitman Publishing Corporation. Holt, J. (1967), How Children Learn, New York: G. P. Pitman Publishing Corporation.How Children Learn, New York: G. P. Pitman Publishing Corporation. Holt, J. (1974), Escape From Childhood, New York: E. P. Dutton, Penguin Group.Escape From Childhood, New York: E. P. Dutton, Penguin Group. Holt, J. (1976), Instead of Education, New York: E. P. Dutton, Penguin Group.Instead of Education, New York: E. P. Dutton, Penguin Group. Khan, S. (2010), "YouTube U. Beats YouSnooze U.," Chronicle of Higher Education, 57 (11), pp. B36-B38. Nonis, S. A., and G. I. Hudson (1999), "The Second Course in Business Statistics and Its Role in Undergraduate Marketing Education," Journal of Marketing Education, 21(3), 232-241. Pfaff, T. J. and A. Weinberg (2009), " Do Hands-On Activities Increase Student Understanding?: A Case Study, Journal of Statistics Education, 17 (3), 1-34. Rasicot, J. (2011) "Education Review: Web Site Offering Free Math Lessons Catches On Like Wildfire," The Washington Post August 4, 2011. Retrieved 09/07/2011, from http://www.washingtonpost.com/lifestyle/magazine/web-site-offering-free-online-math-lessons-catches-on- like-wildfire/2011/07/15/gIQAtL5KuI_story.htmlhttp://www.washingtonpost.com/lifestyle/magazine/web-site-offering-free-online-math-lessons-catches-on- like-wildfire/2011/07/15/gIQAtL5KuI_story.html Rochelle, C. F. and D. Dotterweich (2007), "Student Success in Business Statistics,"Journal Of Economics And Finance Education, 6(1), pp. 19-24. Thompson, C. (2011, July 15, 2011). How Khan Academy Is Changing the Rules of Education. Wired, August 2011 Young, J. R. (2010, June 6,2010). College 2.0: A Self-Appointed Teacher Runs a One-Man 'Academy' on YouTube Are his 10-minute lectures the future? The Chronicle of Higher Education. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

5. Literature Review (continued) Social media is an educational tool as well as an entertainment tool. The future of education is the Internet as online universities, distance teaching, cyber-charter-schools experience rapid growth (Gonul 2005). One recent example where education is not a fear- invoking, must have in order to get a good job endeavor, but a fun, you can learn it, too, activity is the Khan Academy on YouTube, in the spirit of John Holt. John Holt has written a number of impactful books on learning in a noncoercive environment and has emphasized the difference between learning and schooling (see, e.g., Holt, J. 1964; 1967; 1974 1976, and others). Holt’s philosophy goes back as far as Leo Tolstoy and is connected to other deschoolers like George Dennison. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

6. Literature Review (continued) Research shows that when students have previously taken math classes their final grade improves. Rochelle and Dotterweich (2007) use correlations and find that student absences, grade earned in a previous introductory quantitative methods course (algebra and differential calculus), and GPA are highly correlated with performance in Business Statistics. They find demographic variables to be insignificant. When students spend more time studying math they have a higher probability of having a better performance in Business Statistics. Inadequate math preparation and inadequate math prerequisites prior to taking the statistics course have been cited among the potential reasons for poor student performance in statistics (Green, Stone, Zegeye, and Charles 2009). ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

7. Khan Academy Khan Academy is created by Salman Khan as an online resource that offers a set of educational materials and an integrated assessment system (Khan 2010). In our current schooling system lectures are often in one-to-many format with little room for active learning (Dickson and Segars 1999). Khan himself criticizes the current system and offers an alternative through free on-demand videos that enables self-paced learning by students anywhere, any age, in the English speaking world. The site has been described as revolutionary by Bill Gates and Google, and has received funding from the Bill and Melinda Gates Foundation and Google (Dijksman and Khan 2011; Rasicot 2011; Thompson 2011; Young 2010). The Office of Educational Technology at the U.S. Department of Education, and some state and local governments have embraced the Khan system as a valuable addition to student learning (Rasicot 2011). ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

8. Assessment Tools ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

11. Data One of the coauthors of this study teaches 2 sections of the course in Fall 2011. The experiment group in one section consists of 30 students. The control group in another section consists of 29 students. All students registered participate in the study. The control group consists of 29 students registered in the other section. Both classes are held in the same computer lab on the same days. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

12. Descriptive Statistics Figure 1 - Means ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

13. Descriptive Statistics Table 1 – Correlation Matrix ExamScoreTime ExamHWScore_4Time HW_4KHAN ExamScore1 Time Exam-0.2374192681 HWScore_40.512298354-0.0642463391 Time HW_40.1353219740.3732132030.3764431371 KHAN0.088680769-0.149038496-0.109640886-0.0077535031 ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

14. Model Y = {Exam score, Time in exam} + {Attendance, …} X = {Khan, HW Score} + {QPA, QPA in Previous Quantitative/Analytical Courses, Year in College, Had the same professor before/Not, …} Estimation with OLS; simultaneous equations (2SLS); with fixed effects to control for unobserved heterogeneity across students ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

15. Results from OLS Models Multiple Regression 1: HW score increases Exam Score (p-value < 0.01) Khan Academy increases Exam Score (p-value < 0.20) ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

16. Results (continued) Multiple Regression 2: HW score decreases Time in Exam (p-value < 0.05) Khan Academy decreases Time in Exam (p- value < 0.15) Time Spent on HW increases Time in Exam (p- value < 0.01) ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

17. Limitations (1)Time of day effect is not controlled for. Will address this with data from a previous semester with same time slots and no Khan Academy. (2) Small sample size. Will address this with new data from new semester as well as data mentioned in (1) above. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

18. Future Research Account for Khan system in a more detailed way (currently it is a 0/1 dummy): – # of modules completed by student – Time spent per student on each module – When a module is completed (vis-à-vis exam date) – Which module student had trouble with ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

19. In Summary … Purpose: Our study aims to test the impact of Khan Academy(a free web-based resource that aims to increase math skills) on student performance in Business Statistics courses. Methodology: One of the coauthors of this study teaches 2 sections of the course in Fall 2011. The experiment group in one section consists of 30 students. All students registered participate in the study. The control group consists of 29 students registered in the other section. Both classes are held in the same computer lab on the same days. Findings: In OLS models we find that HW score increases Exam Score (p-value < 0.01), Khan Academy increases Exam Score (p-value < 0.20), HW score decreases Time in Exam (p-value < 0.05), Khan Academy decreases Time in Exam (p-value < 0.15), Time Spent on HW increases Time in Exam (p-value < 0.01). Originality/Value: The study offers valuable insights to both students and faculty in quantitative courses in business schools. In current times, when higher education institutions face a reduction in public funds, it becomes increasingly important to exploit such free resources, in order to produce graduates with sufficient quantitative/analytical skills to help them succeed in the workplace. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA