Innovative Teaching: A Test of the Impact of the Khan Academy in Business Statistics Courses Dr. Füsun F. Gönül Associate Professor of Marketing, School of Business, Slippery Rock University Slippery Rock, PA, Dr. Roger A. Solano Associate Professor of Management, School of Business, Slippery Rock University Slippery Rock, PA ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Research Objectives To test the impact of Khan Academy, on student performance, in Business Statistics To glean insights on teaching statistics (impact of Khan Academy on Faculty) To assess interactive learning via YouTube videos In current times, when higher education institutions face a reduction in public funds, it becomes increasingly important to exploit such free resources, in order to produce graduates with sufficient quantitative/analytical skills to help them succeed in the workplace. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Motivation Literature documents that math background improves grades in statistics courses. However, currently, SRU does not have a Math prerequisite for the Business Statistics course. If Khan improves grades, it can be made to replace the prerequisite. We test to find out if it does improve grades. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Literature Review Dickson, G. W. and A. Segars (1999), "Redefining the High-Technology Classroom," Journal of Education for Business, 74 (3), pp Dijksman, J. A., and S. Khan (2011). “Khan Academy: the world's free virtual school.” Paper presented at the APS March Meeting 2011, Dallas, Texas. Gonul, F. F. (2005), “Degrees of Education: Demand is Up, and Online Universities’ Increased Offerings Raise Questions,” Pittsburgh Post- Gazette, August 9, p. E2. Green, J. J., C. C. Stone, C. C., A. Zegeye and T. A. Charles (2009). "How Much Math Do Students Need to Succeed in Business and Economics Statistics? An Ordered Probit Analysis," Journal of Statistics Education, 17 (3). Holt, J. (1964), How Children Fail, New York: G. P. Pitman Publishing Corporation.How Children Fail, New York: G. P. Pitman Publishing Corporation. Holt, J. (1967), How Children Learn, New York: G. P. Pitman Publishing Corporation.How Children Learn, New York: G. P. Pitman Publishing Corporation. Holt, J. (1974), Escape From Childhood, New York: E. P. Dutton, Penguin Group.Escape From Childhood, New York: E. P. Dutton, Penguin Group. Holt, J. (1976), Instead of Education, New York: E. P. Dutton, Penguin Group.Instead of Education, New York: E. P. Dutton, Penguin Group. Khan, S. (2010), "YouTube U. Beats YouSnooze U.," Chronicle of Higher Education, 57 (11), pp. B36-B38. Nonis, S. A., and G. I. Hudson (1999), "The Second Course in Business Statistics and Its Role in Undergraduate Marketing Education," Journal of Marketing Education, 21(3), Pfaff, T. J. and A. Weinberg (2009), " Do Hands-On Activities Increase Student Understanding?: A Case Study, Journal of Statistics Education, 17 (3), Rasicot, J. (2011) "Education Review: Web Site Offering Free Math Lessons Catches On Like Wildfire," The Washington Post August 4, Retrieved 09/07/2011, from like-wildfire/2011/07/15/gIQAtL5KuI_story.htmlhttp:// like-wildfire/2011/07/15/gIQAtL5KuI_story.html Rochelle, C. F. and D. Dotterweich (2007), "Student Success in Business Statistics,"Journal Of Economics And Finance Education, 6(1), pp Thompson, C. (2011, July 15, 2011). How Khan Academy Is Changing the Rules of Education. Wired, August 2011 Young, J. R. (2010, June 6,2010). College 2.0: A Self-Appointed Teacher Runs a One-Man 'Academy' on YouTube Are his 10-minute lectures the future? The Chronicle of Higher Education. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Literature Review (continued) Social media is an educational tool as well as an entertainment tool. The future of education is the Internet as online universities, distance teaching, cyber-charter-schools experience rapid growth (Gonul 2005). One recent example where education is not a fear- invoking, must have in order to get a good job endeavor, but a fun, you can learn it, too, activity is the Khan Academy on YouTube, in the spirit of John Holt. John Holt has written a number of impactful books on learning in a noncoercive environment and has emphasized the difference between learning and schooling (see, e.g., Holt, J. 1964; 1967; , and others). Holt’s philosophy goes back as far as Leo Tolstoy and is connected to other deschoolers like George Dennison. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Literature Review (continued) Research shows that when students have previously taken math classes their final grade improves. Rochelle and Dotterweich (2007) use correlations and find that student absences, grade earned in a previous introductory quantitative methods course (algebra and differential calculus), and GPA are highly correlated with performance in Business Statistics. They find demographic variables to be insignificant. When students spend more time studying math they have a higher probability of having a better performance in Business Statistics. Inadequate math preparation and inadequate math prerequisites prior to taking the statistics course have been cited among the potential reasons for poor student performance in statistics (Green, Stone, Zegeye, and Charles 2009). ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Khan Academy Khan Academy is created by Salman Khan as an online resource that offers a set of educational materials and an integrated assessment system (Khan 2010). In our current schooling system lectures are often in one-to-many format with little room for active learning (Dickson and Segars 1999). Khan himself criticizes the current system and offers an alternative through free on-demand videos that enables self-paced learning by students anywhere, any age, in the English speaking world. The site has been described as revolutionary by Bill Gates and Google, and has received funding from the Bill and Melinda Gates Foundation and Google (Dijksman and Khan 2011; Rasicot 2011; Thompson 2011; Young 2010). The Office of Educational Technology at the U.S. Department of Education, and some state and local governments have embraced the Khan system as a valuable addition to student learning (Rasicot 2011). ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Assessment Tools ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Data One of the coauthors of this study teaches 2 sections of the course in Fall The experiment group in one section consists of 30 students. The control group in another section consists of 29 students. All students registered participate in the study. The control group consists of 29 students registered in the other section. Both classes are held in the same computer lab on the same days. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Descriptive Statistics Figure 1 - Means ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Descriptive Statistics Table 1 – Correlation Matrix ExamScoreTime ExamHWScore_4Time HW_4KHAN ExamScore1 Time Exam HWScore_ Time HW_ KHAN ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Model Y = {Exam score, Time in exam} + {Attendance, …} X = {Khan, HW Score} + {QPA, QPA in Previous Quantitative/Analytical Courses, Year in College, Had the same professor before/Not, …} Estimation with OLS; simultaneous equations (2SLS); with fixed effects to control for unobserved heterogeneity across students ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Results from OLS Models Multiple Regression 1: HW score increases Exam Score (p-value < 0.01) Khan Academy increases Exam Score (p-value < 0.20) ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Results (continued) Multiple Regression 2: HW score decreases Time in Exam (p-value < 0.05) Khan Academy decreases Time in Exam (p- value < 0.15) Time Spent on HW increases Time in Exam (p- value < 0.01) ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Limitations (1)Time of day effect is not controlled for. Will address this with data from a previous semester with same time slots and no Khan Academy. (2) Small sample size. Will address this with new data from new semester as well as data mentioned in (1) above. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

Future Research Account for Khan system in a more detailed way (currently it is a 0/1 dummy): – # of modules completed by student – Time spent per student on each module – When a module is completed (vis-à-vis exam date) – Which module student had trouble with ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA

In Summary … Purpose: Our study aims to test the impact of Khan Academy(a free web-based resource that aims to increase math skills) on student performance in Business Statistics courses. Methodology: One of the coauthors of this study teaches 2 sections of the course in Fall The experiment group in one section consists of 30 students. All students registered participate in the study. The control group consists of 29 students registered in the other section. Both classes are held in the same computer lab on the same days. Findings: In OLS models we find that HW score increases Exam Score (p-value < 0.01), Khan Academy increases Exam Score (p-value < 0.20), HW score decreases Time in Exam (p-value < 0.05), Khan Academy decreases Time in Exam (p-value < 0.15), Time Spent on HW increases Time in Exam (p-value < 0.01). Originality/Value: The study offers valuable insights to both students and faculty in quantitative courses in business schools. In current times, when higher education institutions face a reduction in public funds, it becomes increasingly important to exploit such free resources, in order to produce graduates with sufficient quantitative/analytical skills to help them succeed in the workplace. ACBSP November 12, 2011 Meeting of Region 2, Latrobe PA