1. Implementation and Order of Topics at Hope College

2. In the beginning … We start the course with an overview of the statistical investigative process (we have seven steps) from asking a question to communicating the results. We then focus on the inference part of this process and begin using randomization during the first week of the course.

3. In the beginning … Our first example is a test for a single proportion. We start out modeling the null hypothesis through coin flipping. We had been using the study involving babies preferences for nice versus naughty toy. In that example, 14 of 16 babies picked the nice toy. We will now be using a dolphin communication study example where 15 of 16 trials two dolphins successfully communicated.

4. In the beginning … Both these examples are very significant and we can put off talking about specifics of the p-value for a little while and focus on the process of randomization. As an initial example, we like the dolphin study over the naughty or nice toy study. The dolphin study boils down to a single dolphin pushing one of two buttons and then repeating this process 16 times. Flipping a single coin 16 times, nicely models this process if the dolphin is just guessing.

5. Tactile Methods We begin with tactile methods of randomization. Coin flipping for single proportion Playing cards (red/black) shuffled into two piles for comparing two proportions Playing cards (red/black) shuffled to mix up categorical variable while the quantitative variable stays in the same order when comparing two means. Cards with numbers on them for testing correlation.

6. Technology We use an applet to simulate coin flipping. We have used Fathom for the other randomized methods. We are in the process of converting our materials so that applets are used instead of Fathom. We also use SPSS for some of the traditional methods and projects.

7. Order of Topics (Last two years) One proportion Comparing Two Proportions Comparing Two Means Correlation and Regression Comparing Means Comparing Proportions Single Mean and Proportion Randomization Methods Traditional Methods

8. Other topics Descriptive statistics are interspersed throughout in a just in time approach. Power is discussed in a very intuitive way and how it relates to sample size, difference in sample statistics, etc. The differences between analyzing a process, sampling from a finite population, and an experiment are discussed early. Confidence intervals are introduced as a range of plausible values for the population parameter.

9. Key Features We do little lecture and lots of activities. We meet in a computer classroom. We focus on the entire statistical investigative process. We look at real studies in our examples, activities, homework, case studies, and research papers. Students complete two research projects, one in the middle of the semester and one at the end.

10. How we got started using a randomization-based course at Hope College

11. How we began In 2008, Hope College was awarded a $1.4 million grant from the Howard Hughes Medical Institute. Part of that grant was earmarked for the creation of a computer classroom devoted to teaching statistics. Nathan Tintle lead the effort to get the lab and start redesigning the curriculum.

13. How we began We used part of the HHMI grant as well as a few other small ones to begin development of a new curriculum. Early in 2009, we began by converting randomization- based modules that were previously written by Allan Rossman and Beth Chance into a complete text. Three of us were doing the writing (Nathan Tintle, Jill VanderStoep, and myself). We held a workshop during the summer for the other instructors. In the Fall of 2009 we had a text completed that would be used in all our sections of introductory statistics.

14. Institutional support/resistance The college and the department have historically been very supportive of curricular changes. Nathan spoke with groups from all the client departments about the changes that we were making. He found no resistance, in fact they were excited about the changes. Other instructors of statistics are supportive of this method, though they arent too excited about the constant changes we make.

15. The times they are a-changin We are in the process of rewriting the entire curriculum by Changing the order of topics Making it as flexible as possible (lecture or activity) Having the best possible research examples Etc. For the last year, we have been working with Beth Chance, Allan Rossman, Soma Roy, and George Cobb.

16. Is it worth it? Yes! We believe that this is how introductory statistics should be taught. Students gain a clearer and deeper understanding of the process of inference using a randomization- based approach than with a traditional approach.

17. Assessment I (JSE: March 2011) The Comprehensive Assessment of Outcomes in Statistics (CAOS) Students in our randomization course took this pre- and post-test in the Fall of 2009 (n = 202). These results were compared with students that took our traditional course in the Fall of 2007 (n = 198) and those from a national representative sample (n = 768). Overall, learning gains were significantly higher for students that took the randomization course when compared to either those that took the traditional course at Hope or the national sample.

18. Questions where the new curriculum faired significantly better Understanding that low p-values are desirable in research studies (Tests of significance) Understanding that no statistical significance does not guarantee that there is no effect (Tests of significance) Ability to recognize a correct interpretation of a p-value (Tests of significance) Ability to recognize an incorrect interpretation of a p-value. Specifically, probability that a treatment is not effective. (Tests of significance)

19. Questions where the new curriculum faired significantly better Understanding of the purpose of randomization in an experiment (Data collection and design) Understanding of how to simulate data to find the probability of an observed value (Probability)

20. Questions where the new curriculum faired significantly worse Ability to correctly estimate and compare standard deviations for different histograms. (Descriptive statistics)

21. Assessment II (Submitted to SERJ) Four Month Retention Students again took the CAOS test four months after the end of the course. In 2007 the overall mean decreased by about 4 percentage points from December to April. In 2009 the overall mean decreased by about 0.5 percentage points from December to April.

22. Assessment II Significant differences between 2007 and 2009 were found in questions involving Data Collection and Design Tests of Significance

23. Contact Information swansont@hope.edu http://www.math.hope.edu/isi/