1. Integrating Statistical Knowledge Through an Undergraduate Capstone Course John H. Walker: jwalker@calpoly.edu Heather S. Smith: hsmith@calpoly.edu Department of Statistics Cal Poly, San Luis Obispo Presentations and course syllabus at statweb.calpoly.edu/jwalker

2. Cal Poly Capstone Course2 Outline Cal Poly Statistics  Curriculum  Students  Capstone course Student problems and possible solutions  Choice of method  Presentation of results  Design issues

3. Cal Poly Capstone Course3 Cal Poly Statistics Curriculum Intro Stat Calc 1 & 2 Stat 1 & 2 Calc 3 Calc 4 Methods of Proof SAS Regression ANOVA/ DOE Stat electives: Time Series Categorical Multivariate Sampling Adv. DOE S-Plus Prob. & Math Stat 1, 2 & 3 Stat Comm & Consulting Math electives Linear Algebra

4. Cal Poly Capstone Course4 Cal Poly Statistics Students 10-15 graduates per year Highly variable ability All students must complete 90-120 hr. senior project About 50% go on to graduate school within 1-2 years  Ph.D.: Stanford, UCLA, UCSB, Ohio State, NCSU  Masters: CSU-East Bay, CSU-Long Beach, Loma Linda Univ. About 50% go to industry  Financial services, Market research, Consulting  SAS programming

5. Cal Poly Capstone Course5 The Capstone Course STAT 465: Statistical Communication & Consulting Open only to senior statistics majors 12-14 students per year Team-taught, 4 hours per week A mixture of technical and communications topics  Communications topics, such as: Organization of a consulting session Asking good questions Reporting results that clients can understand  Technical topics, such as: Choice of statistical method Review and integration of topics from earlier classes Students report “very challenging and very valuable”

6. Cal Poly Capstone Course6 Problem #1: Choice of Statistical Method Except for simple problems, students have great difficulty deciding what statistical method to use.  In lower-division classes, clues are too obvious.  In upper-division classes, only a few methods are covered. Specific student problems  Failure to recognize the roles of variables  Assuming all designed experiments are analyzed with ANOVA  Confusing quantitative and categorical data

7. Cal Poly Capstone Course7 Example A: Wine Prices Given as a pretest in the first class of Spring 2006 Data: 22 bottles of wine of the same variety  12 bottles from California, 10 bottles from Argentina  Price per bottle in dollars ($20 to $80 per bottle)  Wine spectator rating (87 to 92 points) Goal: “Do prices differ by region?” 5 of 13 students correctly used a regression of Price vs. Rating and Regional indicator 2 of 13 used ANOVA with Rating and Region as factors 6 of 13 used other methods  Two sample t-test of Price  Two sample t-test of Price/Rating

8. Cal Poly Capstone Course8 Example B: Gas Purification Given as a team project in Spring 2004 and Spring 2006 Data: Designed experiment with 24 experimental runs  Methane removed (ppb)  Temperature (3 or 4 quantitative levels, depending on quarter)  Flow rate (2 or 3 quantitative levels, depending on quarter)  Time (2 quantitative levels: 1 or 2 hours) Goal: “How do temperature, flow rate, and time affect the amount of methane removed?” 0 of 8 teams used regression. All teams used multi-way ANOVA and treated the quantitative predictors as categorical. Students mistakenly thought designed experiments are analyzed using ANOVA since they are taught together.

9. Cal Poly Capstone Course9 Solutions: Choice of Statistical Method In every class, as each method is introduced:  Compare and contrast with previously covered methods  Mention alternative methods covered in future classes Use more examples and more test problems focusing exclusively on “choice of method” in every class.  Don’t dilute problems with calculation or interpretation.  Include problems using methods covered in earlier classes!  Emphasize that choice is based on the goal of the study—not just the form of the data. In DOE classes, give examples where methods other than ANOVA are appropriate.

10. Cal Poly Capstone Course10 Problem #2: Presentation of Results Reporting statistical significance alone (p-value) but omitting the direction and size of the effect Interpreting the direction and size of an effect when the effect is statistically insignificant Reporting the means of main effects when their interactions are statistically significant Writing results as if answering homework problems (part a, part b, …) instead of combined in a meaningful paragraph form

11. Cal Poly Capstone Course11 Solutions: Presentation of Results Emphasize to statistics majors that most of their statistical communication will be with non-statisticians. For each statistical method, emphasize  What results are important to clients when the p-value is small  What results should not be reported when the p-value is large  How to explain these results to a client For some problems, require answers to be  Written as a report to a client  Verbally presented to someone posing as a client

12. Cal Poly Capstone Course12 Problem #3 and Solutions: Design Issues Students have trouble with replication and power. Discuss true replication, pseudo-replication, and multiple levels of replication early in the curriculum.  Students should be able to recognize all three—even if analysis methods aren’t covered until future courses.  Students should recognize when a basic ANOVA is appropriate and when it is not (even if they’ve never heard of a split plot). When discussing power, emphasize  What information is needed to compute power (or sample size)  How to ask clients for such information  How to report the results when some information is unavailable  How to explain power to non-statisticians