Presentation on theme: "By Chris Franck. About LISA Laboratory for Interdisciplinary Statistical Analysis www.lisa.stat.vt.edu FREE services: Collaboration, walk-in consulting,"— Presentation transcript:
About LISA Laboratory for Interdisciplinary Statistical Analysis www.lisa.stat.vt.edu FREE services: Collaboration, walk-in consulting, short courses We are here to help you! Goal is to improve research across the Virginia Tech community.
Eric Vance Director Chris Franck Assistant Director Tonya Pruitt Administrative Superstar
Structure of talk Cover three fundamental aspects of experimental design Scenarios come from consulting experience Highlight LISA services available Key words in red ?? Interesting questions.
Central messages If you face statistical uncertainty at any stage of your research, please come to LISA. Best time to involve the statisticians: Before the data has even been collected. Speaking of the pre-data collection phase…
Why are we here? Choices made at the design stage have the potential to drastically impact the results of any study. Good experimental design gives the researcher an improved chance of a successful experiment. A poorly considered or implemented design can have a ruinous effect on the investigation.
The plan We will discuss basic elements of experimental design: randomization, replication, and blocking. Real world experiments. Interpretation of experimental results will be compared and contrasted with interpretation of results from observational studies.
Study design: food science Research question: Among three genetic varieties of sweet potatoes, which type will brown the least when fried? Also take storage time into account. Measurement of browning done with a machine – beyond the scope of this talk. The following graphic shows the design layout.
Features of the design Suppose we conduct this experiment and conclude the third variety browns the most, and the first variety browns the least. Is this necessarily due to the genetic differences in the potato types? Is there another plausible explanation?
What about cooking order? Notice that for a given week, all of the potatoes are cooked in the same oil. Also, the varieties are always cooked in the same order, making the effect of variety and the effect of cook order inseparable. The effect of variety on browning is confounded with the effect cooking order has on browning.
Why randomize? Randomization is a fundamental feature of good experimental design. In this case, randomization will eliminate the known confound between cooking order and potato variety. Randomization makes groups similar on average, and hence eliminates unknown confounding effects as well!
Sweet potato remarks Since the effect of interest (genetic variety) is confounded with cooking order in the current experiment, recommendation is to repeat the experiment with a randomized design. Many randomized designs exist! Perhaps changing oil more frequently can also improve the project.
Costly In general randomization is not difficult to perform. In this case the cost of repeating experiment in randomized fashion was moderate (about 12 weeks time + materials). Repeating an experiment can be VERY costly. (3 years + materials PhD research)
HW problem A grape researcher is interested in testing the effect of 4 pesticides on the disease rate on his grapes. For his experiment he has 16 total vines arranged in four plots. Each vine has a trunk at the center and two cordons extending from the trunk. Many grape clusters grow on each cordon.
How to assign pesticides? To administer the pesticides, the researcher randomly assigns one pesticide (labeled A, B, C, and D) to each of the plots. He then sprays the assigned pesticide on all four vines in each plot, walking from north to south in each case. Call pesticide treatment.
How many reps for each treatment? A) 4 reps/treatment since there are four vines that receive each pesticide. B)8 reps/pesticide since there are eight cordons that receive a given treatment. C) Many replicates: depends on the number of grapes which grow, since each grape might or might not have the disease. D) Something else.
Answer: Number of experimental replicates: Why!? What went wrong? The experimental unit is the smallest unit in the experiment to which separate treatment assignments are made. What was the experimental unit in this experiment?
Definition of replicate The number of replicates for a given treatment is equal to the number of times the treatment was assigned to the experimental unit.
Consequences of this design We cannot perform usual statistical inference in this experiment. That is, we cannot perform hypothesis tests, construct confidence intervals, etc. The resulting data might suggest a difference in the treatments, but we can’t quantify the uncertainty of the results with confidence levels, p-values, etc.
Improvements Instead of using 4 total plots, we might use 8, 12, 16, etc. This would give 2,3,4 replicates per treatment. Instead of randomizing the treatments to the plots, perhaps we can randomize the treatments to the vines themselves?
Randomizing treatments to vines Now the vine is the experimental unit. 4 replicates for each treatment instead of 1. ?? What if our treatment is sprayed on the vines in such a way that adjacent vines get a little bit of the wrong treatment? Windy day? This is an example of a carryover effect – we can address these advanced issues at LISA collaboration meeting.
Umm, Chris, did you really randomize? Plot 1 has three instances of D! Not one single plot has each of the treatments! What if plot 1 has ideal characteristics? (irrigation, soil quality, sunlight) ?? Won’t treatment D seem better than it otherwise would since it appears in the best plot three times?
Yes, and Yes I did randomize, using a completely randomized design. Yes, if plot 1 has different characteristics than the other plots, and D appears frequently by chance in plot 1, then the three observations on treatment D are a function of both the treatment and the enhanced plot characteristics.
In general If the plots 1,2,3 and 4 have are not identical in terms of the response (disease rate), then plot to plot or inter-plot variability is present. Maybe some of the plots are on inclines, soil characteristics may be different, etc. Call the impact of the different plots on the response the plot effect.
How do we handle the plot effect? We are interested in the disease rates of grapes. We believe the various treatments will affect these disease rates. We also believe that the plots will also have an impact on the disease rates. We don’t really care about the plot effect – our primary goal is to determine how the treatments affect the response. Plots are simply an extra source of variability
Treat the plots as blocks Blocking is a strategy that may be implemented in order to account for known sources of variability in the experimental material. In our case, the plots may show variability we wish to account for.
To implement blocking Blocking is implemented during the design phase of the experiment. We want to assign the treatments into the blocks so that each treatment appears in each block exactly one time. Assigning treatments to blocks in this fashion is a form of restricted randomization.
Notes about RCBD Notice each treatment appears in each block exactly once. Statistical model for this design: y ij = μ + α i + β j + ε ij i=1,…,a is the number of treatments (4) j=1,…,b is the number of blocks (4) y ij is the response for the ith treatment, jth block.
More terms μ is the overall mean. α i is the treatment effect for the ith treatment. β j is the effect of the jth block. ε ij is an error term associated with response at ith treatment and jth block.
Take home message By implementing the randomized complete block design, we can: Compare the performance of the four treatments. Account for variability in the plots that might otherwise obscure the treatment effects. But suppose you did not randomize the treatments into blocks before collecting data. Can I still use above technique?
Can I? Maybe, maybe not. If you used a completely randomized design (as I did initially), you may not be able to fit the RCBD model! This is because some of the parameters in the model may be non-estimable depending on how your randomization works out e.g. in completely randomized design, no information about how treatment A behaves in plot 1. Come see LISA when designing experiments!
Row effects? ?? RCBD seems good, but what if I also have a row effect in addition to a plot effect. E.g. perhaps there is a fertility gradient within each plot. Treatment C appears three times in the second row in RCBD plot. Don’t we have the same problem even with RCBD. Answer:
One benefit of experiments vs. observational studies In general, experiments provide more evidence of causal relationships between variables. Observational studies can show associations between variables, but are NOT sufficient to demonstrate causality. E.g. Survey grape farms, ask which treatment they use to control disease, find one treatment better than others. Causal?
Conclusions The design phase of an experiment is a crucial time to plan carefully. Careful design sets the stage for success. Overlooking this stage can lead to disastrous results. LISA can help you design experiments. Nobody (including LISA) can fully rescue an experiment that has design flaws.
Laboratory for Interdisciplinary Statistical Analysis Collaboration: Visit our website to request personalized statistical advice and assistance with: Experimental Design Data Analysis Interpreting Results Grant Proposals Software (R, SAS, JMP, SPSS...) LISA statistical collaborators aim to explain concepts in ways useful for your research. Great advice right now: Meet with LISA before collecting your data. All services are FREE for VT researchers. We assist with research—not class projects or homework. LISA helps VT researchers benefit from the use of Statistics www.lisa.stat.vt.edu LISA also offers: Educational Short Courses: Designed to help graduate students apply statistics in their research Walk-In Consulting: M-F 12-2PM in 401 Hutcheson Hall for questions requiring <30 mins
Laboratory for Interdisciplinary Statistical Analysis www.lisa.stat.vt.edu To request a collaboration meeting go to www.lisa.stat.vt.edu
Laboratory for Interdisciplinary Statistical Analysis www.lisa.stat.vt.edu To request a collaboration meeting go to www.lisa.stat.vt.eduwww.lisa.stat.vt.edu 1. Sign in to the website using your VT PID and password. 2. Enter your information (email address, college, etc.) 3. Describe your project (project title, research goals, specific research questions, if you have already collected data, special requests, etc.) 4. Wait 0-3 days, then contact the LISA collaborators assigned to your project to schedule an initial meeting.
Other LISA short courses 47 Date Course Title Instructor 10/12/2010 Design of Experiments Chris Franck 10/19/2010 Introduction to JMP Wandi Huang 10/26/2010 T-tests ANOVA and ANCOAV Jennifer Kensler 11/01/2010 Introduction to JMP Wandi Huang 11/02/2010 Analyzing Non-Normal Data with Generalized Linear Models (GLMs) Sai Wang 11/08/2010 Intro to SAS Mark Seiss 11/09/2010 Intro to SAS Mark Seiss 11/15/2010 Using R for Your Basic Statistical Needs Nels Johnson 11/16/2010 Using R for Your Basic Statistical NeedsNels Johnson