Statistical Modelling Chapter X 1 X.Sample size and power X.AHow it is done X.BPower X.CComputing the required sample size for the CRD and RCBD with a single treatment factor X.DSample size for the Latin square design X.ESample size for factorial experiments X.FSample size for the standard split-plot experiment

Statistical Modelling Chapter X 2 However, perhaps there are differences between the penicillin treatments –just that 5 replicates was not enough, given the size of the true differences between the treatment means. How many replicates should we take? X.A How it is done Example X.1 Penicillin yield For experiments, sample size is abut finding the number of replicates, r. In example IV.1 the effects of four treatments (A, B, C and D) on the yield of penicillin were investigated. It was concluded that the following treatment means were not significant:

Statistical Modelling Chapter X 3 Detemining no. replicates Determine the number of pure replicates of a treatment, denoted r. In order to compute the r you will have to specify: 1.the significance level,  ; 2.the power (or probability of detecting a difference when there is actually a difference) desired, 1   ; 3.the number of treatments to be investigated 4.the minimum size of the difference to be detected between a pair of treatment means, as measured by  ; 5.the uncontrolled variation,, expected. In working out values for these, often the results from a previous experiment, you or another researcher has run, will be useful. With this information, use R function no.reps to compute, r. It, and its associated functions power.exp and power.diff, are available in the dae library.

Statistical Modelling Chapter X 4 Usage and arguments for no.reps no.reps(multiple=1, df.num=1, df.denom=expression((df.num+1)*(r-1)), delta=1, sigma=1, alpha=0.05, power =0.8, tol = 0.025, print=FALSE) –Note the values given to arguments are the defaults if the argument is not set in a call to the function. multiple : the multiplier, m, which when multiplied by the number of pure replicates of a treatment, r, gives the number of observations used in computing means for some, not necessarily proper, subset of the treatment factors; m is the replication arising from other treatment factors. However, for single treatment factor experiments the subset can only be the treatment factor and m = 1; df.num : the df of the numerator of the F for testing the term involving the treatment factor subset; df.denom : an expression for the df of the denominator of the F for testing the term involving the treatment factor subset –must involve r, the number of pure replicates, –can involve other arguments to no.reps such as multiple and df.num, –must be enclosed in an expression function so that it is not evaluated when no.reps is called but will be evaluated as different values of r are tried during execution of no.reps ; delta : the true difference between a pair of means for some, not necessarily proper, subset of the treatment factors; sigma : population standard deviation; alpha : the significance level to be used; power : the minimum power to be achieved; tol : the maximum difference tolerated between the power required and the power computed in determining the number of replicates; print : TRUE or FALSE to have or not have a table of power calculation details printed out.

Statistical Modelling Chapter X 5 Example X.1 Penicillin yield (continued) We now determine the number of replicates required to achieve a power of 0.80 in detecting   5 with   In the ANOVA for this experiment, the Residual MSq was so we will take Output from no.reps for the present example is as follows: > no.reps(multiple=1, df.num=3, + df.denom=expression(df.num*(r-1)), delta=5, + sigma=sqrt(20), power=0.8) $nreps [1] 19 $power [1] Required number of replicates is 19 with a power of

Statistical Modelling Chapter X 6 X.BPower a)Type I and II errors It is important to keep in mind that your conclusion about the null hypothesis is not 100% certain to be correct. The possible outcomes — the conclusion (or verdict) reached as a result of a hypothesis test — are: Thus there are two types of errors that can be made in performing a hypothesis test.

Statistical Modelling Chapter X 7 Two types of errors Definition X.1: A type I error is made when the null hypothesis is true and it is rejected. The probability of a type I error is designated  and is: Definition X.2: A type II error is made when the null hypothesis is false and it is not rejected. The probability of a type II error is designated  and is: In a hypothesis test we set the probability of a type I error at , called the significance level. –So set the level of risk prepared to take in making a type I error. But what about the probability of a Type II error,  ? Rather than , often consider 1 , called the power of the test.

Statistical Modelling Chapter X 8 b)Power of a hypothesis test about expectation model terms Definition X.3: The power of a hypothesis test is the probability of rejecting the null hypothesis when it is false is: Now to compute this probability means that we need to know the condition under which the null hypothesis is rejected.

Statistical Modelling Chapter X 9 Rejecting the null hypothesis In general the null hypothesis is rejected when the computed value of the probability of the F statistics from the analysis of variance is less than . –Will occur whenever observed F >  % value from the Snedecor’s F distribution. In this case any observed value of the test statistic greater than 3.49 would result in a p-value of less than 0.05 and so be rejected at the 5% level.

Statistical Modelling Chapter X 10 How false is the null hypothesis? Know H 0 is false. That is, there are differences between the population treatment means. But, how big a difference? For the single, treatment-factor experiments a measure of the size of the differences, relative to the magnitude of the uncontrolled variation, is given by provided the number of replicates for all treatments is r. However, to use this formula requires that we know the values of the  s which is unlikely as estimating these is the purpose of the experiment.

Statistical Modelling Chapter X 11 Overcoming unknown  s A way to overcome this is to specify , the difference such that if any two population means differ by this amount, H 0 should be rejected. Then it can be shown that a general formula for the minimum value of is where –r is the number of pure replicates of each treatment –m is the multiplier of r that gives no. of observations (rm) used in computing one of the means. For experiments involving a single treatment factor m  1. For a single, treatment-factor RCBD r  b. For a single, treatment-factor LS r  t.

Statistical Modelling Chapter X 12 F distribution when H 0 is false We can now be more specific about computing the power: P{H 0 rejected | H 0 false} –need the probability of getting a value of F greater than that of the  % value from Snedecor’s F distribution when there is a difference between the treatments of the order specified by  (or ). Clearly, Snedecor’s distribution cannot be used to compute this probability as it is the distribution that applies when the null hypothesis is true and   0. We need a distribution of F for when the null hypothesis is false. This is provided by the noncentral F distribution – a modification of the (central) Snedecor's F distribution to incorporate a noncentrality parameter.

Statistical Modelling Chapter X 13 Shape of the noncentral F distribution Depends on 1, 2 and — for the central F distribution  0. Distribution for: – = 0 is distribution of F 3,12 when H 0 true –  4 is distribution of F 3,12 when H 0 is not true –  10 is distribution of F 3,12 when H 0 is even less true

Statistical Modelling Chapter X 14 Computing the power Specifically, to compute the power of an analysis of variance test for a fixed factor: where is the F value from the central F distribution such that The R function pf computes probabilities for the noncentral F distribution –its arguments are q (= F), df1, df2 and ncp (= ). Also, power.exp for computing power in an experiment — available in dae library.

Statistical Modelling Chapter X 15 Usage and arguments for power.exp power.exp(rm=5, df.num=1, df.denom=10, delta=1, sigma=1, alpha=0.05, print=FALSE) rm : the number of observations used in computing a mean. df.num : the degrees of freedom of the numerator of the F for testing the term involving the means; df.denom : the degrees of freedom of the denominator of the F for testing the term involving the means; delta : the true difference between a pair of means; sigma : population standard deviation; alpha : the significance level to be used print : TRUE or FALSE to have or not have a table of power calculation details printed out.

Statistical Modelling Chapter X 16 Suppose expected that the minimum difference between a pair of treatment means for this experiment is 5 and that  = We continue to take Example X.1 Penicillin yield (continued) Also r  5 and m  1.

Statistical Modelling Chapter X 17 Example X.1 Penicillin yield (continued) Output from power.exp call to compute power. > rm <- 5 > power.exp(rm=rm, df.num=3, df.denom=3*(rm-1), delta=5, + sigma=sqrt(20), print=TRUE) rm df.num df.denom alpha delta sigma lambda powr [1] Power for detecting a minimum difference of 5 in a pair of treatment means when  U  4.47 is * (rm - 1) and sqrt(20) will be evaluated prior to call. need to set rm outside expression alpha not set in call — so default value of 0.05 used. = 3.125

Statistical Modelling Chapter X 18 Example X.1 Penicillin yield (continued) No difference between treatments –probability of rejecting H 0 determined from central F to be Treatment difference corresponding to  –probability of rejecting H 0 determined from noncentral F to be

Statistical Modelling Chapter X 19 How can we improve power? The power of the hypothesis test being 0.22 is not good because not a high chance of correctly rejecting H 0. Examine formula for noncentrality parameter Leads us to conclude that, for a fixed number of treatments, the noncentrality parameter will increase if –increase the number of replicates, r; –increase the size of the differences between the treatments, as measured by  ; –decrease the uncontrolled variation,. So we can get better power by changing these. How can these be improved? –Most often have to increase r.

Statistical Modelling Chapter X 20 X.CComputing the required sample size for the CRD and RCBD with a single treatment factor As before, determining sample size amounts to determining number of pure replicates of a treatment, denoted r. Achieved by computing the power for different rs until smallest r that has at least the required power is identified. Given the discussion on the power of a hypothesis test, as already noted, in order to compute the r you will have to specify: 1.the significance level,  ; 2.the power desired, 1   ; 3.the number of treatments to be investigated 4.the minimum size of the difference to be detected between a pair of treatment means, as measured by  ; 5.the uncontrolled variation,, expected. In working out values for these, often the results from a previous experiment, you or another researcher has run, will be useful.

Statistical Modelling Chapter X 21 Using R to compute required r Can be achieved by computing the power for different r s using the function power.exp –If the computed the power for a particular r is: a)too high, decrease r, or b)too low, increase r, continuing in both cases until you have identified the smallest r that has at least the required power.

Statistical Modelling Chapter X 22 Example X.1 Penicillin yield (continued) We now determine the number of replicates required to achieve a power of 0.80 in detecting   5 with   We continue to take 5 replicates achieved a power of only 0.22 and is clearly much smaller than required.

Statistical Modelling Chapter X 23 Example X.1 Penicillin yield (continued) Take r = 15 as our first guess and find that this would provide a power of > rm <- 15 > power.exp(rm=rm, df.num=3, df.denom=3*(rm-1), + delta=5, sigma=sqrt(20)) [1] Next increase r to 20 and this indicates a power of will be achieved. > rm <- 20 > power.exp(rm=rm, df.num=3, df.denom=3*(rm-1), + delta=5, sigma=sqrt(20)) [1] As this is in excess of the required power, r is reduced to 19 and the power reduces to As this is still in excess of the required power, r is reduced to 18 and the power reduces to Clearly, 19 replicates is required to achieve a power of at least 0.80 — fewer will have less power. More convenient method is provided by function no.reps.

Statistical Modelling Chapter X 24 Example X.1 Penicillin yield (continued) Output from no.reps for the present example is as follows: > no.reps(multiple=1, df.num=3, df.denom=expression(df.num*(r-1)), delta=5, + sigma=sqrt(20), power=0.8, print=TRUE) rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr rm df.num df.denom alpha delta sigma lambda powr $nreps [1] 19 $power [1] Confirms required number of replicates is 19 with a power of

Statistical Modelling Chapter X 25 X.DSample size for the Latin square design Formulae for computing the power and sample size for an RCBD also apply to the Latin square except that the number of replicates is t. However, number of rows, columns and treatments must be equal. So you cannot increase the treatment replication without changing the number of treatments. Consequently, main interest will be determine if the proposed design will have the desired power. Use the function power.exp.

Statistical Modelling Chapter X 26 X.ESample size for factorial experiments Sample size for a two-way factorial experiment can be computed using the R function no.reps. Similar to the single, treatment-factor experiments described in the previous sections. First step is to determine which effects you wish to specify to be determined with a nominated power: A main, B main and/or interaction effects. Then determine the power for each set of effects.

Statistical Modelling Chapter X 27 Cells that vary for the different types of effects Note that for –A and B,  is the difference between a pair of A or B means. –A#B interaction,  is the difference between the simple effect for one of the factors and the corresponding main effect for that factor. –For example, represents the difference between an A simple effect and the A main effect.

Statistical Modelling Chapter X 28 Example VII.4 Animal survival experiment Poison and Treatment did not interact in their effect on the death rate. However, consider the following combined table of means. Main effect for Treats 2 vs 3: – = If no interaction, would expect differences between Treats 2 and 3 for each Poison to be about –They are not exactly all equal to this value. But, how different from would be an important difference? This is what  is.

Statistical Modelling Chapter X 29 DF denominator depends on design employed

Statistical Modelling Chapter X 30 Example X.2 Fertilizing oranges Suppose that for the experiment to investigate 3 levels of N and 2 levels of P, outlined in example VII.1, you wish to determine the number of blocks to use in an RCBD. –Like to be able to detect with 80% power, a difference between a simple effect and a main effect of 10. –You believe that the standard deviation will be about 7.5 and you will use a significance level of 5%. –How many replicates are required to achieve the desired power? So we require the number of blocks in an RCBD to detect an interaction effect. The R output for obtaining this is as follows: > no.reps(multiple=1, df.num=2, df.denom=expression(5*(r-1)), + delta=10, sigma=7.5, power=0.80) $nreps [1] 12 $power [1] So 12 blocks needed to detect, with 80% power, a difference of 10 between a simple effect and a main effect.

Statistical Modelling Chapter X 31 X.FSample size for the standard split- plot experiment Require guesstimates of these two variances. Then sample sizes computed as described for ordinary factorial experiments in section X.E –except that the values of the variance (  2 ) are varied as follows. Determining the sample size for the standard split-plot experiment is complicated as involve 2 sources of uncontrolled variation: –main-plot variation –subplot variation

Statistical Modelling Chapter X 32 X.GExercises Ex asks you to compute the sample size for an RCBD. Ex asks you to compute the power for an RCBD. Ex asks you to determine if a Latin square has adequate power. Ex asks you to determine if a repeated Latin square has adequate power. Ex asks you to compute the number of replicates for a factorial experiment laid out as a CRD.