1. Statistical Power And Sample Size Calculations
Minitab calculations
Mike Cox, Newcastle University, me fecit 11/03/2016
Manual calculations
Sunday, 01 December :56 PM

2. When Do You Need Statistical Power Calculations, And Why?
A prospective power analysis is used before collecting data, to consider design sensitivity .

3. When Do You Need Statistical Power Calculations, And Why?
A retrospective power analysis is used in order to know whether the studies you are interpreting were well enough designed.

4. When Do You Need Statistical Power Calculations, And Why?
In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis.
Cohen, J “The statistical power of abnormal-social psychological research: A review” Journal of Abnormal and Social Psychology 4

5. What Is Statistical Power? Essential concepts
the null hypothesis Ho
significance level, α
Type I error
Type II error
Information point: Type I and Type II errors
Crichton, N.
Journal Of Clinical Nursing 9(2)

6. What Is Statistical Power? Essential concepts
Recall that a null hypothesis (Ho) states that the findings of the experiment are no different to those that would have been expected to occur by chance. Statistical hypothesis testing involves calculating the probability of achieving the observed results if the null hypothesis were true. If this probability is low (conventionally p < 0.05), the null hypothesis is rejected and the findings are said to be “statistically significant” (unlikely) at that accepted level. 6

7. Statistical Hypothesis Testing
When you perform a statistical hypothesis test, there are four possible outcomes

8. Statistical Hypothesis Testing
whether the null hypothesis (Ho) is true or false
whether you decide either to reject, or else to retain, provisional belief in Ho

9. Statistical Hypothesis Testing
Decision
Ho is really true i.e., there is really no effect to find
Ho is really false i.e., there really is an effect to be found
Retain Ho
correct decision:
prob = 1 - α
Type II error: prob = β
Reject Ho
Type I error: prob = α
correct decision: prob = 1 - β

10. When Ho Is True And You Reject It, You Make A Type I Error
When there really is no effect, but the statistical test comes out significant by chance, you make a Type I error.
When Ho is true, the probability of making a Type I error is called alpha (α). This probability is the significance level associated with your statistical test.

11. When Ho is False And You Fail To Reject It, You Make A Type II Error
When, in the population, there really is an effect, but your statistical test comes out non-significant, due to inadequate power and/or bad luck with sampling error, you make a Type II error.
When Ho is false, (so that there really is an effect there waiting to be found) the probability of making a Type II error is called beta (β).

12. The Definition Of Statistical Power
Statistical power is the probability of not missing an effect, due to sampling error, when there really is an effect there to be found.
Power is the probability (prob = 1 - β) of correctly rejecting Ho when it really is false.

13. Calculating Statistical Power Depends On
the sample size
the level of statistical significance required
the minimum size of effect that it is reasonable to expect.

14. How Do We Measure Effect Size?
Cohen's d
Defined as the difference between the means for the two groups, divided by an estimate of the standard deviation in the population.
Often we use the average of the standard deviations of the samples as a rough guide for the latter.

15. Cohen's Rules Of Thumb For Effect Size
Correlation coefficient
Difference between means
“Small effect”
r = 0.1
d = 0.2 standard deviations
“Medium effect”
r = 0.3
d = 0.5 standard deviations
“Large effect”
r = 0.5
d = 0.8 standard deviations

16. Calculating Cohen’s d
Cohen, J., (1977). Statistical power analysis for the behavioural sciences. San Diego, CA: Academic Press.
Cohen, J., (1992). A Power Primer. Psychological Bulletin 16

17. Calculating Cohen’s d 17

18. Calculating Cohen’s d from a t test
Interpreting Cohen's d effect size: an interactive visualization 18

19. Conventions And Decisions About Statistical Power
Acceptable risk of a Type II error is often set at 1 in 5, i.e., a probability of 0.2 (β).
The conventionally uncontroversial value for “adequate” statistical power is therefore set at  = 0.8.
People often regard the minimum acceptable statistical power for a proposed study as being an 80% chance of an effect that really exists showing up as a significant finding.
Understanding Statistical Power and Significance Testing — an Interactive Visualization

20. 6 Steps to determine to determine an appropriate sample size for my study?
1. Formulate the study. Here you detail your study design, choose the outcome summary, and you specify the analysis method.
2. Specify analysis parameters. The analysis parameters, for instance are the test significance level, specifying whether it is a 1 or 2-sided test, and also, what exactly it is you are looking for from your analysis. 20

21. 6 Steps to determine to determine an appropriate sample size for my study?
Specify effect size for test. This could be the expected effect size (often a best estimate), or one could use the effect size that is deemed to be clinically meaningful.
Compute sample size or power. Once you have completed steps one through three you are now in a position to compute the sample size or the power for your study. 21

22. 6 Steps to determine to determine an appropriate sample size for my study?
5. Sensitivity analysis. Here you compute your sample size or power using multiple scenarios to examine the relationship between the study parameters on either the power or the sample size. Essentially conducting a what-if analysis to assess how sensitive the power or required sample size is to other factors. 22

23. 6 Steps to determine to determine an appropriate sample size for my study?
6. Choose an appropriate power or sample size, and document this in your study design protocol.
However other authors suggest 5 steps (a, b, c or d)!
Other options are also available! 23

24. A Couple Of Useful Links
For an article casting doubts on scientific precision and power, see The Economist 19 Oct “I see a train wreck looming,” warned Daniel Kahneman. Also an interesting read The Economist 19 Oct 2013 on the reviewing process.
A collection of online power calculator web pages for specific kinds of tests.
Java applets for power and sample size, select the analysis.

25. Statistical Power Analysis In Minitab
Next Week
Statistical Power Analysis In Minitab
Note that GPower3.1 is installed on University Machines. It is more complex to use than Minitab, but does provide a wider range of tests. For further information see the link, and look down for the desired software.

26. Statistical Power Analysis In Minitab
Minitab is available via RAS
Stat > Power and Sample Size >

27. Statistical Power Analysis In Minitab
Recall that a comparison of two proportions equates to analysing a 2×2 contingency table.
Note that you might find web tools for other models.
The alternative normally involves solving some very complex equations.

28. Statistical Power Analysis In Minitab
Note that you might find web tools for other models.
The alternative normally involves solving some very complex equations.
Simple statistical correlation analysis online
See Test 28 in the Handbook of Parametric and Nonparametric Statistical Procedures, Third Edition by David J Sheskin

29. Factors That Influence Power
Sample Size
alpha  
the standard deviation

30. Using Minitab To Calculate Power And Minimum Sample Size
Suppose we have two samples, each with n = 13, and we propose to use the 0.05 significance level
Difference between means is 0.8 standard deviations (i.e., Cohen's d = 0.8), so a t test
All key strokes in printed notes

31. Using Minitab To Calculate Power And Minimum Sample Size
Note that all parameters, bar one are required.
Leave one field blank.
This will be estimated.

32. Using Minitab To Calculate Power And Minimum Sample Size
Power and Sample Size
2-Sample t Test
Testing mean 1 = mean 2 (versus not =)
Calculating power for mean 1 = mean 2 + difference
Alpha = 0.05 Assumed standard deviation = 1
Sample
Difference Size Power
The sample size is for each group.
Power will be

33. Using Minitab To Calculate Power And Minimum Sample Size
If, in the population, there really is a difference of 0.8 between the members of the two categories that would be sampled in the two groups, then using sample sizes of 13 each will have a 49.92% chance of getting a result that will be significant at the 0.05 level.

34. Using Minitab To Calculate Power And Minimum Sample Size
Suppose the difference between the means is 0.8 standard deviations (i.e., Cohen's d = 0.8)
Suppose that we require a power of 0.8 (the conventional value)
Suppose we intend doing a one-tailed t test, with significance level 0.05.
All key strokes in printed notes

35. Using Minitab To Calculate Power And Minimum Sample Size
Select “Options” to set a
one-tailed test

36. Using Minitab To Calculate Power And Minimum Sample Size

37. Using Minitab To Calculate Power And Minimum Sample Size
Power and Sample Size
2-Sample t Test
Testing mean 1 = mean 2 (versus >)
Calculating power for mean 1 = mean 2 + difference
Alpha = 0.05 Assumed standard deviation = 1
Sample Target
Difference Size Power Actual Power
The sample size is for each group.
Target power of at least 0.8

38. Using Minitab To Calculate Power And Minimum Sample Size
Power and Sample Size
2-Sample t Test
Testing mean 1 = mean 2 (versus >)
Calculating power for mean 1 = mean 2 + difference
Alpha = 0.05 Assumed standard deviation = 1
Sample Target
Difference Size Power Actual Power
The sample size is for each group.
At least 21 cases in each group

39. Using Minitab To Calculate Power And Minimum Sample Size
Power and Sample Size
2-Sample t Test
Testing mean 1 = mean 2 (versus >)
Calculating power for mean 1 = mean 2 + difference
Alpha = 0.05 Assumed standard deviation = 1
Sample Target
Difference Size Power Actual Power
The sample size is for each group.
Actual power

40. Using Minitab To Calculate Power And Minimum Sample Size
Suppose you are about to undertake an investigation to determine whether or not 4 treatments affect the yield of a product using 5 observations per treatment. You know that the mean of the control group should be around 8, and you would like to find significant differences of +4. Thus, the maximum difference you are considering is 4 units. Previous research suggests the population σ is So an ANOVA.

41. Using Minitab To Calculate Power And Minimum Sample Size

42. Using Minitab To Calculate Power And Minimum Sample Size
Power and Sample Size
One-way ANOVA
Alpha = 0.05
Assumed standard deviation = 1.64
Number of Levels = 4
SS Sample Maximum
Means Size Power Difference
The sample size is for each level.

43. Using Minitab To Calculate Power And Minimum Sample Size
To interpret the results, if you assign five observations to each treatment level, you have a power of 0.83 to detect a difference of 4 units or more between the treatment means.
Minitab can also display the power curve of all possible combinations of maximum difference in mean detected and the power values for one-way ANOVA with the 5 samples per treatment.