1. Sample Size Estimation
Dr. Tuan V. Nguyen
Garvan Institute of Medical Research
Sydney, Australia

2. The classical hypothesis testing
Define a null hypothesis (H0) and a null hypothesis (H1)
Collect data (D)
Estimate p-value = P(D | H0)
If p-value > a, accept H0; if p-value < a, reject H0

3. Diagnosis and statistical reasoning
Disease status
Present Absent
Test result
+ve True +ve False +ve
(sensitivity)
-ve False -ve True -ve
(Specificity)
Significance Difference is
Present Absent
(Ho not true) (Ho is true)
Test result
Reject Ho No error Type I err.
1-b a
Accept Ho Type II err. No error
b 1-a
a : significance level
1-b : power

4. Study design issues Setting
Participants: inclusion / exclusion criteria
Design: survey, factorial, etc
Measurements: outcome, covariates
Analysis
Sample size / power issues

5. Sample size issues How many judges / consumers?
Practical and statistical issues
Ethical issues
Unnecessarily large number of judges may be deemed unethical
Too small a sample may also be unethical as the study can’t show anything.

6. Practical difference vs statistical significance
Outcome Group A Group B
Improved
No improved
Total
% improved 30% 60%
Outcome Group A Group B
Improved
No improved
Total
% improved 30% 60%
Chi-square: 5.4; P < 0.05
“Statistically significant”
Chi-square: 3.3; P > 0.05
“Statistically insignificant”

7. Effect of sample size on preference proportion
Number of judges
% preferred A (p)
Variance of p
Std Dev of p
Z test for p=0.5
P-value 20
0.65
0.0114
0.1067
1.41
0.160 30
0.0076
0.0871
1.72
0.085 40
0.0057
0.0754
1.99
0.047 50
0.0046
0.0675
2.22
0.026

8. Effect of sample size: a simulation
True mean: 100
True SD: 15
True mean: 100
True SD: 35
Sample size Est. M SD Est. M SD

9. What are required for sample size estimation?
Parameter (or outcome) of major interest
Magnitude of difference in the parameter
Variability of the parameter
Bound of errors (type I and type II error rates)

10. Parameter of interest Type of measurement of primary interest:
Continuous or categorical outcome
Examples:
Proportion: proportion (or probability) of preference for a product
Hedotic scale: 0-10
Nominal scale

11. Variability of the parameter of interest
If the parameter is a continuous variable:
What is the standard deviation (SD) ?
If the parameter is a categorical variable:
SD can be estimated from the proportion/probability.

12. Magnitude of difference of interest
Distinction between practical and statistical relevance.
Examples:
Probability of preference: 85% vs 50%
Tasting scores: difference between products by 1 SD.

13. The Normal distribution
0.95
0.95 Z2 Z1
-1.96
1.96
1.64
0.05
0.025
0.025
Prob. Z1 Z2

14. The Normal deviates Alpha Za Za/2 0.20 0.84 1.28 0.10 1.28 1.64
c (One-sided) (Two-sided)
Power Z1-b

15. Study design and outcome
Continuous measurement
Categorical outcome
Correlation
Single population
Two populations

16. Single group

17. Sample size for estimating a population mean
How close to the true mean
Confidence around the sample mean
Type I error.
N = (Za/2)2 s2 / d2
s: standard deviation
d: the accuracy of estimate (how close to the true mean).
Za/2: A Normal deviate reflects the type I error.
Example: we want to estimate the average weight in a population, and we want the error of estimation to be less than 2 kg of the true mean, with a probability of 95% (e.g., error rate of 5%).
N = (1.96)2 s2 / 22

18. Effect of standard deviation
Std Dev (s) Sample size
10 96
12 138
14 188
16 246
18 311
20 384

19. Sample size for estimating a population proportion
How close to the true proportion
Confidence around the sample proportion.
Type I error.
N = (Za/2)2 p(1-p) / d2
p: proportion to be estimated.
d: the accuracy of estimate (how close to the true proportion).
Za/2: A Normal deviate reflects the type I error.
Example: The proportion of preference for product A is around 80%. We want to estimate the preference p in a community within 5% with 95% confidence interval.
N = (1.96)2 (0.8)(0.2) / = 246 consumers.

20. Effect of accuracy N = (1.96)2 (0.3)(0.7) / 0.022 = 2017 subjects.
Example: The proportion of preference in the general population is around 30%. We want to estimate the prevalence p in a community within 2% with 95% confidence interval.
N = (1.96)2 (0.3)(0.7) / = 2017 subjects.

21. Sample size for estimating a correlation coeffcient
In observational studies which involve estimate a correlation (r) between two variables of interest, say, X and Y, a typical hypothesis is of the form:
Ho: r = 0 vs H1: r not equal to 0.
The test statistic is of the Fisher's z transformation, which can be written as:
Where n is the sample size and r is the observed correlation coefficient.
It can be shown that t is normally distributed with mean 0 and unit variance, and the sample size to detect a statistical significance of t can be derived as:

22. Sample size for estimating r: example
Example: According to the literature, the correlation between salt intake and systolic blood pressure is around A study is conducted to test the correlation in a population, with the significance level of 1% and power of 90%. The sample size for such a study can be estimated as follows:
A sample size of at least 87 subjects is required for the study.

23. Sample size for difference between two means
Hypotheses:
Ho: m1 = m2 vs. Ha: m1 = m2 + d
Let n1 and n2 be the sample sizes for group 1 and 2, respectively; N = n1 + n2 ; r = n1 / n2 ; s: standard deviation of the variable of interest.
Then, the total sample size is given by:
If we let Z = d / s be the “effect size”, then:
If n1 = n2 , power = 0.80, alpha = 0.05, then (Za + Z1-b)2 = ( )2 = 10.5, then the equation is reduced to:
Where Za and Z1-b are Normal deviates

24. Two-group comparisons

25. Sample size for two means vs.“effect size”
For a power of 80%, significance level of 5%

26. Sample size for difference between 2 proportions
Hypotheses:
Ho: p1 = p2 vs. Ha: p1 = p2 + d .
Let p1 and p2 be the sample proportions (e.g. estimates of p1 and p2) for group 1 and group 2. Then, the sample size to test the hypothesis is:
Where: n = sample size for each group ; p = (p1 + p2) / 2 ; Za and Z1-b are
Normal deviates
A better (more conservative) suggestion for sample size is:

27. Sample size for difference between 2 prevalence
For most diseases, the prevalence in the general population is small (e.g. 1 per 1000 subjects). Therefore, a difference formulation is required.
Let p1 and p2 be the prevalence for population 1 and population 2. Then, the sample size to test the hypothesis is:
Where: n = sample size for each group; Za and Z1-b are
Normal deviates.

28. Sample size for two proportions: example
Example: The preference for product A is expected to be 70%, and for product B 60%. A study is planned to show the difference at the significance level of 1% and power of 90%.
The sample size can be calculated as follows:
p1 = 0.6; p2 = 0.7; p = ( )/2 = 0.65; Z0.01 = 2.81; Z1-0.9 = 1.28.
The sample size required for each group should be:
Adjusted / conservative sample size is:

29. Sample size for two proportions vs. effect size
Difference from p1 by: P1
Note: these values are “unadjusted” sample sizes

30. Sample size for estimating an odds ratio
In case-control study the data are usually summarized by an odds ratio (OR), rather then difference between two proportions.
If p1 and p2 are the proportions of cases and controls, respectively, exposed to a risk factor, then:
If we know the proportion of exposure in the general population (p), the total sample size N for estimating an OR is:
Where r = n1 / n2 is the ratio of sample sizes for group 1 and group2; p is the prevalence of exposure in the controls; and OR is the hypothetical odds ratio. If n1 = n2 (so that r = 1) then the fomula is reduced to:

31. Sample size for an odds ratio: example
Example: The prevalence of vertebral fracture in a population is 25%. It is interested to estimate the effect of smoking on the fracture, with an odds ratio of 2, at the significance level of 5% (one-sided test) and power of 80%.
The total sample size for the study can be estimated by:

32. Sample size for 2 correlation coefficients
In detecting a relevant difference between two correlation coefficients r1 and r2 obtained from two independent samples of sizes n1 and n2, respectively, we need to firstly transform these coefficients into z value as follows:
The total sample size N required to detect the difference between two correlation coefficients r1 and r2, with a significance level of a and power 1-b, can be estimated by:
Where Za and Z1-b are Normal deviates

33. Sample size for two r’s: example
The sample size required to detect the difference between r1 = 0.8 and r2 = 0.4 with the significance level of 5% (two-tailed) and power of 80% can be solved as follows:
z1 = 0.5 ln ((1+0.4) / (1-0.4)) = 0.424
z1 = 0.5 ln ((1+0.8) / (1-0.8)) = 1.098
46 subjects is needed in each group.

34. Some comments The formulae presented are theoretical.
They are all based on the assumption of Normal distribution.
The estimator [of sample size] has its own variability.
The calculated sample size is only an approximation.
Non-response must be allowed for in the calculation.

35. Computer programs
Software program for sample size and power evaluation
PS (Power and Sample size), from Vanderbilt Medical Center. This can be obtained from me by sending to Free.
On-line calculator:
References:
Florey CD. Sample size for beginners. BMJ 1993 May 1;306(6886):1181-4
Day SJ, Graham DF. Sample size and power for comparing two or more treatment groups in clinical trials. BMJ 1989 Sep 9;299(6700):663-5.
Miller DK, Homan SM. Graphical aid for determining power of clinical trials involving two groups. BMJ 1988 Sep 10;297(6649):672-6
Campbell MJ, Julious SA, Altman DG. Estimating sample sizes for binary, ordered categorical, and continuous outcomes in two group comparisons. BMJ 1995 Oct 28;311(7013):
Sahai H, Khurshid A. Formulae and tables for the determination of sample sizes and power in clinical trials for testing differences in proportions for the two-sample design: a review. Stat Med 1996 Jan 15;15(1):1-21.
Kieser M, Hauschke D. Approximate sample sizes for testing hypotheses about the ratio and difference of two means. J Biopharm Stat 1999 Nov;9(4):