Presentation is loading. Please wait.

Presentation is loading. Please wait.

How many patients do I need for my study? Realistic Sample Size Estimates for Clinical Trials.

Similar presentations


Presentation on theme: "How many patients do I need for my study? Realistic Sample Size Estimates for Clinical Trials."— Presentation transcript:

1 How many patients do I need for my study? Realistic Sample Size Estimates for Clinical Trials

2 Sample Size Estimation 1. General considerations 2. Continuous response variable –Parallel group comparisons Comparison of response after a specified period of follow-up Comparison of changes from baseline –Crossover study 3.Success/failure response variable –Impact of non-compliance, lag –Realistic estimates of control event rate (Pc) and event rate pattern –Use of epidemiological data to obtain realistic estimates of experimental group event rate (Pe) 4. Time to event designs and variable follow-up

3 Useful References Lachin JM, Cont Clin Trials, 2:93-113, 1981 (a general overview) Shih J, Cont Clin Trials, 16:395-407, 1995 (time to event studies with dropouts, dropins, and lag issues) – see size program on biostatistics network Farrington CP and Manning G, Stat Med, 9:1447- 1454, 1990 (sample size for equivalence trials) Whitehead J, Stat Med, 12:2257-2271, 1993 (sample size for ordinal outcomes) Donner A, Amer J Epid, 114:906-914, 1981 (sample size for cluster randomized trials)

4 Key Points Sample size should be specified in advance (often it is not) Sample size estimation requires collaboration and some time to do it right (not solely a statistical exercise) Often sample size is based on uncertain assumptions (estimates should consider a range of values for key parameters and the impact on power for small deviations in final assumptions should be considered) Parameters that do not involve the treatment difference (e.g., SD) on which sample size was based should be evaluated by protocol leaders (who are blinded to treatment differences) during the trial It pays to be conservative; however, ultimate size and duration of a study involves compromises, e.g., power, costs, timeliness.

5 Some Evidence that Sample Size is Not Considered Carefully: A Survey of 71 “Negative” Trials (Freiman et al., NEJM, 1978) Authors stated “no difference” P-value > 0.10 (2-sided) Success/failure endpoint Expected number of events >5 in control and experimental groups Using the stated Type I error and control group event rate, power was determined corresponding to: –25% difference between groups –50% difference between groups

6 0-910-1920-2930-3940-4950-5960-6970-7980-8990-99 0 5 10 15 20 25 Power (1 - ß) 5.63% 25% Reduction Frequency Distribution of Power Estimates for 71 “Negative” Trials References: Frieman et al, NEJM 1978.

7 0-910-1920-2930-3940-4950-5960-6970-7980-8990-99 0 5 10 15 20 25 Power (1 - ß) 29.58% Frequency Distribution of Power Estimates for 71 “Negative” Trials 50% Reduction References: Frieman et al, NEJM 1978.

8 Implications of Review by Frieman et al. Many investigations do not estimate sample size in advance Many studies should never have been initiated; some were stopped too soon “Non-significant” difference does not mean there is not an important difference Design estimates (in Methods) are important to interpret study findings Confidence intervals should be used to summarize treatment differences

9 Studies with Power to Detect 25% and 50% Differences 6 6 6 6 l l l l 0 5 10 15 20 25 30 35 40 45 50 Percent of Studies with at Least 80% Power 6 25% Difference l 50% Difference 1975198019851990 Moher et al,JAMA, 272:122-124,1994

10 These Results Emphasize the Importance of Understanding that the Size of P-Value Depends on: Magnitude of difference (strength of association); and Sample size “Absence of evidence is not evidence of absence”, Altman and Bland, BMJ 1995; 311:485.

11 Steps in Planning a Study 1)Specify the precise research question 2)Define target population 3)Assess feasibility of studying question (compute sample size) 4)Decide how to recruit study participants, e.g., single center, multi-center, and make sure you have back-up plans

12 Beginning: A Protocol Stating Null and Alternative Hypotheses Along with Significance Level and Power Null hypothesis (H O ) Hypothesis of no difference or no association Alternative hypothesis (H A ) Hypothesis that there is a specified difference (Δ) No direction specified (2-tailed) A direction specified (1-tailed) Significance Level (  ): Type I Error The probability of rejecting H 0 given that H 0 is true Power = (1 -  ): (  = Type II Error) Power is the probability of rejecting H 0 when the true difference is Δ

13 End: Test of Significance According to Protocol Statistically Significant? YesNo Reject H O Do not reject H O Sampling variation is an unlikely explanation for the discrepancy Sampling variation is a likely explanation for the discrepancy

14 Normal Distribution If Z is large (lies in yellow area), we assume difference in means is unlikely to have come from a distribution with mean zero.

15 Continuous Outcome Example Observations:Many people have stage 1 (mild) hypertension (SBP 140-159 or DBP 90-99 mmHg) For most, treatment is life-long Many drugs which lower BP produce undesirable symptoms and metabolic effects (new drugs are needed) ResearchCan new drug T adequately control BP for patients Question:with mild hypertension? Objective:To compare new drug T with diuretic treatment for lowering diastolic blood pressure (DBP)

16 Parallel Group Design Comparing Average Diastolic BP (DBP) After One Year HypothesisH O :DBP after one year of treatment with new drug T equals the DBP for patients given a diuretic (control) H A :DBP after one year is different for patients given new drug T compared to diuretic treatment (difference is 4 mmHg or more) Study Population: Those with mild hypertension Drug TDBP at year 1DiureticDBP at year 1

17 Parallel Group Design Comparing Average Difference (Year 1 – Baseline) in DBP. HypothesisH O :DBP change from baseline after one year of treatment with new Drug T equals the DBP change from baseline after one year for patients given a diuretic (control) H A :DBP change from baseline after one year of treatment with new Drug T is different than the DBP change from baseline after one year for patients given a diuretic (control) treatment (difference is 4 mmHg or more) Study Population: Those with mild hypertension Drug T Change in DBP(Year 1 – Baseline) Diuretic Change in DBP (Year 1 – Baseline)

18 Why Δ= 4 mmHg? An important difference on a population-wide basis Clinical trials (Lancet 1990;335:827-38) 14 randomized trials; 36,908 participants 5-6 mmHg DBP difference (treatment vs. control) 28% reduction in fatal/non-fatal CVD Observational studies (Lancet 2002;360:1903-13) 58 studies; 958,074 participants 5 mm Hg lower DBP among those 40-59 years 41% (30%) lower risk of death from stroke (CHD)

19 Considerations in Specifying Treatments Effect (Delta) Smallest difference of clinical significance/interest Stage of research Realistic and plausible estimates based on: –previous research –expected non-compliance and switchover rates –consideration of type of participants to be studied Resources (compromise) Delta is a difference that is important NOT to miss if present.

20 Principal Determinants of Sample Size Size of difference considered important (Delta) Type I error (  ) or significance level Type II error (  ), or power (1-  ) Variability of response/frequency of event Constants

21 Sample Size for Two Groups: Equal Allocation General Formula 2 x Variability x [Constant ( ,  )] 2 Delta 2 N Per Group = Delta = Δ = clinically relevant and plausible treatment difference

22 Sample Size Formula Derivation: One Sample Situation

23 Sample Size Derivation (cont.)

24 Weighing the Errors  Type 2 error: Sponsor’s concern Type 1 error: Regulator’s Concern

25 0.05(1.96)0.80(0.84)7.84 0.90(1.28)10.50 0.95(1.645)13.00 0.01(2.575)0.80(0.84)11.67 0.90(1.28)14.86 0.95(1.645)17.81 Typical Values for (Z 1-  /2 + Z 1-  ) 2 Which Is Numerator of Sample Size Type I Error (  ) or Significance Level (Z 1-  /2 ) 2-sided test Power (1 -  ) (Z 1-  ) (Z 1-  /2 + Z 1-  ) 2

26

27 Example Hypertension Study HOHO 11 22 11 22 :=;-= 0 HAHA 11 22 11 22 :≠;-= 4 mmHg HOHO HAHA  04 mmHg Usually formulated in terms of change from baseline (e.g., H o = D 1 - D 2 = 0)

28 Another Derivation Solve for N using these 2 equations and by noting that Δ = sum of 2 parts from the previous figure.

29 Sources of Variability of BP Measurements Ref: Rose GA. Standardization of Observers in Blood Pressure Measurement. Lancet 1965;1:673-4. Variability of blood pressure readings True variations in arterial pressure Known factors Unknown factors Recent physical activity Emotional state Position of subject and arm Room temperature and season of year Measurement errors Instrument Observer Inaccuracy of sphygmomanometer Cuff width and length Chiefly affecting the mean pressure estimate Distorting the frequency distribution curve (and sometimes affecting the mean) Mental concentration Hearing acuity Confusion of auditory and visual Interpretation of sounds Rates of inflation and deflation Reading of moving column Terminal digit preference Prejudice, e.g., excess of readings at 120/80

30 Estimates of Variability for Diastolic Blood Pressure Measurements (MRFIT) Estimated Using Random-Zero (R-Z) Readings Estimate Variance Component(mmHg) 2 Between Subject58.4 Within Subjects36.3 ss 2  ee 2 

31 Estimates of Variability for Diastolic Blood Pressure Measurements Estimated Using Random-Zero (R-Z) Readings at Screen 2 and Screen 3 in MRFIT (2 Readings at Each Visit) Estimate Variance Component (mmHg) 2 Between Subject 58.4 Between Visits 26.1 Between Readings 10.2 ss 2  vv 2 ee 2 Within subject analyzed further

32 Consequences on Sample Size of Using Multiple Readings for Defining Diastolic BP  =0.05, 1-  =0.90 Inter-subject variability=58.4 (mmHg) 2 No. of visitsreadings/visit∆ = 8∆ = 4 1131124 1230118 2125100 222497 Between visit variability = 26.1 (mmHg) 2 Within visit variability = 10.2 (mmHg) 2 N per Group

33 Parallel Group Design Comparing Average DBP After One Year. HypothesisH O :DBP after one year of treatment with new Drug T equals the DBP for patients given a diuretic (control) H A :DBP after one year is different for patients given new Drug T compared to diuretic treatment (difference is 4 mmHg or more) Study Population: Those with mild hypertension Drug TDBP at year 1DiureticDBP at year 1

34 Parallel Group Studies Comparing Average DBP After One Year 1 measure, 1 visit (  =0.05,  =.10)  2 =58.4 + 26.1+10.2=94.7  =4 mmHg  =8 mmHg

35 Parallel Group Design Comparing Average Difference (Year 1 – Baseline) in DBP. HypothesisH O :DBP change from baseline after one year of treatment with new Drug T equals the DBP change from baseline after one year for patients given a diuretic (control) H A : DBP change from baseline after one year of treatment with new Drug T is different than the DBP change from baseline after one year for patients given a diuretic (control) treatment (difference is 4 mmHg or more) (2-Tailed) Study Population: Those with mild hypertension Drug T Change in DBP (Year 1 – Baseline) Diuretic Change in DBP (Year 1 – Baseline)

36 Sample Size for Two Groups: Equal Allocation General Formula 2 x Variability x [Constant ( ,  )] 2 Delta 2 N Per Group = Delta = Δ = clinically relevant and plausible treatment difference

37 Estimate of Variability for Change Outcome Prior studies ( For MRFIT, SD of DBP change after 12 months = 9.0 mmHg [baseline is one visit, 2 readings; follow-up is one visit, 2 readings]. For comparison, SD of 12 month DBP is 9.5 mmHg) Use correlation (ρ) of repeat readings for participants to estimate  e 2. (For MRFIT, correlation of DBP at baseline and 12 months is 0.55; note that SD (diff) can be written as 2σ T 2 (1-ρ) = 2σ e 2 = 2(81)(1-0.55) = 72.9 (SD of change ≈ 8.5 mmHg) Estimate of SD change using analysis of covariance (regression of change on baseline) (For MRFIT, SD = 7.9 mmHg)

38

39 Study Population: Those with mild hypertension Drug TWashout PeriodDiuretic Washout PeriodDrug T Crossover Group Design Comparing Average Difference (Diuretic – Drug T) in DBP HypothesisH O :Average of paired differences for the two treatment sequences differences is zero. H A : Average is 4 mmHg or more)

40 Crossover Study Design 12Diff. Iy 1 y 2 d l II y 1 y 2 d ll Period Var(d l ) = 2 ee 2 Var(d ll ) = 2 ee 2 = ∆ =  T -  C = E d l + d ll 2 –– D l + D ll 2

41 With parallel group comparison we had: or equivalently: With crossover we have: H O =  T –  C = 0 H O :  T =  C or H O : D T = D C where D l + D ll 2 H O == 0 D T and D C refer to the difference between follow-up and baseline levels of outcome

42 Variance for Sample Size Formula:

43 Substitution into Sample Size Formula Gives: n | = n || =number randomly allocated to each sequence - I (AB) or II (BA). This follows because the variance of the pooled treatment difference across the 2 sequences is ¼ (2  2 e + 2  2 e )

44 Crossover Sample Size Compared to Parallel Design (no baseline)

45 But the crossover design will require twice the number of measurements. So, if ρ= 0 then number of measurements are equal, but sample size for crossover is ½.

46 Consider an Experiment with Diastolic BP Response Type 1 error = 0.05 (2-sided) and Power = 0.95

47 Examples DBP (mmHg)58360.620.19 Cholesterol (mg/dl) 12004000.750.125 Overnight urine3256250.340.33 excretion Na+ (meq/8 hours) 2 overnights3253120.510.24 7 overnights325900.780.11 ss 2  ee 2  ncnc n

48

49

50 Sample size for  =.05 (2-sided) and  =.05 Parallel Number/group163724126 (no baseline) Parallel Baseline80362012 number/group (  =0.75) Crossover (Number/seq.)  = 0.0082362113  = 0.2562271510  = 0.504118107  = 0.7520953 0.81.00.40.6   

51 Key Points Sample size should be specified in advance Sample size estimation requires collaboration Often sample size is based on uncertain assumptions, therefore estimates should consider a range of values for key parameters (i.e., investigate the impact on power if sample size and treatment effect is not achieved) Parameters on which sample size is based should be evaluated during the trial It pays to be conservative; however, ultimate size and duration of a study involves compromises, e.g., power, costs, timeliness.

52

53 Power Prob (rej H o |when H A is true)= 1-  1-  = (rej H o |  1 -  2 =  ) = Prob x 1 - x 2 -0  1 2 n 1   2 2 n 2  1   2  1 -  2 =   x 1 - x 2 -0  1 2 n 1   2 2 n 2  1   2  1 -  2 =  Assume  1 2  2 2  2 andn 1  n 2  n A measure of how likely the study will detect a specific treatment difference (∆), if present.

54 Power (cont.) 1-  = Prob x 1 - x 2 -  2  2 n  1   2  0-  2  2 n  1 -  2 =                 x 1 - x 2 -  2  2 n  1   2  0-  2  2 n  1 -  2 =                  1   2   2  2 n                 1   2   2  2 n      Usually one of these probabilities will be very close to zero, depending on whether ∆ is positive or negative.

55 Power (cont.)

56 Sensitivity of Power to Variations in Other Sample Size Parameters (Assume  2 = 100) 0.054100-0.870.81 0.014100-0.250.60 0.056100-2.280.99 0.016100-1.670.95 0.054200-2.040.98 0.014200-1.4250.92  ∆n cc Power

57 Unequal Sample Sizes

58 Another Formulation: Unequal Allocation Comparison of means (Treatment C vs. E): P C and P E = fraction of patients assigned control (C) and experimental treatment (E); P C + P E = 1 Total N=  2 1 P C  1 P E         Z 1   2  Z 1          2 Δ 2

59 Total Sample Size for Different Allocation Ratios Allocation Ratio (E:C) ∆ (mm Hg)1:12:13:11:2 4250280332280 864708470 Sample sizes rounded up

60 Multiple Treatments and Unequal Allocation Example:m experimental treatments and control; comparison of means Problem: Find n which minimizes variance

61 Solution:Take derivative with respect to n and set = zero Then, No. of patients in control group = no. of patients in experimental times square root no. of treatments

62 Other Issues with Multiple Groups Multiple comparisons (  adjustment) Interim analyses – possible early termination of some, but not all, treatment groups

63 Minimum Clinically Important Difference (MCID) For a given sample size (N) the null hypothesis (H O : difference in means = 0) will be rejected if the observed difference (d) Chuang-Stein C et al. Pharmaceutical Stat 2010. d can be smaller than MCID and p<0.05!

64 Sample Size for Dual Criteria: Statistical Significance and Clinical Significance In some cases, you may want to establish with high probability that the treatment effect is as large as MCID –For example, a new HIV vaccine might be assumed to have 60% efficacy but the study is designed to have sufficient power to rule out efficacy lower than 30% –This will require a larger sample size –For example, if Δ=2MCID, then sample size is 4 times greater

65 Summary (General) It is important that sample size be large enough to achieve the goals of the study – too many studies are conducted which are under- powered. Sample size assumptions are frequently very rough so they should be re-evaluated as the study progresses. A good knowledge of the subject matter (background on disease and intervention, outcomes, and target population) is necessary to estimate sample size.

66 Summary (Crossover versus Parallel Group) Efficiency of crossover increases as  increases. Design using change from baseline as response is better than design which just uses follow-up responses if  > 0.50. With multiple measurements on each patient, to establish baseline and follow- up levels, sample size can be reduced.


Download ppt "How many patients do I need for my study? Realistic Sample Size Estimates for Clinical Trials."

Similar presentations


Ads by Google