Presentation is loading. Please wait.

Presentation is loading. Please wait.

Sample Size Calculation PD Dr. Rolf Lefering IFOM - Institut für Forschung in der Operativen Medizin Universität Witten/Herdecke Campus Köln-Merheim.

Similar presentations

Presentation on theme: "Sample Size Calculation PD Dr. Rolf Lefering IFOM - Institut für Forschung in der Operativen Medizin Universität Witten/Herdecke Campus Köln-Merheim."— Presentation transcript:

1 Sample Size Calculation PD Dr. Rolf Lefering IFOM - Institut für Forschung in der Operativen Medizin Universität Witten/Herdecke Campus Köln-Merheim

2 sample size Sample Size Calculation uncertainty costs & effort & time

3 Sample Size Calculation Single study group - continuous measurement - count of events Comparative trial (2 or more groups) - continuous measurement - count of events

4 Which true value is compatible with the observation? Confidence interval... range where the true value lies with a high probability (usually 95%) Confidence Interval

5 Example: 56 patients with open fractures, 9 developed an infection (16%) n=56 sample all patients with open fractures infection rate: 16% true value ???

6 Formula for event rates n = sample size p = percentage Example: n = 56 p = 16% CI 95 = 16 +/- 1,96 * (16*84) / 56 = 16 +/- 9,6 [ 6,4 - 25,6 ] Confidence Interval p * (100 - p) CI 95 = P +/- 1,96 * n

7 Confidence Interval 95% confidence interval around a 20% incidence rate

8 CI 95 = M 1,96 * SE Mean: M = mean SE = standard error SD = standard deviation n = sample size Remember: SE = SD / n 1,65 für 90% 1,96 für 95% 2,58 für 99% Formula for continuous variables Confidence Interval

9 Sample Size Calculation Comparative trials

10 What is the sample size to show that early weight-bearing therapy is better ? Which key should I press here now ? What is the sample size to show that early weight bearing therapy, as compared to standard therapy, is able to reduce the time until return to work from 10 weeks to 8 weeks, where time to work has a SD of 3 ? 36 cases per group !

11 Outcome Measures Wound infection Wellbeing Pain Sepsis Beweglichkeit Inedpemdence, autonomy Hospital stay Organ failure Recurrence rate Blood pressure Fear Fatigue Anxiety Social status Lab values Survival Complications Depressionen

12 Relevance Does this endpoint convince the patient / the scientific community? Reliability; measurability Could the outcome easily be measured, without much variation, also by different people? Sensitivity Does the intervention lead to a significant change in the outcome measure? Robustness How much is the endpoint influenced by other factors? Select Outcome Measure

13 Primary endpoint Main hypothesis or core question; aim of the study Statistics:confirmative Secondary endpoints Other interesting questions, additional endpoints Statistics:explorative (could be confirmative in case of a large difference) Advantage:prospective selection in the study protocol Retrospektively selected endpoints Selected when the trial is done, based on subgroup differences Statistics:ONLY explorative ! Select Outcome Measure

14 Sample size Certainty - error Power Difference to be detected Sample Size Calculation

15 A statistical test is a method (or tool) to decide whether an observed difference* is really present or just based on variation by chance *this is true for a test for difference which is the most frequently applied one in medicine Statistical Testing

16 Test for difference Intervention A is better than B Test for equivalence Intervention A and B have the same effect Test for non- inferiority Intervention A is not worse than B Statistical Testing

17 How a test procedure works 1. Want to show: there is a difference 2. Assume: there is NO difference between the groups; (equal effects, null-hypothesis) 3. Try to disprove this assumption: - perform study / experiment - measure the difference 4. Calculate: the probability that such a difference could occur although the assumption (no difference) was true = p-value Statistical Testing

18 statistical test for difference: The p-value is the probability for the case that the observed difference occured just by chance Statistical Testing

19 statistical test for difference : p is the probability forno difference Statistical Testing

20 Germany and Spain are equally strong soccer teams ! Game tonight: 6 : 0 für Germany Null hypothesis trial n=6 statistical test: p = 0,031 p-value says: How big is the chance that one of two equally strong teams scores 6 goals, and the other one none. Spain could still be equally strong as Germany, but the chance is small (3,1%) Statistical Testing

21 small difference large difference large sample small sample p=0,68 p=0,05 p<0,001 p=0,05 Statistical Testing

22 The more cases are included, the better could equality be disproved Example:drug A has a success rate of 80%, while drug B is better with a healing rate of 90% 208/109/100, /2018/200, /5045/500, /10090/1000, /200180/2000, /500450/500<0,001 drug Adrug B sample size80%90%p-value Statistical Testing

23 A significant p-value... does NOT prove the size of the difference, but only excludes equality! Statistical Testing

24 p-value large (>0.05) The observed difference is probably caused by chance only, or the sample size in not sufficient to exclude chance null-hypothesis in maintained no difference p-value small ( 0.05) chance alone is not sufficient to explain this difference there is a systematic difference null-hypothesis is rejected significant difference p-value Statistical Testing

25 The decision - for a difference (significance, p 0.05) - or against it (equality, not significant, p > 0.05) is not certain but only a probability (p-value). Therefore, errors are possible: Type 1 error: Decision for a difference although there is none => wrong finding Type 2 error: Decision for equality although there is one => missed finding Errors Statistical Testing

26 Errors Truth Test difference difference significant type 1 error wrong finding not significant missed finding type 2 error

27 type 1 error type 2 error wrong findingmissed finding Fire detectorwrong alarmno alarm in case of fire Courtconviction ofset a an innocentcriminal free Clinical studydifference difference was significant was missed by chance Statistical Testing

28 What is the Power of the study ? Type 2 error probability to miss a difference Power = 1 - probability to detect a difference Power depends on: - the magnitude of a difference - the sample size - the variation of the outcome measure - the significance level ( ) Power

29 What is the Power of the study ? POWER is the probability to detect a certain difference X with the given sample size n as significant (at level ). Does the study have enough power to detect a difference of size X ? Power

30 When to perform power calculations? 1.Planning phase – sample size calculation: if the assumed difference really exists, what risk would I take to miss this difference ? 2.Final analysis – in case of a non-significant result: what size of difference could be rejected with the present data ? Power

31 Example Clinical trial:Laparoscopic versus open appendectomy Endpoint:Maximum post-operative pain intensity (VAS points) Patients:30 cases per group Results:lap.: 28 (SD 18) open:32 (SD 17) p = 0.38 not significant ! What is the power of the study ??? Power

32 Sample size Certainty - error Power Difference to be detected Sample Size Calculation

33 Sample size = 0.05 = 0.20 Difference to be detected errorRisk to find a difference by chance errorRisk to miss a real difference Sample Size Calculation

34 Sample size = 0.05 = 0.20 P T & P C or Difference & SD Event rates: Percentages in the treatment and the control group Continuous measures: difference of means and standard deviation Sample Size Calculation

35 SD unknown if the variation (standard deviation) is not known, the expected advantage could be expressed as effect size which is the difference in units of the (unknown) SD Example: pain values are at least 1 SD below the control group (effect size = 1.0) the difference will be at least half a SD (effect size = 0.5) Continuous Endpoints Sample Size Calculation

36 Test with non-parametric rank statistics non-normal distribution, or non-metric values Mann-Whitney U-test; Wilcoxon test Use t-Test for sample size calculation and add 10% of cases Sample Size Calculation Continuous Endpoints

37 Guess … How many patients are needed to show that a new intervention is able to reduce the complication rate from 20% to 14% ? ( =0.05; =0.20, i.e. 80% power) Sample Size Calculation

38 Dupont WD, Plummer WD Power and Sample Size Calculations: A Review and Computer Program Contr. Clin. Trials (1990) 11: Sample Size Calculation


40 Multiple Testing Mehr als eine Versuchs-/Therapiegruppe Mehrere Zielgrößen Mehrere Follow-Up Zeitpunkte Zwischenauswertungen Subgruppen-Analysen Multiple testing increases the risk of arbitrary significant results Overall statistical error in 8 tests at the 0.05 level: α = = 1 - 0,66 = 0.34

41 correctat least 1 error 1 test (with 5% error) 95%5% 2 tests (with 5% error each) 90,25%9,75% 3 tests 4 tests 5 tests ….. 90,25% 4,75% 0,25% Multiple Testing

42 correctat least 1 error 1 test (with 5% error) 95%5% 2 tests (with 5% error each) 90,2%9,8% 3 tests85,7%14,3% 4 tests81,5%18,5% 5 tests77,4%22,6% ….. Multiple Testing

43 Select ONE primary and multiple secondary questions Combination of endpoints multiple complications Negative event multiple time points AUC, maximum value, time to normal multiple endpoints sum score acc. to OBrian Adjustment of p-values, i.e. each endpoint is tested with a stronger α level e.g. Bonferroni: k tests at level α / k (5 tests at the 1% level, instead of 1 Test at 5% level) A priori ordered hypotheses predefine the order of tests (each at 5% level) What could you do? Multiple Testing

44 Fixed sample size end of trial Sequential designafter each case Group sequential designafter each step Adaptive designafter each step Interim Analysis

45 aus:TR Flemming, DP Harrington, PC OBrian Design of group sequential tests. Contr. Clin Trials (1984) 5: Interim Analysis

Download ppt "Sample Size Calculation PD Dr. Rolf Lefering IFOM - Institut für Forschung in der Operativen Medizin Universität Witten/Herdecke Campus Köln-Merheim."

Similar presentations

Ads by Google