November 10, 2010DSTS meeting, Copenhagen1 Power and sample size calculations Michael Væth, University of Aarhus Introductory remarks Two-sample problem with normal data Comparison of two proportions Sample size and power calculations based on Wald’s test Two-sample problem with censored survival data Non-inferiority trials and equivalence trials Sample size and confidence intervals

November 10, 2010DSTS meeting, Copenhagen2 Power and sample size calculations “Investigators often ask statisticians how many observations they should make (fortunately, usually before the study begins). To be answerable, this question needs fuller formulation. There is resemblance to the question, How much money should I take when I go on vacation? Fuller information is needed there too. How long a vacation? Where? With whom?” Moses(NEJM,1985)

November 10, 2010DSTS meeting, Copenhagen3 A study should:  Allow conclusive answers to the questions being addressed  Provide estimates of relevant quantities with sufficient precision Standard approach Identify a maximal risk of wrong conclusions or Quantify the size of a sufficient precision Determine the minimum sample size for which the study achieves the design goals Power and sample size calculations

November 10, 2010DSTS meeting, Copenhagen4 Implementation of standard approach Use commercial special-purpose software Simulations Analytic methods Power and sample size calculations

November 10, 2010DSTS meeting, Copenhagen5 RCT, equal allocation probabilities Outcome follow a normal distribution. Means Common standard deviation, assumed known: Estimated treatment difference: Expected treatment difference Minimal relevant difference Hypothesis: Test statistic: Two-sample problem with continuous outcome

November 10, 2010DSTS meeting, Copenhagen6 If the test statistic has a standard normal distribution. In general, the test statistic is normal: mean and standard deviation 1 Two-sample problem with continuous outcome (2)

November 10, 2010DSTS meeting, Copenhagen7 A: Distribution of the test statistic when B: Distribution of the test statistic for an alternative value of Level of significance Power Two-sample problem with continuous outcome (3)

November 10, 2010DSTS meeting, Copenhagen8 Assume so only the upper term matters Two-sample problem with continuous outcome (4) Only contribution from one term unless power close to level of significance Basic relation

November 10, 2010DSTS meeting, Copenhagen9 Two-sample problem with continuous outcome (5) Sample size for given power Power for given sample size

November 10, 2010DSTS meeting, Copenhagen10 Two-sample problem with continuous outcome (6) Depends on the error probabilities Depend on the problem Level ofStatistical power significance50%80%90%95% 5% % % % for selected values ofTable of

November 10, 2010DSTS meeting, Copenhagen11 Comparison of two proportions Score test: Basic relation becomes with

November 10, 2010DSTS meeting, Copenhagen12 Comparison of two proportions (2) Wald’s test: Basic relation becomes The simple structure N = (model term)(error term) is recovered

November 10, 2010DSTS meeting, Copenhagen13 Comparison of two proportions (3) Example 1 N(Score) = 2894 N(Wald) = 2888 N(Score) = 4610 N(Wald) = 4278 N(Score) = 4422 N(Wald) = 4749 Other sample fractions

November 10, 2010DSTS meeting, Copenhagen14 Sample size and power calculations based on Wald’s test Data and Statistical model Question: Hypothesis about 1-dim. parameter Wald’s test with Sample size for given power Power for given sample size

November 10, 2010DSTS meeting, Copenhagen15 Sample size and power calculations based on Wald’s test (2) Example 1 (ctd.) Same problem, but now use Wald’s test based on ln(odds) N = 2906 N = 4778 N = 4304 Score Wald

November 10, 2010DSTS meeting, Copenhagen16 Sample size and power calculations based on Wald’s test (3) Use of simulations Computer generates a large number of independent sample of size from a scenario representing a relevant difference Power estimated as proportion of samples for which Wald’s test is statistically significant at level Sample size for power level

November 10, 2010DSTS meeting, Copenhagen17 Sample size and power calculations based on Wald’s test (4) Use of simulations Sample size multiplier

November 10, 2010DSTS meeting, Copenhagen18 Two-sample problem with censored survival data Time-to-event data Two sample, proportional hazards model Hazard rates Parameter of interest Wald’s test with the number of events in group i

November 10, 2010DSTS meeting, Copenhagen19 Wald’s test is approximately normal with sd = 1 and mean average probability of an event in group i Sample size Two-samples with censored data (2) Sample size depends primarily on number of events

November 10, 2010DSTS meeting, Copenhagen20 Two-samples with censored data (3) Example 2 Design of a RCT with survival endpoint Comparison of new and standard treatment Endpoint: All-cause mortality Requirements: max. 6 years; power = 80% for HR = A T = A + F Study start Accrual ends Study ends Accrual period Follow-up period No additional follow-upIn general

November 10, 2010DSTS meeting, Copenhagen21 Two-samples with censored data (3) Example 2 (ctd.) KM-estimate: standard treatment1 - KM Std. Treatment: Average event probability = AUC/baseline Average event probability with new treatment

November 10, 2010DSTS meeting, Copenhagen22 Two-samples with censored data (4) Example 2 (ctd.) AccrualFollow-upTotalAverage mortality probabilityNumber ofPatients years StandardNewOverallpatientsper year designs with the the same expected number of events (635) 635 events are needed to meet the design requirements This can be achieved in different ways Competing risk: Replace 1-KM with Cumulative Incidence

November 10, 2010DSTS meeting, Copenhagen23 Non-inferiority & equivalence trials Null hypothesis Minimal relevant difference  Maximal irrelevant difference

November 10, 2010DSTS meeting, Copenhagen24 Non-inferiority & equivalence trials Non-inferiority: a one-sided hypothesis Two-sample problem with normal data (Wald’s test approach for 1-parameter problem) Basic relation Sample size Note: If the power is assessed at a zero difference, then the sample size needed to achieve this power will be underestimated if the effect of the new product is less than that of the active control

November 10, 2010DSTS meeting, Copenhagen25 Non-inferiority & equivalence trials Equivalence: union-intersection test Two one-sided tests Basic relations Sample size: is specified for Note: If the power is assessed at a zero difference, then the sample size needed to achieve this power is underestimated if the true difference is not zero.

November 10, 2010DSTS meeting, Copenhagen26 Sample size and confidence intervals Design phase: Sample size considerations are traditionally phrased in the terminology of hypothesis testing Formulas are derived by controlling error probabilities Reporting and interpreting results Focus on estimates and confidence intervals Hypothesis tests are downplayed Why not use the same approach on both occasions?

November 10, 2010DSTS meeting, Copenhagen27 Sample size and confidence intervals Power calculations when reporting the results? Probability statements should utilize the collected data and not be based on anticipated values of the parameters. Interpretation: Probability of replication Some statistical packages provide calculation of ”post-hoc power” or ”observed power”, i.e. Power computed at the estimated parameter value. This does not make sense. The power becomes a (known) function of the significance level.

November 10, 2010DSTS meeting, Copenhagen28 Sample size and confidence intervals Sample size calculations based on confidence intervals? Two-sample problem with normal data (Wald’s test approach for 1-parameter problem) 95% confidence interval Choose smallest N such that a confidence interval centered at excludes 0 Corresponds to power = 0.50

November 10, 2010DSTS meeting, Copenhagen29 Sample size and confidence intervals Use the fundamental relation between hypothesis test and confidence intervals to formulate the sample size requirements in confidence interval terminology Greenland(AJE,1988), Daly(BMJ,1991) To compute a sample size specify 1.The confidence level 2.The minimum size parameter-value that we wish to estimate unambigously, i.e. with a confidence interval that excluded the null value 3.The probability of achieving this if the true value is the this minimum value

November 10, 2010DSTS meeting, Copenhagen30 "How not to collaborate with a biostatistician”How not to collaborate with a biostatistician Power and sample size calculations A ”commentary” on the world-wide-web: