Presentation on theme: "A Spreadsheet for Analysis of Straightforward Controlled Trials"— Presentation transcript:
1 A Spreadsheet for Analysis of Straightforward Controlled Trials Will G Hopkins Auckland University of Technology Auckland NZPreamble: controlled trials, crossovers, spreadsheetsControlled trials: unpaired t statistic, transformations, plots for non-uniformity, back-transformations, reliability, individual responses, comparison of groups in pre-test, uncertainties.Crossovers: paired t statisticexptalcontrolprepost1post2TrialYCABTreatmentY
2 Best design to determine effects of treatments. Controlled TrialsBest design to determine effects of treatments.Measurements at least once pre treatment and at least once during and/or post treatment.prepost1post2TrialYexptalcontrolControl and experimental groups.Outcome statistic is difference between groups in their mean change due to the experimental and control treatments.
3 All subjects receive all control and experimental treatments. CrossoversAll subjects receive all control and experimental treatments.CABTreatmentYAim for balance (equal number of subjects on each treatment order).Aim for enough time following each treatment to allow washout.Outcome statistic is the mean change between treatments.SpreadsheetsInstructive and save time.OK for straightforward designs.
4 Features of Spreadsheet for Controlled Trials Usual analysis of the raw values of the dependent variable.Based on the unequal-variances unpaired t statistic.Use for yes/no variables (score as 0 or 100) and Likert scales.Analysis of transformed values of the dependent variable.To reduce any systematic effect of an individual's pre-test value on the change due to the treatment.Log transformation for most physiological and performance measures, where effects are percents or factors.Square-root transformation for counts of injuries or events.Arcsine-root transformation for proportions.Percentile-rank transformation (= non-parametric analysis) when a transformation function is unclear or unspecifiable.
5 Another feature of Spreadsheet for Controlled Trials Plots of change scores of raw and transformed data against pre-test values.To check for outliers.To confirm that the chosen transformation results in a similar magnitude of change across the range of pre-test values.Achieve the same purpose as plots of residual vs predicted values in more powerful statistical packages.Addresses need to avoid heteroscedasticity = non-uniformity of error = non-uniformity in the effect of the treatment.If all pre-test values are similar, transformation is irrelevant, but…Choose a transformation to minimize the effect of potentially wide variation in pre-test values on the effect of the treatment.Beware of regression to the mean: lower pre-test values tend to produce more-positive changes.
6 More features of Spreadsheet for Controlled Trials Various solutions to the problem of back-transformation of treatment effects into meaningful magnitudes.Back transformation of logs into percents and factors.Novel approach: estimate the value of the effect at a chosen value of the raw variable. (No need with log transformation.)Cohen effects for raw analysis and all transformations.Estimates of reliability in the control group.Control group is a reliability study.For comparison with reliability studies.Typical error = (SD of change score)/2.Change in mean.A large change due to familiarization can account for large typical error via individual differences in familiarization.
7 Even more features of Spreadsheet for Controlled Trials Estimates of individual responses to the treatment.Expressed as a standard deviation for the mean effect.Example: effect of the treatment is typically 3.0 ± 2.0 units (mean ± SD)…where the SD = (diff in SD2 for change scores).For all transformations and back transformations.Comparison of pre-test values of means and standard deviations in the two groups.If means differ and plots show that the pre-test value affects change scores, do an ANOVA with pre-test as a covariate.Estimate the treatment effect at the mean value of the covariate.Use for comparison of independent groups in a non-repeated measures study. (Ignore all the change-score stats.)
8 Yet another feature of Spreadsheet for Controlled Trials Estimates of uncertainty expressed as confidence limits at any percent level (95%, 90%…) for all effects.Including confidence limits for standard deviations representing individual responses!A negative standard deviation implies no individual responses.There is no adjustment of p values for multiple comparisons.Such adjustment is a relic of hypothesis testing, but even so…It never applied to the most important pre-planned effect.Ignore the uncertainties for comparison of groups in the pre-test, because…What matters is how different the groups were, not how different their corresponding populations might be.But use the uncertainties for comparison of independent groups in non-repeated measures study .
9 One more feature of Spreadsheet for Controlled Trials Chances that the true value of an effect is importantYou provide a value for the effect that you consider is the smallest that would be important for your subjects.The spreadsheet estimates the chances that the true value is greater than this smallest important value.It also shows the chances in a qualitative form (unlikely, possible, almost certain…).The default smallest value for the Cohen effect size is 0.2.Try 0.6, 1.2, or 2.0 to estimate the chances that the true value is moderate, large, or very large; then state something like…"The mean effect could be trivial or small, but it is unlikely to be moderate and is almost certainly not large."Might help get your otherwise inconclusive study into a journal.
10 Features of Spreadsheet for Crossovers Can have more than one control and experiment treatment.Can use for a time series (= only one treatment).Based on paired t statistic.Uses column of zeros to pair with change scores, which…Allows analysis of other effects from within-subject modeling.No analysis of individual responses.But possible with two control treatments (preferably balanced) in a crossover or two baseline treatments in a time series.Typical error is provided for comparison with reliability study.But may be inflated by individual responses to treatment.Familiarization effect between trials can also inflate error, but…Need analysis via mixed modeling to reduce this error.
11 Article and spreadsheets available at: ConclusionCan't (yet) use the spreadsheet to estimate effects of covariates such as gender and age on the treatment effects.But the spreadsheets will work for most data and help you get more complex analyses right with a stats package.Article and spreadsheets available at:See Sportscience 7, 2003