1. Trial Sequential Analysis (TSA)
Type I and II errors from low risk of bias randomised controlled trials of postoperative analgesics: a trial sequential analysis
Brett Doleman, Jonathan Lund, John Williams
Introduction
Trial Sequential Analysis (TSA)
Discussion
Randomised controlled trials (RCTs) are the optimum method to evaluate medical interventions. However, if methodological standards are not adhered to, then this can bias their conclusions
Previous research has identified that deficiencies in methodology, such as inadequate allocation concealment and blinding can exaggerate results from RCTs [1,2]
Our recent research has shown this may also be the case for postoperative analgesics [3]
In addition, concerns have been raised recently regarding the validity of meta-analyses of RCTs with sparse data [4], which may be overcome by using trial sequential analysis (TSA) (Fig 1)
Therefore, the aim of this study was to identify if sufficient data exists from low risk of bias trials to reject type I and II errors from meta-analyses of postoperative analgesics.
Only 10% of published RCTs of postoperative analgesics are at low risk of bias
We have previously demonstrated this can exaggerate effects in meta-analyses [3]
There is currently only sufficient low risk of bias evidence for the use of gabapentin in reducing 24-hour morphine consumption
Therefore, future trials are required that use low risk of bias methodology
Future reviews of postoperative analgesics such aim to use TSA of low risk of bias trials in order to inform clinical practice
Figure 1: Annotated TSA plot
A on the plot indicates the monitoring boundaries for adjusted statistical significance with a larger Z score (Y-axis) required to reach significance early in the review process (X-axis indicates participants) analogous to adjustment for multiple comparisons in RCTs
B indicates the traditional boundary for statistical significance (p<0.05), which is equivalent to a Z score of 1.96
C indicates the required information size (IS) for a conclusive review analogous to a sample size calculation
D indicates the area of futility where the addition of further trials will unlikely show any benefit or harm
E shows the Z curve (blue) with each point indicating the addition of another trial, if this crosses A then the result is statistically significant adjusted for multiple comparisons and if it crosses C then an adequate sample size exists
Limitations
Poor reporting may mean some RCTs used low risk of bias methods but did not adequately report them
We identified RCTs from previous reviews and therefore these may be subject to publication bias
In addition, identifying RCTs from previous reviews may mean evidence is outdated
Methods
We undertook a systematic search of MEDLINE, EMBASE, Cinahl, AMED and the Cochrane Database of Systematic Reviews and identified RCTs from previous reviews
We included the following analgesics: paracetamol, NSAIDS and COX-2 inhibitors, ketamine, alpha-2 agonists, gabapentin, pregabalin, magnesium, lidocaine, nefopam, tramadol and dexamethasone
We undertook risk of bias assessment using the Cochrane risk of bias tool. Low risk trials were defined as those receiving low risk of bias for randomisation, allocation concealment, blinding and attrition bias
We then performed TSA using the outcome of 24-hour morphine consumption. We constructed O’Brien-Fleming monitoring boundaries (to reduce type I errors) and calculated the required information size (to reduce type II errors)
Conclusion
There is currently insufficient low risk of bias evidence for the use of many postoperative analgesics to exclude type I and II errors in analysis
Results
References
We identified 344 RCTs. Only 36 of these were regarded as low risk of bias (10%)
When performing TSA, there were too few data to construct monitoring boundaries and/or information sizes for alpha-2 agonists, pregabalin, nefopam and magnesium
Only the results for gabapentin (seven trials) crossed the monitoring boundaries and reached the required information size (344 participants)
[1] Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet 2002; 359:
[2] Wood L, Egger M, Gluud L et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. British Medical Journal 2008; 336:
[3] Doleman et al. Baseline morphine consumption may explain between-study heterogeneity and improve accuracy and precision of effect estimates in meta-analyses of adjuvant analgesics. Anesthesia and Analgesia [in press]
[4] Afshari A, Wetterslev J, Smith AF. Can systematic reviews with sparse data be trusted? Anaesthesia 2017; 72: 12–16.