1. Interventional trials
EM Critical Appraisal Day
13th July 2016
Tom Abbott | MRC/BJA Clinical Research Training Fellow
William Harvey Research Institute
@_tomabbott

2. Levels of Evidence
Belsey What is EBM?

3. Critical appraisal Population Intervention Comparator Outcome(s)
Study design
SIGN = Scottish intercollegiate guidelines network.

4. Effect measures Risk ratio = (a/[a+b]) / (c/[c+d])
Outcome – yes
Outcome - No
Exposure - Yes a b
Exposure - No c d
Risk ratio = (a/[a+b]) / (c/[c+d])
Odds ratio = (a/b) / (c/d) or ad/bc
RR ~ OR, except where small sample (different denominator) or where very common outcome (different denominator).

5. Risk ratio = (104/[104+1548]) / (193/[193+2626]) = 0.92
PARAMEDIC trial
30-day survival
30-day mortality
LUCAS-2
104
1548
Manual compressions
193
2626
Risk ratio = (104/[ ]) / (193/[ ]) = 0.92
Odds ratio = (104/1548) / (193/2626) = 0.91
LUCAS-2 vs. manual CPR in out of hospital cardiac arrest. No difference between groups.

6. Absolute vs. relative risk
In a multicenter trial conducted in the tertiary care setting, protocol-based resuscitation of patients in whom septic shock was diagnosed in the emergency department did not improve outcomes.

7. Absolute vs. relative risk
Mortality
Survival
EGDT 92
347
Standard care 86
370
AR = (92/[92+347]) - (86/[86+370]) = 0.02
RR = (92/[92+347]) / (86/[86+370]) = 1.11
Or, RR = AR / CER = 0.02 / 0.19 = 11.0%
NNT (H) = 1 / AR = 1 / 0.02 = 50
CER = control event rate.

8. Interpreting an observation
True observation
Confounding
Effect
Bias
Chance

9. Confounding
Exposure
Confounder
Outcome

10. Bias Selection bias Observer bias Bias Information bias Recall bias
Information bias = measurement error, misclassification

11. Randomisation…
Reduces confounding by minimising between group differences.
Equal distribution of confounding variables.
Reduces selection bias by random allocation.
Methods of randomisation.

12. Allocation concealment ≠ blinding
Concealment hides the allocation sequence.
Done automatically by computer generated allocation, but important if using sealed envelope/block randomiation etc.
Reduces selection bias.

13. Blinding reduces observer bias
Information bias and differential misclassification.

14. Who was blinded? Single: Either researcher or subject is blinded.
Double: Both researcher and subject blinded.
Triple: Researcher, patient and statistician blinded.
….....Open study group allocation.

15. Intention to treat analysis
ITT avoids artefact due to attrition by non-random cross over.
Does not require monitoring of compliance.
Important in pragmatic trials where the effect measure effectiveness not efficacy. i.e. treatment policy rather than treatment actually received.
Figures from: The Drs guide to critical appraisal Gosall and Gosall

16. Sample size Larger samples  more precise estimates
Greater precision = narrow confidence intervals
Two approaches:
Estimation of a measure with a specified precision
Based on hypothesis test or demonstrating a specified significance level
SIGN = Scottish intercollegiate guidelines network.

17. Error
Type I error
Probability of rejecting null hypothesis when it is true
Type II error
Probability of not rejecting the null hypothesis when it is false
SIGN = Scottish intercollegiate guidelines network.

18. Power
Probability that a hypothesis test with give a statistically significant result if an effect truly exists
1 – type II error rate
Chosen a priori
Usually 80%
SIGN = Scottish intercollegiate guidelines network.

19. Primary outcome
Composite outcome?

20. Abbott et al. Resuscitation 2015.
Example of a composite outcome measure. Abbott et al 2015
Abbott et al. Resuscitation 2015.

21. CONSORT

22. Trial registration

23. Standard outcome measures….
COMET = Core outcome measures in effectiveness trials

24. Levels of Evidence
Belsey What is EBM?

25. P-value vs. confidence interval
MRC CRASH Trial, Lancet 2005; 365:

26. Two trials
Small group work

27. Critical appraisal Population Intervention Comparator Outcome(s)
Study design
SIGN = Scottish intercollegiate guidelines network.