Presentation on theme: "1 Levels of Evidence Why? What? How?. 2 Time-poor clinician suffering from Information Overload."— Presentation transcript:
1 Levels of Evidence Why? What? How?
2 Time-poor clinician suffering from Information Overload
3 Evidence-Based Medicine EBM is...”the conscientious, explicit and judicious use of current best evidence in making decisions about the care of an individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research” (Sackett, D. BMJ 1996;312:71-72).
5 Steps in EBM Defining the question or problem Searching for the evidence Critically appraising the literature Applying the results Auditing the outcome
6 The Evidence Pyramid is a guideline to the hierarchy of study design
7 Type of question = type of study design Type of questionSuggested study TherapyRCT > prospective cohort DiagnosisProspective, blind comparison to a gold standard Etiology/HarmRCT > cohort > case control > case series PrognosisCohort study > case control > Case series PreventionRCT > cohort study >case control > case series CostEconomic analysis
8 NHMRC Levels of Evidence LevelInterventionDiagnostic accuracyPrognosisAetiologyScreening Intervention I 4 A systematic review of level II studies IIA randomised controlled trial A study of test accuracy with: an independent, blinded comparison with a valid reference standard, 5 among consecutive persons with a defined clinical presentation 6 A prospective cohort study A randomised controlled trial III-1 A pseudorandomised controlled trial (i.e. alternate allocation or some other method) A study of test accuracy with: an independent, blinded comparison with a valid reference standard, 5 among non-consecutive persons with a defined clinical presentation 6 All or none A pseudorandomised controlled trial (i.e. alternate allocation or some other method) III-2 A comparative study with concurrent controls: ▪ Non- randomised, experimental trial 9 ▪ Cohort study ▪ Case- control study ▪ Interrupted time series with a control group A comparison with reference standard that does not meet the criteria required for Level II and III-1 evidence Analysis of prognostic factors amongst persons in a single arm of a randomised controlled trial A retrospective cohort study A comparative study with concurrent controls: ▪ Non-randomised, experimental trial ▪ Cohort study ▪ Case-control study III-3 A comparative study without concurrent controls: ▪ Historical control study ▪ Two or more single arm study 10 ▪ Interrupted time series without a parallel control group Diagnostic case-control study 6 A retrospective cohort study A case-control study A comparative study without concurrent controls: ▪ Historical control study ▪ Two or more single arm study IVCase series with either post- test or pre-test/post-test outcomes Study of diagnostic yield (no reference standard) Case series, or cohort study of persons at different stages of disease A cross-sectional study or case series Case series
9 NHMRC Assessment of study quality – Grades of Recommendations 1. The evidence base, in terms of the number of studies, level of evidence and quality of studies (risk of bias). 2. The consistency of the study results. 3. The potential clinical impact of the proposed recommendation. 4. The generalisability of the body of evidence to the target population for the guideline. 5. The applicability of the body of evidence to the Australian healthcare context.
10 Checklist for appraising the quality of studies of interventions (Cochrane handbook) 1. Method of treatment assignment a. Correct, blinded randomisation method described OR randomised, double-blind method stated AND group similarity documented b. Blinding and randomisation stated but method not described OR suspect technique (eg allocation by drawing from an envelope) c. Randomisation claimed but not described and investigator not blinded d. Randomisation not mentioned 2. Control of selection bias after treatment assignment a. Intention to treat analysis AND full follow-up b. Intention to treat analysis AND <15% loss to follow-up c. Analysis by treatment received only OR no mention of withdrawals d. Analysis by treatment received AND no mention of withdrawals OR more than 15% withdrawals/loss-to-follow-up/post-randomisation exclusions 3. Blinding a. Blinding of outcome assessor AND patient and care giver b. Blinding of outcome assessor OR patient and care giver c. Blinding not done 4. Outcome assessment (if blinding was not possible) a. All patients had standardised assessment b. No standardised assessment OR not mentioned Source: National Health and Medical Research Council (NHMRC). How to review the evidence: systematic identification and review of the scientific literature. Canberra: NHMRC, 1999: p.45
11 NHMRC Grades of Recommendations Grade of recommendationDescription ABody of evidence can be trusted to guide practice BBody of evidence can be trusted to guide practice in most situations CBody of evidence provides some support for recommendation(s) but care should be taken in its application DBody of evidence is weak and recommendation must be applied with caution
12 Clinical Guidelines for Stroke Management National Stroke Foundation
13 Brain Trauma, F., S. American Association of Neurological, et al. (2007). "Guidelines for the management of severe traumatic brain injury." Journal of Neurotrauma 24 Suppl 1. Class of evidenceStudy design Quality Criteria IGood quality randomized controlled trial (RCT) Adequate random assignment method Allocation concealment Groups similar at baseline Outcome assessors blinded Adequate sample size Intention-to-treat analysis Follow-up rate 85% No differential loss to follow-up Maintenance of comparable groups IIModerate quality RCTViolation of one or more of the criteria for a good quality RCT IIGood quality cohort Blind or independent assessment in a prospective study, or use cohort of reliable data in a retrospective study Non-biased selection Follow-up rate 85% Adequate sample size Statistical analysis of potential confounders IIGood quality case-controlAccurate ascertainment of cases Nonbiased selection of cases/controls with exclusion criteria applied equally to both Adequate response rate Appropriate attention to potential confounding variables IIIPoor quality RCTMajor violations of the criteria for a good or moderate quality RCT IIIModerate or poor quality cohortViolation of one or more criteria for a good quality cohort IIIModerate or poor quality case-controlViolation of one or more criteria for a good quality case-control IIICase series, databases or registries
14 Table 1. Applying Classification of Recommendations and Level of Evidence Morgenstern, L. B., J. C. Hemphill, 3rd, et al. (2010). "Guidelines for the management of spontaneous intracerebral haemorrhage: a guideline for healthcare professionals from the American Heart Association/American Stroke Association." Stroke 41(9):
15 Is Evidence-Based Surgery an oxymoron? What if there is no level I evidence? Surgical RCTs have well-recognized disadvantages: high costs, administrative complexity, prolonged time to completion, recruitment difficulty, blinding, randomization technique standardization, poor generalizability or external validity, patient compliance, underpowered studies, crossovers and drop outs, multiple surgical options, technological advancement, patient complexity, variability and preference and selection bias....
16 It is the well-defined research question that dictates the study design, not that every study should be a RCT because it’s the gold standard” “The proper use of evidence-based information is not the strict adherence to only RCTs, but more accurately, the informed and effective use of all types of evidence … Large, prospective cohort studies in a surgical setting are often thought to be on a par with RCTs and provide superior generalizability” Fisher, C. G. and K. B. Wood (2007). "Introduction to and techniques of evidence-based medicine." Spine 32(19 Suppl): S66-72.
17 ISAT (Lancet, 2002; Lancet, 2005) 9559 eligible patients, 2143 randomised –43 Participating centres; enrolled 1-44% of eligible patients Equipoise did not exist in over 75% –3615 underwent surgery –2737 underwent endovascular treatment –1064 unknown treatment Differences between groups –Cross-overs, time to treatment, Small but significant difference in time between randomisation and first procedure –Coiling 1.1 daysSurgery 1.7 days Cross-overs –Coiling surgery 9 patients –Surgery coiling 38 patients Different experience of INR and surgeons Unknown differences between centres
18 Know which levels and grades of recommendations are being used and quote/reference them Stay up to date with developments or changes to levels and grades Use NHMRC levels and grades where possible Look for other levels of evidence when RCTs or level 1 studies are not possible Make evidence-based decisions Become a lifelong learner of EBM sid= sid= Using the evidence....