Presentation on theme: "Leeds Institute of Health Sciences Getting better evidence Stephen Morley."— Presentation transcript:
Leeds Institute of Health Sciences Getting better evidence Stephen Morley
Why bother with trials … ? From: Moore & McQuay Bandoliers little book of making sense of the evidence 2006 Knowledge Wisdom Sys Reviews & Meta-anal Evidence in clinical practice Distillation Integration Quality Experience Values Conditions Information Single RCTs
Trials GoodPoor Reviews Good IdealMay help Poor Can repeat Will mislead From: Moore & McQuay Bandoliers little book of making sense of the evidence
Issues Trial quality –Design, size matters –Quality and effect size Outcomes –Variety, validity and clinical relevance –Efficacy and effectiveness Treatment content and coherence –Is there a model? –Mediation?
Cumulative trials over years Hoffman et al 2007 Morle y et al 1999 Words of caution … What to count? Quality … Content … CBT on the label may not be CBT in the tin What is CBT in this context ?
Tools for assessing quality IdealMay help Can repeat Will mislea d Filter out poor quality trials by setting cut-offs Investigate influences of feature on conclusions – Meta-regression WHY? Trials +- + MA -
Quality scales: The Jadad Scale 1.Is the trial randomised (1 point) +1 point if method described and appropriate 2.Is the trial double blind (1 point) +1 point if method given and appropriate 3.Is there a description of withdrawals and drop outs (1 point ) Suggested cut-off = 3 Its simple Quick Captures major biases Can be reliable with basic training But Criterion 2 eliminates all complex interventions Doesnt capture important features of psychological trials
Quality scale for psychological trials Yates et al, Pain 2005: 117; Identify and recruit Delphi panel Panel generates and agrees Items: 3 rounds Expert panel writes QS Reliability and validity studies using novice and expert raters Data from 31 published trials Final QS
Quality scale for psychological treatment trials 3 2 parts Is there a good description of the sample in the trial? Sample characteristics0 1 Group equivalence parts Have adequate steps been taken to minimise biases? Randomisation0 1 2 Allocation Bias0 1 Measurement Bias0 1 Treatment expectations0 1 Reliability ICC absolute agreement Full scale > 0.9 Treatment quality > 0.9 Design quality = 0.85 Kappa for items range 0.0 to 0.74 Agreement coefficient for items >80%
Strengths and weakness of psychological trials Yates et al, Pain 2005: 117; Design Treat ment
Quality over time – the good news Morley, Eccleston & Williams, unpublished
Effect size and quality Yates et al, Pain 2005: 117; TotalQS β = -.35, p =.057 Treatment QS ns Design QS, β = -.4, p <.05
Size matters Data from Hoffman et al 2007
underlying scale x y z x dysfunctional or clinical sample functional or normal sample a b c CSC criteria Morley in McQuay et al Systematic reviews in pain research 2008, IASP press
Turning continuous outcomes into dichotomous ones Deteriorated from pre- treatment good functioning Reliably deteriorated Reliable improvement but not clinically significant Reliable and clinically significant improvement No reliable change Reliable improvement but not clinically significant Reliably deteriorated
Heterogeneity of outcomes in trials Data from Morley et al, Pain 1999: 80; 1-13 IMMPACT core outcomes 1.Pain 2.Physical Functioning (interference/disability) 3.Emotional functioning 4.Global improvement 5.Symptoms/adverse effects 6.CONSORT data Dworkin et al, Pain 2005: 113; 9-19
Stakeholders and outcomes: who wants what change? Health care provider Researcher The patient Employers What outcome do you want? Sleep Weakness Fatigue Emotional well-being Enjoyment of life Doing tasks IMMPACT, 2008 Pain:137;
How much change do you want? Mdn % change desired ES(d) %meeting RCI %meeting CSC Severity Impact Interfere Activity Thorne & Morley in preparation
How much change do you want? Thorne & Morley in preparation Interference Pain severity
The evidence cycle Efficacy studies Randomised Controlled Trials Evidence-based practice as policy Practitioners Practice-based evidence Effectiveness studies Routine Clinical Treatment Practitioners
Practice based evidence Morley, Williams & Hussain, Pain 2008; 137:
Benchmarking from RCT data From Minami et al J Consult Clin Psychol 2007;75:
Effectiveness + benchmark WLC group Tx Group RCT is: Williams et al. Pain 1996;66(1):13-22.
Quality controlling treatment Manuals – protocols? Training for therapist and teams? Supervision? Patient monitoring systems?
Is there a model: whats in the tin? Unpublished data from Morley et al, Pain 1999: CBT treatment components across trials
Whats the model? Generic –Principles of engagement and delivery Collaborative, information provision / education –Changing key cognitive appraisals through behavioural experimentation –Techniques: principled application or self service store? In PMP/CPM programmes embedded within pharmacotherapy, functional restoration, medical management –How coherent and integrated are they? Developing more specific models? A debate (JV)
Thanks to … Chris Eccleston Amanda Williams Henry McQuay Andrew Moore Wendy Callaghan Johan Vlaeyen Lance McCracken Shona Yates Sumerra Hussain Fiona Thorne
Seen in Leeds …. PAIN is just weaknessleavingtheBODY