1. Leeds Institute of Health Sciences Getting better evidence Stephen Morley

2. Why bother with trials … ? From: Moore & McQuay Bandoliers little book of making sense of the evidence 2006 Knowledge Wisdom Sys Reviews & Meta-anal Evidence in clinical practice Distillation Integration Quality Experience Values Conditions Information Single RCTs

3. Trials GoodPoor Reviews Good IdealMay help Poor Can repeat Will mislead From: Moore & McQuay Bandoliers little book of making sense of the evidence

4. Issues Trial quality –Design, size matters –Quality and effect size Outcomes –Variety, validity and clinical relevance –Efficacy and effectiveness Treatment content and coherence –Is there a model? –Mediation?

5. Trial quality

6. Cumulative trials over years Hoffman et al 2007 Morle y et al 1999 Words of caution … What to count? Quality … Content … CBT on the label may not be CBT in the tin What is CBT in this context ?

7. Tools for assessing quality IdealMay help Can repeat Will mislea d Filter out poor quality trials by setting cut-offs Investigate influences of feature on conclusions – Meta-regression WHY? Trials +- + MA -

8. Quality scales: The Jadad Scale 1.Is the trial randomised (1 point) +1 point if method described and appropriate 2.Is the trial double blind (1 point) +1 point if method given and appropriate 3.Is there a description of withdrawals and drop outs (1 point ) Suggested cut-off = 3 Its simple Quick Captures major biases Can be reliable with basic training But Criterion 2 eliminates all complex interventions Doesnt capture important features of psychological trials

9. Quality scale for psychological trials Yates et al, Pain 2005: 117; 314-325 Identify and recruit Delphi panel Panel generates and agrees Items: 3 rounds Expert panel writes QS Reliability and validity studies using novice and expert raters Data from 31 published trials Final QS

10. Quality scale for psychological treatment trials 3 2 parts Is there a good description of the sample in the trial? Sample characteristics0 1 Group equivalence0 1 4 4 parts Have adequate steps been taken to minimise biases? Randomisation0 1 2 Allocation Bias0 1 Measurement Bias0 1 Treatment expectations0 1 Reliability ICC absolute agreement Full scale > 0.9 Treatment quality > 0.9 Design quality = 0.85 Kappa for items range 0.0 to 0.74 Agreement coefficient for items >80%

11. Strengths and weakness of psychological trials Yates et al, Pain 2005: 117; 314-325 Design Treat ment

12. Quality over time – the good news Morley, Eccleston & Williams, unpublished

13. Effect size and quality Yates et al, Pain 2005: 117; 314-325 TotalQS β = -.35, p =.057 Treatment QS ns Design QS, β = -.4, p <.05

14. Size matters Data from Hoffman et al 2007

15. Outcomes

16. underlying scale x y z x dysfunctional or clinical sample functional or normal sample a b c CSC criteria Morley in McQuay et al Systematic reviews in pain research 2008, IASP press

17. Turning continuous outcomes into dichotomous ones Deteriorated from pre- treatment good functioning Reliably deteriorated Reliable improvement but not clinically significant Reliable and clinically significant improvement No reliable change Reliable improvement but not clinically significant Reliably deteriorated

18. Heterogeneity of outcomes in trials Data from Morley et al, Pain 1999: 80; 1-13 IMMPACT core outcomes 1.Pain 2.Physical Functioning (interference/disability) 3.Emotional functioning 4.Global improvement 5.Symptoms/adverse effects 6.CONSORT data Dworkin et al, Pain 2005: 113; 9-19

19. Stakeholders and outcomes: who wants what change? Health care provider Researcher The patient Employers What outcome do you want? Sleep Weakness Fatigue Emotional well-being Enjoyment of life Doing tasks IMMPACT, 2008 Pain:137; 276-285

20. How much change do you want? Mdn % change desired ES(d) %meeting RCI %meeting CSC Severity601.4561.220.9 Impact751.7073.257.7 Interfere661.8275.063.2 Activity441.3836.816.2 Thorne & Morley in preparation

21. How much change do you want? Thorne & Morley in preparation Interference Pain severity

22. The evidence cycle Efficacy studies Randomised Controlled Trials Evidence-based practice as policy Practitioners Practice-based evidence Effectiveness studies Routine Clinical Treatment Practitioners

23. Practice based evidence Morley, Williams & Hussain, Pain 2008; 137: 670-680

24. Outcome categories - efficacy 858858 354354 20620 Crude NNT values Morley, Williams & Hussain, Pain 2008; 137: 670-680

25. Benchmarking from RCT data From Minami et al J Consult Clin Psychol 2007;75: 232-43

26. Effectiveness + benchmark WLC group Tx Group RCT is: Williams et al. Pain 1996;66(1):13-22.

27. Treatment

28. Quality controlling treatment Manuals – protocols? Training for therapist and teams? Supervision? Patient monitoring systems?

29. Is there a model: whats in the tin? Unpublished data from Morley et al, Pain 1999: 80 1-13 CBT treatment components across trials

30. Whats the model? Generic –Principles of engagement and delivery Collaborative, information provision / education –Changing key cognitive appraisals through behavioural experimentation –Techniques: principled application or self service store? In PMP/CPM programmes embedded within pharmacotherapy, functional restoration, medical management –How coherent and integrated are they? Developing more specific models? A debate (JV)

31. Thanks to … Chris Eccleston Amanda Williams Henry McQuay Andrew Moore Wendy Callaghan Johan Vlaeyen Lance McCracken Shona Yates Sumerra Hussain Fiona Thorne

32. Seen in Leeds …. PAIN is just weaknessleavingtheBODY