1 Service Models: What should be adhered to? Meta-regression of Intensive case management studies Tom Burns University of Oxford

“When the facts change, I change my opinion. What, sir, do you do?” John Maynard Keynes, economist 2

Special problems researching community care interventions Complexity –(what is working and what just a passenger?) Pioneer effect –Good people make anything work better Sustainability –Is it worth the effort when the research is over? What are you comparing it to? ACT literature as an example 3

“Exact method based on inexact data can lead to the most remarkable mistakes.” General Psychopathology - Volume 1, Karl Jaspers, Page 24, John Hopkins University Press: 1913

5 Case Management vs Standard Care Hospital admissions

6 ACT vs Standard Care Hospital Admissions

7 Results and dilemmas ACT mandated by UK government 300 teams established nationally But: No European study has replicated the reduced hospitalisation Furore over UK700 study

Poor model fidelity Assertive Community Treatment (ACT) is distinct from, and superior to, other forms of case management (Max Marshall) The failure of UK studies of Assertive Outreach to demonstrate reduced hospital care reflect poor service implementation (‘Model Fidelity’) Max Marshall et al. We know this not to be the explanation –Fiander et al, (2003), BJPsych, 182,

9 Attempting to answer the question empirically: Going beyond definitions

10 How Meta-regression maximises data from the trials Skewed data included Data without SDs included where these can be imputed by statistical means Contacted trialists for missing information Used Independent Patient Data Split multi-centre trials

11 Inclusion criteria All randomised control trials (Cochrane Randomisation Category A or B) of intensive case management versus low intensity case management, standard care, or some combination of the two Intensive case management was defined as case management with a caseload of 20 or less Excluded if a majority of subjects were >65 yrs or not suffering from severe mental illness

12 Trials identified 42 included trials with 7817 participants 9 trials were multi-centre –8 disaggregated into a further 23 eligible trials with fidelity data for each Individual patient data obtained for 2084 participants in 5 trials –UK700 (n=708, 4 centres) –Rosenheck et al (n=873, 10 centres) –Drake et al (n=223, 7 centres) –Marshall et al (n=80, 1 centre) –McDonel et al (n=200, 2 centres)

13 Meta-regression used to test for impact on variation of: Date of study –Earlier studies more reduction? Size of study –Smaller studies bigger effect size as evidence of publication bias Baseline hospitalisation rates –Higher rates permits greater reduction Model fidelity –Higher model fidelity greater reduction

14 Meta-regression used to test for impact on variation of: Date of study –Earlier studies more reduction? No Size of study –Smaller studies bigger effect size as evidence of publication biasNo Baseline hospitalisation rates –Higher rates permits greater reductionYes Model fidelity –Higher model fidelity greater reductionYes

15 Impact of current bed usage

16 Copyright ©2007 BMJ Publishing Group Ltd. Burns, T. et al. BMJ 2007;335:336 Metaregression of Intensive Case management studies Baseline hospital use v mean days per month in hospital. Negative treatment effect indicates reduction relative to control

17 Copyright ©2007 BMJ Publishing Group Ltd. Burns, T. et al. BMJ 2007;335:336 Metaregression of Intensive Case management studies Control group mean v mean days per month in hospital. Negative treatment effect indicates reduction relative to control

18 Impact of model fidelity (ACT)

19 IFACT scale (McGrew et al 1995) Expert consensus: – 20 experts rated importance of 73 program features 14 item scale tested in 18 “ACT” programs Items specified three domains –membership, –structure & organisation –care practices

20 Meta-regression of Fidelity v Reduction in IP days

21 Separating the IFACT Domains

22 M-R of Team staffing v Reduction in IP days

23 M-R of Team organisation v Reduction in IP days

24 Conclusions High staffing levels in ACT are not associated with reducing hospitalisation ‘Organisational’ elements are associated with reducing hospitalisation These organisational elements appear to be present in standard CMHTs Can we be more precise about them? –Which are they?

25 A more positive message.. What does work

26 Testing for characteristics of home- based care using cluster analysis and regression

Core service components Figure 1: Associations between service components & hospitalisation: regression analysis Wright C, Catty J, Watt H, Burns T. A systematic review of home treatment services. Classification and sustainability. Soc Psychiatry Psychiatr Epidemiol 2004;39: Regularly visiting at home Responsible for health and social care High % of contacts at home Multidisciplinary teams Psychiatrist integrated in team Smaller caseloads

A Guide to current CMHT practices Generic CMHTs Assertive Outreach teams Crisis Resolution / Home Treatment teams Early onset teams Specialist and international perspectives

29 Treatment as usual studies The Dodo Bird society: –‘Dedicated to making Treatment as Usual studies history’ Burns T, Priebe S. Mental health care systems and their characteristics: a proposal. Acta Psychiatrica Scandinavica 1996 December;94(6): Burns T ‘End of the line for TAU studies’ BJPsych, 2009

30 Treatment as usual studies The danger of being restricted to your hypothesis in interpretation Killaspy follow up study (REACT) confirmed earlier REACT study that ACT has equivalent outcomes to CMHTs Conclusion ACT is ‘not superior’ Is that right? Conclusion is CMHTs superior!

Assessing Community Research Ensure that the comparator is relevant to you Ensure it is well described Check for sustainability (pioneer effect) Be sceptical about ‘ complex packages’ –Studies should try to isolate key ingredients If it looks too good to be true it probably is 31

32