Presentation is loading. Please wait.

Presentation is loading. Please wait.

Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood Health Care of Older People Nottingham University Hospitals.

Similar presentations

Presentation on theme: "Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood Health Care of Older People Nottingham University Hospitals."— Presentation transcript:

1 Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood Health Care of Older People Nottingham University Hospitals

2 What we may be witnessing is not just... the passing of a particular period of post-war history, but the end of history as such... That is, the end point of mankind's ideological evolution and the universalization of Western liberal democracy as the final form of human government.

3 The end of Medical History John R Hampton

4 Examples of complex services for older people Rehabilitation (stroke, community hospitals, intermediate care) Most psychiatry Falls services Continence services Care home and long term care interventions Education, training, guidelines, care pathways, service improvement

5 intervention … comprising a number of separate elements which seem essential to its proper functioning; components … may act independently and inter-dependently usually include: behaviours parameters of behaviours (frequency, timing) methods of organising and delivering those behaviours (types of practitioner, setting and location). Complex interventions MRC 2000

6 Briefly describe 4 trials Highlight design, implementation, analysis and interpretation issues Consider design compromises in a new trial of a complex intervention constrained by service demands Plan


8 Stroke Unit Evaluation 1991-3 Operational since 1982, 15 beds Restrictive inclusion criteria 315/1760 (18%) stroke patients included SU median 76 days, standard care 54 days Follow up: 275 (87%) at 3 mo; 241 (77%) at 12 mo At 12mo: Died 14% vs 19% Home 65% vs 62% Barthel 17 vs 16 Rivermead ADL 12 vs 9 NEADL 7 vs 4 GHQ 17 vs 20 psychological adjustment better at 3 & 6 months

9 Stroke Unit Evaluation issues Keep unit full (changed to 60:40 randomisation) Blinded assessment Loss to follow up (23% at 12 months) Contamination Power Multiple outcome measures … but what is a stroke unit? … why the difference in outcome?


11 SUE observation of process Non participant observations of 76 patients 3x8h shifts over 3 days, 10min intervals location rehab 6.4% vs 3%; bedside 61% vs 65%; social 5% vs 2% activity isolated disengagement 15% vs 24% positioning - poor 13% vs 30% contacts more time with nurses, therapists; less time with doctors, visitors rehab time (on ward) 46 min/d vs 23min/d


13 Established multidisciplinary service 428 referrals randomised (50:50 then 40:60) over 2y Group A: routine care including day hospital or OP (n=189) Group B: CST (n= 232). Median 18 sessions (range 0-115, IQR 10- 42). Mean 4.8 h PT; 3.8h OT; 2h SLT, 1.9h MH nurse; 0.5h Rehab Support Six months later, contacted by letter, requested consent to follow up, postal outcome questionnaire Consent procedure: avoid perception of missing out by control group reduce team loyalty bias logistically easier Offer of assistance or visit to complete questionnaire Community Stroke team

14 Community Stroke Service trial: issues

15 Patient CST vs standard Barthel/20 16 vs 16 NEADL/44 24 vs 26 GHQ/2413 vs 15 EQ5D/10052 vs 55 Satisfaction emotional support/4 3 vs 2 (CST better) overall/43 vs 2 (CST better) Carer GHQ/2413 vs 15 Carer strain/128 vs 10 (CST better) EQ5D73 vs 75 satisfaction knowledge/4 3 vs 2 (CST better) overall/4 3 vs 2 (CST better) Outcomes at 6 months

16 Low and differential follow up (47% vs 43%) No baseline data No record of routine services Time but not content of CST intervention recorded Was the service successful or not? If this didnt work what could …? Community Stroke Team: problems


18 A new EDRS was established in 1998. 9-month development phase. Staffing: 2 OTs, 2 PTs, 3 nurses, 1 SW, 7 rehab assistants, secretarial support. No doctors. Visits for assessment, monitoring, rehabilitation, or assistance and care. Up to four visits per day, up to 7 days per week, 8am to 10pm. Duration up to 4 weeks. EDRS: service structure

19 370 elderly medical, stroke and surgical patients Randomised between EDRS and conventional care Mean 22 visits per patient over 4 week Mean 7 days shorter hospital length of stay EDRS: randomised trial

20 EDRS: outcomes

21 Qualitative evaluations Exploratory, in depth Subject to rules Identify: what an intervention comprises in practice how an intervention is useful how it achieves its effect the conditions necessary for its effect Introduce rigour into otherwise potentially anecdotal evidence

22 Patients interviewed: before hospital discharge after 4 weeks after 3 months. Staff responsible interviewed 6–8 weeks later 64 interviews of 20 patients 15 interviews of 11 staff EDRS: Qualitative study

23 Before discharge Participants anxious about return home, but keen to do so Felt vulnerable, feared not being able to cope, or being a burden on families. After discharge Patients felt they had improved … but described pain, loneliness, fatigue, reduced mobility, problems climbing stairs or getting outdoors. Lack of confidence and anxiety about activities Qualitative study

24 EDRS delivered : skilled assessment, negotiated treatment goals meaningful to the patient, and demonstrated team working. Interventions : functional rehabilitation, teaching skills, information giving, advice, overcoming emotional barriers, provision of aids and appliances and provision of personal and domestic care. Emotional support : improved confidence and morale, and led to greater task performance and psychological well-being. Qualitative study


26 1 intervention ward, 2 standard care wards Multi-component delirium prevention regimen hydration vision and hearing orientation mobilisation non-pharmacological sleep promotion. RCT pilot failed - 120% bed occupancy Prospective individual matching design Delirium prevention trial

27 Prospectively assessed: 2424 patients admitted to 3 wards Research staff not involved in care, unaware of nature of study, or patient group assignment. Over 70, at intermediate or high risk of delirium Recruited all patients on intervention ward Matched these with standard care patients 5 years sex baseline risk of delirium U&E, visual and cognitive impairment, APACHE-II. Prospective individual matching design

28 Assessed 2424 patients Excluded 1265 severe dementia 154 aphasia 38 no English 92 coma or terminal illness 69 LOS<48h 219 other incl. no interviewer or patient unavailable 355 Declined 250 No match 67 (24 intervention, 43 standard) Recruited 852 (426 intervention, 426 standard) Exclusions

29 Results

30 Able to do research interviews on all at admission Demonstrated power equivalence of RCT Recruitment Demonstrated good matching at baseline on 32 variables Contamination Bias and blinding Issues


32 MRC 2000 framework

33 MRC 2008

34 NIHR Medical and mental health unit controlled clinical trial

35 Medical and mental health unit: development Initial 250 patient register/cohort study. Multi-professional operational group Additional staff: RMNs and MH specialist OT/PT Programme of staff training Advice from MH service Environmental changes New approach to families Develop policies and new working practices

36 Medical and mental health unit sufficiently different from standard care to have a reasonable chance of demonstrating different outcomes in a trial

37 Medical and mental health unit: controlled clinical trial Modelled on stroke unit trials Compared with standard care Starting May 2010 (pilot); July 2010 (main trial) up to 2 year recruiting, 3 months follow up Target n=240 per group Effectiveness, economic and qualitative studies: not just does it work, but how does it work? is it value for money? Service funding ceases April 2013

38 Medical and mental health unit: controlled trial Outcomes at 3 months: number of days spent at home: length of stay, readmissions, deaths, care home placements number of days in care home for those starting there scales: DemQoL, NPI (behaviour), Barthel, EQ5D, London Handicap Carer strain and psychological wellbeing resource use and costs non-participant observer study

39 Medical and mental health unit: recruitment and randomisation Constraints No waiting on AMU (for assessment/recruitment/consent/MMHU bed) No empty beds on MMHU Keep MMHU full with suitable patients Minimise ward moves for confused patients Likely to lose trial beds to non-randomised patients

40 Medical and mental health unit: Recruitment and randomisation Randomisation prior to consent, recruit to follow up later. Confused, over 65 Web based stratification (care home or not) Transferred to MMHU or standard ward Once on ward recruited to trial (consecutively or randomly) Other issues Need to persuade AMU to do something of no benefit to them Majority lack capacity to consent: effort and delay 10% lack a consultee 40% of cohort declined

41 CLRN/MHRN to the rescue Initial plan was limited by availability of researchers to recruit and follow up Current plan to recruit all randomised patients who agree to take part Option to collect primary outcome on those declining full study

42 Summary and lessons for the triallist The RCT can be adapted to evaluate complex interventions Logistics can be tricky and compromises needed Interpretation is more difficult multiple outcome measures what is the outcome? does the outcome match the intervention? condition specific vs generic outcomes qualitative evaluation Look at the content of the intervention what is the intervention? how different it is from the comparator

43 Complex interventions Thank you. Watch this space!

Download ppt "Evaluating complex interventions (or the tribulations of trials) Prof. Rowan Harwood Health Care of Older People Nottingham University Hospitals."

Similar presentations

Ads by Google