Using CORE-34 and other measures – an example from NHS practice Dr Annie Nehmad and Dr Kim Dent-Brown UKCP Research Conference 18 July 2015

Acknowledgement Mansur Quraishi, qualified CAT Therapist, and Honorary Therapist within the CAT service in The City & Hackney, performed the analysis of outcome measures on Excel in August 2011.

Background How can routinely collected clinical data, often neglected, be made more useful? What simple data analysis and presentation techniques could help practitioners, and perhaps influence commissioners?

4 Cognitive Analytic Therapy (CAT) Developed by Anthony Ryle from the late 1970’s within the NHS Brief: most patients are offered a contract of 16 sessions. People with a diagnosis of Borderline Personality Disorder are often offered 24 sessions. Integrative: Cognitive, Object Relations, and other ideas brought together into an integrated synthesis Collaborative and Accessible: patients do not need to be “psychologically minded” Effective for most types of problems and most types of people, including those who present frequent challenges to the therapeutic alliance (eg. Borderline Personality Disorder, Narcissistic issues) Supervision is essential (especially for trainees, and for challenging cases) and must be adequate in quality and quantity More information, including Evidence Base:

The CAT Service in The City & Hackney The City & Hackney Psychotherapy Department (a Secondary Care service for complex cases) is part of East London NHS Foundation Trust. Hackney is an inner London borough with high indices of deprivation. Assessment was usually within three weeks of referral; waiting list for treatment was 6 to 12 months after assessment. Many of the patients had a diagnosis of Borderline Personality Disorder, or Borderline features. One-third had been considered unsuitable for CBT or Psychodynamic, when assessed within those services.

The therapists About half the cases were treated by qualified CAT Therapists on the payroll. The other half were treated by Honorary Therapists. Some of the Honoraries were experienced in other approaches, and were learning CAT. A few were experienced CAT therapists. Others were First Year Clinical Psychology trainees – often treating their first-ever psychotherapy patient. These trainees received half an hour’s supervision for every hour of therapy they provided, within a group setting (eg. two hours of supervision for four cases)

A pilot audit of 16 CAT cases by Trainees, The City & Hackney Psychotherapy Department, 2010

Method Audit of CAT Service, City & Hackney, on 29/8/11 All 142 patients referred between 1/1/08 and 31/12/10 (36 month period), and accepted onto CAT waiting list. “Snapshot” on 29 August 2011, using outcome measure scores entered onto an Excel file (designed primarily for admin purposes) Measures taken at four points: Assessment, Start of CAT (6-12 months later), End of CAT (usually 16 or 24 sessions), Follow up (3-12 months after the last session).

Outcome Measures PHQ-9 (Depression) GAD-7 (Anxiety) CORE-OM (34 item, psychological distress) IIP-32 (Interpersonal problems) WSA (Work and social functioning) PSQ (Dissociation/Fragmentation)

Sample Excel SurnameFirst Name Assessment CORE Start COREEnd CORE Follow Up CORE RedRose WhiteWilliam BlueBertram GreenGeorgina YellowYvonne Mean average

Analysis For the sake of brevity, we will report on the findings from the CORE-OM. But we have analysed all six measures and the pattern is similar for all of them. Assessmentn=127 Start of therapy n=124 End of therapy n=80 Follow up n=50 Note: The numbers get smaller because at the time of the “snapshot” many were still on waiting list, or had not completed therapy

CORE-OM Reported on 0-40 scale Higher score = more distress Anything above 10 is clinically significant

Mean CORE score profile Mean = 19.7 Confidence Interval means we can be 95% certain the true mean lies between 18.6 and 20.9

Mean CORE score profile Mean appears to drop slightly while on waiting list. But the confidence intervals overlap – ie we cannot be certain there is any significant change.

Mean CORE score profile The drop in mean score (18.2 to 12.8) from start to end of therapy is statistically significant. We know because there is no overlap in CI of mean.

Mean CORE score profile Means do not change significantly from end of therapy to follow up (because CIs overlap)

Mean CORE score profile No change on waiting list Improvement during therapy Improvement maintained to follow up

Similarity with other measures PHQ-9 GAD-7 IIP WSA

Another way of tracking change Looking at mean scores tells you about the average change across a whole group It ignores the fact that even if the whole group doesn’t change significantly, some individuals do. Moreover, even if the group improves as a whole, some individuals don’t improve (and some even deteriorate) A different kind of chart can show greater detail – the Jacobson plot

Line of no change – end score same as start

Measurement error – scores within this band have not changed significantly

No reliable change

A real world example Jacobson plot for the City and Hackney CAT audit N=76 (there were 76 people for whom we had both start of therapy and end of therapy CORE data)

8 reliable deterioration (11%) 35 no reliable change (46%) 14 reliable improvement (18%) 19 recovery (25%)

Tracking individuals We have 36 individuals for whom we have CORE data at all four time points We can track these individuals’ progress over time (though looking at all 36 is confusing at first!)

Tracking individuals

It is easier to look at individuals when there are fewer lines on the chart Six cases are shown on the next chart (NOT typical – chosen because they are worth looking at as exceptions) How can we understand their unique therapy journeys? The ‘average’ expected trajectory is shown by the dotted line

Tracking individuals