1. The Philosophy and Practice of Feedback: Using CORE OM as a Clinical Tool Dr James Macdonald Director of Clinical Training CORE IMS

2. The long and winding road Using Session by Session Measures

3. How have you been this week? Um. I think a bit better. I haven’t been sleeping very well Can you tell me a bit more about your sleep? Well, I’ve had some horrible dreams recently

4. Creativity and Measurement Creativity has been defined as the development of original ideas that have value (Sir Ken Robinson, 2011) In small groups (e.g. 4/5 people) discuss creative ways in which you might use formal measurement (e.g. outcome measures) to benefit your clients (10 minutes)

5. Measurement and Creativity in Medicine – an example In the 1950s one in thirty newborns died at birth. Dr Virginia Apgar [d. 1974] was an anaesthetist who was appalled by the care that many newborns received. Babies who were born malformed or too small or just blue and not breathing well were listed as stillborn, placed out of sight and left to die. She developed a scale, the Apgar score, as it became universally known, which allowed nurses to rate the condition of babies at birth on a scale from zero to ten. An infant got two points if it was pink all over, two for crying, two for taking good vigorous breaths, two for moving all four limbs, and two if its heart rate was over a hundred. Ten points meant a child born in perfect condition. Four points or less meant a blue, limp baby. Published in 1953 to revolutionary effect, the score turned an intangible and impressionistic clinical concept – the condition of new babies – into numbers that people could collect and compare. Using it required more careful observation and documentation of the true condition of every baby. Moreover, even if only because doctors are competitive, it drove them to want to produce better scores – and therefore better outcomes – for the newborns they delivered. Around the world, virtually every child born in a hospital came to have an Apgar score recorded at one minute after birth and at five minutes after birth. It quickly became clear that a baby with a terrible Apgar score at one minute could often be resuscitated – with measures like oxygen and warming – to an excellent score in five minutes. Neonatal intensive care units sprang into existence. The score also began to alter how childbirth itself was managed. Spinal and then epidural anesthesia were found to produce babies with better scores than general anesthesia. Prenatal ultrasound came into use to detect problems for deliveries in advance. Fetal heart monitors became standard. Over the years, hundreds of adjustments and innovations were made, resulting in what’s sometimes called “the obstetrics package”. And that package has produced dramatic results. In the United States today, a full-term baby dies in just one childbirth out of five hundred, and a mother dies in less than one in ten thousand. If the statistics of the 1930s had persisted, 27,000 mothers would have died last year (instead of fewer than five hundred) – and 160,000 newborns (instead of one-eighth that number). From: Atul Gwande (2008). Better: A Surgeon’s Notes on Performance. London: Profile Books. P169ff, mainly p 186-188. (One page).

6. Discussion What do questionnaires do well? What do people do well? What thoughts did you have in your groups about the creative use of measures in therapy?

7. Changing paradigm…? Client response Client individuality Client agency Therapy relationship Therapist responsiveness ‘Meta-communication’ Models Techniques Therapist Objectivity

8. Process-Outcome Research Orlinsky, Rønnestad & Willutzki (2004); (see also Bohart & Wade, 2013; Norcross et al, 2011) major review of process-outcome research: ‘quality of patients’ participation [emerges] as the most important determinant of outcome (p324)’ [more than therapist attitudes, behaviours or techniques] – Characterised as ‘active shoppers’ vs ‘passive recipients’ – Collaborative clients do better – Resistance to therapeutic processes predict poor outcomes (openness predicts good) – Zuroff et al (2007) autonomous motivation

9. Bohart, A. & Wade, A. (2013). The Client in Psychotherapy. In M. Lambert [ed.] Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change (Sixth Edition), p 219-257. Hoboken, NJ: John Wiley & Sons. ‘Clients are not passive recipients of treatment like patients in surgery. Rather, they actively intersect with what therapists have to offer (even if that ‘activity’ sometimes consists of adopting a passive or resistant stance). How they learn involves their degree of involvement, their resonance with therapists and methods, how much effort they put in, their own creativity, and how they interpret and implement the input they receive.’ (p220)

10. A Safety Net

11. Therapist Blindsidedness Hannan et al (2005); Chapman et al (2012): Prediction of deterioration Hatfield et al (2010): Noticing deterioration Walfish et al (2009); Dew & Reimer (2003): Self- assessment bias

12. The Cognitive Science of Mistakes Base rate fallacy Confirmation bias Attentional bias Availability heuristic Anchoring bias What You See Is All There Is

13. "all the women are strong, all the men are good looking, and all the children are above average"

14. My Outcomes Are Great! Trust me – I’m a psychotherapist Should clients and services rely on clinician self-assessment?

15. Why is assessment of our own performance so difficult in psychotherapy ? Highly complex ‘low validity’ Environments (Kahnemann, 2011; Kahneman & Klein, 2009) Self-assessment bias Client deference and opaque mental states (e.g. Regan & Hill, 1992; Fonagy et al 2003)

16. Therapist Blindsidedness: Small Groups Discussion Discuss the findings about the limitations of therapists’ knowledge about how therapies progress. Reflecting back to your creative ideas for the use of measures discuss how you think questionnaires could be used to mirror information to therapists that may be lacking due to the blindspots we have discussed. 10 minutes

17. Using CORE Net to Improve Outcomes Introduction to the system Introducing clients to feedback informed therapy Problem solving with off track clients – Ruptures – Motivation – Life events

18. CORE System Tools for Feedback-led Therapy Copyright: CORE IMS Ltd

19. Highlighting more problematic items…

20. A tracking graph…

21. ‘Flags’

22. Introducing feedback… What do you think might be the key elements in introducing session by session monitoring to a couple at the beginning of therapy? Themes: What is involved….Why you are suggesting it …. How do they feel about it?

23. Introducing Feedback The message to your client: ‘ Your input is crucial; your participation matters. We invite you to be a partner in your care. We respect what you have to say, so much so that we will modify the treatment to see that you get what you want.’ (Miller, Hubble, Duncan & Wampold, 2010)

24. In the beginning: Introducing session by session monitoring Role Play: In pairs role play introducing session by session monitoring to a new client. (10 mins)

25. Principles for Working with Off-Track Cases Systematic Use of Questionnaires to Spot Problems Awareness of Solutions to Common Problems Problem Solving Dialogue

26. Using CORE Net with Off-Track Cases

27. Working with Off-Track Cases… Key Processes Linked to Outcomes? The Therapy Relationship (Bond, Goal & Task) Client Motivation/ Understanding of Therapy Adverse Life Events…. Lack of Social Support Perfectionism/ Harsh self- criticism Something missed in the formulation Eg. Substance misuse. Mike Lambert: Clinical Support Tools

28. Using a process measure ARM 5 HAT (Helpful Aspects of Therapy) Form

29. Scenario 1: No Change – Problem in the Therapeutic Relationship

30. Score on the ARM 5 (relationship measure) ANSWERQUESTION 7. Strongly agree1. My therapist is supportive 6. Moderately agree2. My therapist and I agree about how to work together 3. Slightly agree3. My therapist and I have difficulty working as a partnership 7. Strongly agree4. I have confidence in my therapist and his/her techniques 4. Neutral5. My therapist is confident in him/herself and his/her techniques ARM score 5.4 (Max score is 7)

31. Therapeutic orientation to ruptures Genuine and gentle Enquiry into how the client has experienced the relationship with you in the session (leading to exploration of the client’s ‘construal’ of the therapeutic relationship) Empathy with client’s experience in the session (‘no wonder you felt angry if you thought I was judging you like that’) ‘Meta-communication’ (e.g. ‘it seems to me at the moment as though we’re in a dance where I’m the cat and you’re the mouse and each time I comment on how I am understanding your experience, you say ‘no, that’s not it’ and move away, as if you fear being ‘caught’. Is that anything like how you are experiencing being with me at the moment?’ ‘Wrecking ball’) Taking responsibility for one’s own part in any rupture (e.g. ‘yes, thinking back on it, I think I may have been pursuing you and trying to pin you down somehow. I think I might have felt some frustration that we could not reach a shared understanding of what was happening for you, but I understand now how I came across in a way which was frightening and made you pull back more. I’m really sorry to have put you more in that position. Thank you for discussing this with me’)

32. Summary of common rupture repair interventions (Safran et al, 2011) 1.Repeating the therapeutic rationale 2.Changing Task or Goals 3.Clarifying misunderstandings 4.Exploring relational themes associated with the rupture 5.Linking the rupture to common patterns in client’s life 6.New Relational Experience

33. Role Play Take it in turns to role play a recent client of yours whose failure to improve appears to be connected to problems in the therapeutic relationship. Imagine like Lil he/she has not improved on CORE 10 and that his/her ARM score shows that he/she is having some difficulties collaborating or agreeing with you in therapy As therapist, aim first to empathically understand the client’s experience of the rupture with you. You can then take the conversation in any of the directions suggested by Safran et al for resolving ruptures. As observer, note strategies used by the therapist and impact on the client Discussion: what did you learn from this? What was difficult? What seemed to work?

34. Scenario 2: No Change – Problem with client motivation

35. Stages of Change Precontemplation – 4 “Rs” Reluctance; Rebellion; Resignation; Rationalisation Contemplation – Ambivalence Preparation Action Maintenance (Prochaska & Norcross)

36. Assessing motivational problems “Honestly, I really don’t understand what I can get from therapy” “I am not really sure what to work on in therapy” “I had thoughts about quitting therapy; it’s just not for me” “Although I am currently unhappy with life, there is nothing I can do about it” “I am in therapy because someone is requiring it of me” From Lambert et al ‘Clinical Support Tool’

37. Motivational Interviewing Express Empathy – Understanding client’s frame of reference – Accepting ambivalence about change as normal – Therapist reflection ‘a guess as to what client means’ Develop Discrepancy – Make explicit values and discrepancy with current behaviour Roll with Resistance – Normalise/accept resistance – Explore and deepen awareness of pros and cons from client perspective Support Self-Efficacy Miller & Rollnick 2002

38. Role Play Take it in turns to role play a recent client of yours whose failure to improve appears to be connected to ambivalence about working in psychotherapy. Imagine like Lil he/she has not improved on CORE 10. Take it in turns to role play as therapist, using open-ended questions and empathic summaries/reflections, etc, to help the client articulate and deepen their understanding of their ambivalence and discrepancies between their values and their current behaviour Observer: note strategies used by therapist and impact on client Discussion: what did you learn from this? What was difficult? What seemed to work?

39. Other scenarios associated with lack of change 1: Social Support Client Lacks Social Support (Lambert CST) – Assess client’s social network – Refer for group therapy – Role play to develop social skills – Bring significant others to session – Specific anxiety reduction practices for social anxiety – Encourage participation in positive social activities

40. Other scenarios associated with a lack of change 2: Life events; Diagnosis 1.Life Events Divorce, illness, stress at work, housing problems, relationship conflict Interact with other client variables Explore nature of stressor and client coping style 2.Diagnosis e.g. substance misuse, unrecognised bipolar disorder Consider medication as an adjunct to psychotherapy

41. Scenario 5: Client deterioration

42. ‘Learning to Fail Successfully’ (Duncan, 2010) 1) Discuss possible reasons (follow procedures above) 2) Review client’s experience of therapeutic relationship (ARM) and experience of the sessions 3) Supervision 4) Consider changing approach 5) Consider changing therapist 6) Duncan (2010) ‘Checkpoint Conversations’ (3 rd or 4 th session) and ‘Last Chance Discussions’ (6 th or 7 th session).

43. Experience of the Workshop and Next Steps What do you think you will be able to take from this afternoon’s workshop? What was most surprising to you? What still seems unclear or difficult to you? How do you see yourselves taking the next steps with CORE Net?

44. Further reading… [james@coreims.co.uk] http://www.headingtonpsychotherapy.co.uk/downloads/FeedbackasPsychoanalyticTechniqueMacdonald2013.pdf

46. Basics of using feedback Reinforcing ‘on track’ work Exploring ‘lack of change’. Considering – Motivation – Therapeutic relationship – Unacknowledged substance abuse – Lack of social support – Stressful life events Acting in the case of deterioration – ‘Failing successfully’

47. How do you know how you do?

48. How easy is it to know how you are doing as a therapist? What gets in the way of knowing how you are doing?

49. In the Fog of Complexity, Mirages Abound Expert prediction and intuition are notoriously problematic in highly complex situations (Kahnemann, 2011; Kahnemann & Klein, 2009) In such complex environments, where straightforward feedback is lacking, assessment of one’s own performance may be more subject to self-assessment biases

51. Paul Meehl and the Fog of Therapy Clinical psychologist, Academic and Psychoanalyst Grove & Meehl (1996) meta-analysis of studies comparing actuarial versus clinical predictions. ‘Of the 136 studies, 64 favored the actuary …, 64 showed approximately equivalent accuracy, and 8 favoured the clinicians’ (p298).

52. Ægisdóttir et al. (2006) The Meta-Analysis of Clinical Judgement Project: Fifty Six Years of Accumulated Research on Clinical vs Statistical Prediction Clinical decisions are accurate 47% of the time, whereas statistical decisions are accurate 53% of the time p 359. 13% increase in accuracy with statistical predictions More information to clinicians resulted in less accuracy

53. ‘High Validity environments’ Expert intuition can be relied on in ‘high validity’ environments – in which there are environmental regularities which provide cues to the situation, – and where the expert has had adequate opportunity to learn these regularities. – Examples of high validity environments include: chess, fire fighting and aspects of nursing

54. ‘Low validity’ environments Expert intuition is suspect in ‘low validity’ environments – These are highly complex and irregular environments, which don’t provide reliable predictive cues. – Examples of low validity environments include: medium to long-term political forecasting, stock-market predictions, personnel selection and longer-term clinical judgements, – In these situations, statistical algorithms often outperform professional judgements (Kahneman & Klein, 2009; Grove & Meehl, 1996).