1. Session by Session Monitoring of Outcome and Alliance
David C. Low
Family & Systemic Psychotherapist
Feedback Informed Treatment (ORS & SRS) Trainer
Tel

2. Aim of the Presentation
Provide introduction to the research and clinical rational for Session by Session monitoring of outcome and alliance.
Consider how session by session measuring fits with systemic thinking and practice.
Introduce the Outcome Rating Scale (ORS) and Session Rating Scale (SRS) and how they can be used with families.
Introduce how the measures can be used to bring the clients’ voice into clinical supervision.

3. “So What?” 5 Minute Exercise
Get in small groups of 3.
Introduce yourselves.
Choose someone to feedback themes to the rest of us.
Very briefly discuss the relevance of session by session measuring has to each of you, and why any part of it would be important to you.

4. The Turn to “Outcome”
“Outcome,” “Evidence,” “Effectiveness” &“Accountability” are the watchwords of the day.
Part of a world wide discourse and trend and not specific to mental health.

5. What Works in Therapy: the Tale of Two Paradigms
The Medical Model Evidence Based Practice Paradigm:
Diagnosis + RCT Tx = Good Outcome?
NICE and PBR “What Works for Whom.” To
The Contextual Model Practice Based Evidence Paradigm:
Is this relationship between this client and this therapist working for this young person/family at this time and place?
i.e. Session by Session Measuring

6. Session by Session Monitoring the Evidence
Currently, 13 RCT’s involving 12,374 clinically, culturally, and economically diverse clients provides evidence that routine monitoring and feedback
As much as doubles treatment “effect size”;
Decreases drop-out rates by as much as half;
Decreases deterioration by 33%;
Reduces hospitalizations and shortened length of stay by 66%;
Significantly reduced cost of care (non-feedback groups increased).
Miller, S.D. (2010). Psychometrics of the ORS and SRS. Results from RCT’s and meta-analyses of routine outcome monitoring and feedback: The available evidence.

7. How Does Session by Session Measures Fit with Systemic Thinking?
Cybernetics from Greek kubernētēs steersman, from kubernan to steer, control
Cybernetics concerned with self-regulating systems and how their stability and functioning were maintained through mechanisms of information, control and feedback.
A key notion of cybernetic is the principle of feedback– how information about the results or consequences of a previous action return or loop back into the system in order regulate subsequent sessions.
Session by session, ultra brief, reliable and valid measures provide feedback into the therapeutic system to inform when we are on track and when we are off track and need to adjust or do something different in order to have a positive outcome.

8. Why do I need Formal Measures as I Already ask for Feedback?
Routine tracking of client progress using standardized measures is especially important given clinicians’ tendencies to:
be overly optimistic about the meaning of clients’ lack of progress
their over optimism of eventual improvement in the face of slow progress,
and their failure to judge when clients are headed toward a negative outcome.
Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring,
and feedback in clinical practice. Washington, DC: American Psychological Association.

9. Self-Assessment Bias: Clinicians Have an Overly Positive Assessment of Personal Performance
25% of mental health professionals viewed their skill to be at the 90th percentile when compared to their peers
Most thought they were above average.
None viewed themselves as below average.
Clinicians tended to overestimate their rates of client improvement, and underestimate their rates of client deterioration.
Using session by session measures keeps us grounded, and improving our performance and outcomes.
Walfish, S, McAlister, B., O'Donnell, P., & Lambert, M.J. (2012)An Investigation of Self-assessment Bias in Mental Health Providers. Psychological Reports: Volume 110, Issue, pp

10. Benefits of Session by Session Measuring
Puts the client’s voice at center stage
Allows services to be client and family driven
Brings consumers into the inner circle of decisions
Partners in monitoring the benefit and fit of services
Collaboration, the Next Generation: Going boldly where we have not gone before.

11. Ok, But Every Session?
The evidence demonstrates that frequent measurement, leads to improved outcomes, decreases deterioration rates, and reduces no shows.
Would you advise a parent that it is ok to be consistent some of the time with parenting strategy and still expect their preferred outcome.
Can you imagine a GP who is treating you for high blood pressure saying,  “Do I need to measure the blood pressure EVERY TIME?”
Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring,
and feedback in clinical practice. Washington, DC: American Psychological Association.

12. The purpose of the ORS & SRS:
To give therapists ongoing, formalized feedback about the progress and process of therapy, allowing them to maximize the effect of services.
The ultra brief design and formats for different age groups including small children, allows all members of the family an important feedback voice into the therapeutic system, and enhances everyone’s engagement and participation in family work.
The ORS and SRS have good validity and reliability, and high feasibility.

13. Reluctant Using the ORS & SRS as It Might Infringe on Therapy Time
Using the ORS and SRS to get feedback about the benefit & fit of treatment need not be cumbersome or intrusive on therapy time.
Clients appreciate your dedication to getting it right,
It is painless and can fit your natural way of working.

14. Using the ORS and SRS to Help Level Power Imbalances
Invites clients into inner circle of decision making
Available in multiple languages
Gives a voice to diversity and the disenfranchised
Local culture and context over privileged knowledge

15. ORS & CORS Integrating Outcome Into Care
The ORS, a brief measure of global subjective distress, is administered at the beginning of each session to get real time feedback from young people and carers regarding their perspective of treatment progress.
There is a version for those 13 to adult, and a younger version with smiley faces called CORS for 6-12 year olds.

16. Postmodernism and ORS/CORS
Other measures focus on symptoms and diagnostic constructs and tends to measure dysfunction (or rather peoples perception on constructed notions of dysfunction), and increase possibilities of inviting a person to adopt a 'mental illness' identity.
In contrast, the ORS is a simple generic measure that takes at face value what people construct as meaningful for themselves.
It is easy and feasible to integrate into a variety of practice modalities including post modern models that help clients re-story and develop new narratives.

17. Introducing the ORS/CORS to Young Person and Carer
“The ORS is an outcome measure that allows
us to track where you’re at, how you’re doing,
how things are changing or if they are not. It
allows us to determine whether the therapy is
being helpful so we can do something different
if it’s not helping. It also is way to make sure
that your perspective stays central here, that
we are addressing what you think is most
important. It only takes a minute to fill out and
most clients find it to be very helpful. Would
you like to give it a try? Are you ok with that?”
“Ok, so let me go over the instructions with you”.

18. Introducing the ORS/SRS to other Members of the Family
At most first interviews you won’t know where the distress is, so you can ask all family members present to complete the ORS on themselves. This allows you to "see" who is distressed.
To carer and other family members present: “I would also be grateful if the rest of you would also fill the form out on yourselves to help me understand how things are going for you, and to make sure what we are doing here is benefitting you too.
Even if things are going ok for you, I would be grateful if you could do this anyway, so we can monitor and ensure that whatever we do together doesn't impact you negatively.”

19. Outcome Rating Scale (ORS) (Ages 13 to Adult)
Individually
(Personal well-being)
I I
Interpersonally
(Family, close relationships)
Socially
(Work, school, friendships)
Overall
(General sense of well-being)

20. Child Outcome Rating Scale (CORS) Ages 6-12
(How am I doing?)
I I
Family
(How are things in my family?)
School
(How am I doing at school?)
Everything
(How is everything going?)

21. The ORS: The Bare Bones
The ORS has no specific content other than the 4 domains—a skeleton to which clients add the flesh & blood of their experiences.
At the moment clients connect the marks with what they find distressing, the ORS is a meaningful measure of progress & potent clinical tool. Get a good rating

22. First Session: Connect ORS to the Client’s Described Experience
Principles:
• Where is the distress?
• Constructing meaning of the scores
i.e. Connect the client’s described experience of their lives to the marks on the scales
• Systemic questioning

23. Graphing: A Helpful Visual
Adds a visual component — another way to focus tx
At a glance tells the story of the client’s progress.
Indispensable with couples & families, helps discuss multiple viewpoints

24. ORS/CORS Graph CORS Cutoff (age 6-12) child = 32 carer = 28 ORS Cutoff40 35 30 25 20 15 10 5
Date
CORS Cutoff
(age 6-12)
child = 32
carer = 28
ORS Cutoff
Age 13-17= 28
Age = 25

25. Making the Measures Work with Families
Invite all members of the family to Complete the ORS or CORS.
Invite all members of the family to interpret.
Summarize progress as entry point into the meeting.

26. 15 Year Old ORS
• What stands out from this teens scores in addition to being above the clinical cut off which is 28?
• Suppose the mother’s collateral scoring of the teen is much lower than the teens self report scoring, how could we use this information to begin or focus the session?
7.6
5.7
8.0
7.5
28.8

27. Linking Outcome to Treatment
See clients more frequently when the slope of change is steep.
Begin to space the visits as the rate of change lessens.
See clients as long as there is meaningful change & they desire to continue.

28. ORS/CORS Reliable Change Index
Deterioration
5 point or more worsening from pre-treatment score
Reliable
Change
5 point or more improvement from pre-treatment score
Clinically Significant Change!
5 point or more improvement from pre-treatment score and crossed the cutoff score for age group

29. Alliance comprised of four empirically established components which the SRS Measures:
agreement on goals, meaning or purpose of treatment;
agreement on the means and methods used;
agreement on the therapist’s role (including being perceived as warm, empathic and genuine; and
accommodating the client’s preferences.
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change 2nd ed.: Delivering what works in therapy (pp ).Washington, D.C.: American Psychological Association

30. Why is Monitoring the Alliance So Crucial?
Researchers have repeatedly found that the therapeutic alliance as rated by the client, is one of the best predictors of outcome across different types of therapy including psychopharmocology.
The feedback from the SRS helps clinicians make the necessary adjustments to align simultaneously with all family members in the pursuit of common agreement on goals and tasks in therapy.
Martin, D.J., Garske, J.P., & Davis, M.K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68,
Wampold, B.E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.;
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change 2nd ed.: Delivering what works in therapy (pp ).Washington, D.C.: American Psychological Association.

31. But I Already Ask My Clients for Feedback, Why do I Need use an Alliance Measure?
Research has shown that clinicians are poor at gauging their client’s experience of the alliance and they need to request more formal real time alliance feedback.
The key here is that people THINK they ask more often than they do.
Plus, and more importantly, since the data says it’s the client's point of view that is most predictive of outcome, you don't want to leave it up to the therapist to decide WHEN to ask. 
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change 2nd ed.: Delivering what works in therapy (pp ).Washington, D.C.: American Psychological Association.
Anker, M.G., Duncan, B.L., Owen, J., & Sparks, J.A. (2010). The alliance in couple therapy: Partner influence, early change, ad alliance patterns in a naturalistic sample. Journal Consulting and Clinical Psychology, 78(5),
Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice. Washington, DC: American Psychological Association.

32. Quickest Way to Prevent Drop Out
Clients drop out for 2 reasons: therapy is not helping (monitor outcome) & alliance problems—not engaged or turned on.
Direct way to improve effectiveness is to keep people engaged in therapy.
Gotta measure the alliance to improve your successes!

33. SRS & CSRS Using the Client’s Experience of the Alliance to Guide Practice
The SRS is administered at the end of each session to get feedback on the alliance and therapy process from young people and carers perspective.
Like the ORS there is a version for those 13 on up, and a younger version called CORS for 6-12 year olds.
There is also a Group Version called GSRS.

34. Introducing the SRS/CSRS
“Ok, we need to end in a couple minutes, but before we do I would be grateful if you would take a minute to fill out this other form which asks your opinion about our work together today?
I am not hoping for a perfect score: life is not perfect and I know I am not either. I rely on this feedback to keep me on track, and let me know when I am off track and need to make some changes to ensure that what we are doing is benefitting you.
The ultimate purpose of using these forms is to make every possible effort to make our work together beneficial. There might be some small things that you think you might be able to live with or are not important, but your feedback on even the small things are critical to me as it can be the small things that make the difference. If anything feels missing, or I said something that did not sit right or left you feeling blamed, you would be doing me the best favor if you let me know. Can you help me out?
Ok. Here is one for each of you. Remember please feel free to give your honest opinion when filling this out. Ok?”

35. Session Rating Scale (SRS) (Ages 13 to Adult)
Relationship
I I
Goals and Topics
I I
Approach or Method
Overall
I did feel heard understood, and respected.
I did not feel heard, understood, and respected.
We did work on or talk about what I wanted to work on and talk about.
We did not work on or talk about what I wanted to work on and talk about.
The therapist’s approach is a good fit for me.
The therapist’s approach is not a good fit for me.
There was something missing in the session today.
Overall, today’s session was right for me.

36. Child Session Rating Scale (CSRS) Ages 6-12
Listening
I I
How Important
What We Did
Overall
did not always listen to me.
listened to me.
What we did and talked about were important to me.
What we did and talked about was not really that important to me.
I did not like what we did today.
I liked what we did today.
I wish we could do something different.
I hope we do the same kind of things next time.

37. SRS/CSRS Graph 40 35 30 25 20 15 10 5
Date
SRS Cutoff =36
Discuss

38. SRS Scores Above the Cutoff Score: Keep Convey Negative Feedback is a Gift
Keep encouraging client to let you know…
No bad news on the SRS! Not a measure of competence or anything negative about you or the client.
Negative Feedback is a gift from the client that helps you to be better.
Unless you really want it, you are unlikely to get it.
You won’t get it from everyone.

39. What About Below 36 or 9cm Scores < 36 or 9cms should be discussed.
Don’t expect specifics or revelations—any feedback is a godsend
“Is there anything else I could have done, something I should have done more of or less of, some question or topic I should have asked, was there anything I said that did not sit right with you?”

40. Low SRS Scores Can be OK Caution:
Don’t assume a low SRS score means that there are alliance issues.
The most effective therapists tend to have lower SRS scores and seem to be more skilled at generating honest feedback and using negative feedback to improve performance and outcomes
Key is to consider if the ORS/CORS scores are improving. If not, there may be a rupture or problem with the alliance.

41. Does Not Seem to be Improving
“Okay, so things haven’t changed since the last time we talked. How do you make sense of that?
Should we be doing something different here, or should we continue on course steady as we go?
If we are going to stay on the same track, how long should we go before getting worried?
When will we know when to say ‘when’ and do something different?”

42. Using Session by Session Measures in Supervision
The measures and the graphs bring the feedback and voice of the young person and carer directly into the supervisory session.
Supervisors and supervisees can utilize the measures and graphs from a variety of cases to reflect on potential areas for professional growth and development.

43. How to Get the Measures Free
The ORS/CORS and SRS/CSRS measures are licensed for member groups of CORC and CYP IAPT by Scott D. Miller.
Individual clinicians may also obtain a licence and the measures at no cost in English, and languages other than English, at
Note: When down loading the measures for the first time, you may need to adjust the analogue scales to make sure they are exactly 10cm. Beyond that adaptation the license agreement involves no alterations to the ORS/CORS and SRS/CSRS.

44. Further Help and Training in using the ORS/CORS and SRS/CSRS
Information on Training in how to integrate real time outcome and alliance feedback using the ORS & SRS to improve clinical effectiveness with young people and families contact:
David C. Low, Family and Systemic Psychotherapist,
Scott D. Miller, Ph.D., Director of the International Center for Clinical Excellence --

45. “So What, Now?” 5 Minute Exercise
Get back into your small groups of 3.
Choose someone to feedback themes to the rest of us.
Very briefly discuss what you each might try doing as a result of the material covered this morning.