How To Be An Evidence-Based Psychologist

Slides:



Advertisements
Similar presentations
Evidence-based Dental Practice Developing guidelines or clinical recommendations Slide #1 This lecture follows the previous online lecture on evidence.
Advertisements

Integrating the NASP Practice Model Into Presentations: Resource Slides Referencing the NASP Practice Model in professional development presentations helps.
Chronic disease self management – a systematic review of proactive telephone applications Carly Muller Dean Schillinger Division of General Internal Medicine.
Striving to Keep Up with the Field of Evidence-Based Interventions: Redesign of a Child Psychotherapy Seminar Jennifer West PhD, Wendi Cross PhD, and Pamela.
Development and Implementation of a Recovery-Based System: Comparison of Instruments for Assessing Recovery Jeanette M. Jerrell, Ph.D. Professor of Neuropsychiatry,
Attention deficit hyperactivity disorder Implementing NICE guidance 2008 NICE clinical guideline 72.
Depression in adults with a chronic physical health problem
Definitions of EBP Popular in SW
Strategies for Implementing Outcomes in Practice Carolyn Baum, PhD, OTR, FAOTA.
USE OF EVIDENCE IN DECISION MODELS: An appraisal of health technology assessments in the UK Nicola Cooper Centre for Biostatistics & Genetic Epidemiology,
BG Studies of Psychopathology One of the most actively researched areas in recent years It is estimated that 1/3 of the U.S. population suffers from some.
Improving Access to Psychological Therapies (IAPT) in London - Implementing NICE Guidance Professor Stephen Pilling PhD Director, National Collaborating.
1 Phase III: Planning Action Developing Improvement Plans.
US Office of Education K
Click the arrows to advance forward and backward. Click the Next link below to advance to the assessment. The A B C & D’s of Suicide Assessment and Clinical.
Implementation of MST in Norway Iceland June 2008 Bernadette Christensen Clinical Director of the Youth Department Anne Cathrine Strütt MST Consultant.
From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education Ronnie Detrich Wing Institute.
Measuring: the real life challenges Lessons and reflections the NHS Lothian Early Implementer Site Linda Irvine Strategic Programme Manager, Mental Health.
What is going on with psychotherapy today? Carolyn R. Fallahi, Ph. D.
99.98% of the time patients are on their own “The diabetes self-management regimen is one of the most challenging of any for chronic illness.” 0.02% of.
Biennial Report: Effective Psychosocial Interventions for Youth with Behavioral and Emotional Problems Training School Psychologists to be Experts.
Introduction to evidence based medicine
Daniel Flynn 1, Mary Kells 1, Mary Joyce 1&2, Catalina Suarez 1&2 1. Health Service Executive 2. National Suicide Research Foundation The National Dialectical.
The Evidence Base for Psychosocial and Psychopharmacological Interventions for Children with Attention-Deficit/Hyperactivity Disorder, Major Depressive.
The European Network for Traumatic Stress Training & Practice
PSYCHOTHERAPY LECTURE OUTLINE The context of psychotherapy Evaluating the effects of psychotherapy Evidence-based practice Marital, family, and group therapy.
Implementing NICE guidance
EVIDENCE BASED PRACTICE
Role of Empirically Supported Treatments in Teaching Evidence- Based Practice Sheila Woody channelling…. Dianne L. Chambless University of Pennsylvania.
Janice Berry Edwards, PhD, LICSW, LCSW-C, BCD, ACSW
Presenter-Dr. L.Karthiyayini Moderator- Dr. Abhishek Raut
INFORMATION FROM: DUNCAN, B.L., MILLER, S.D., WAMPOLD, B.E., & HUBBLE, M.A. (EDS). (2010). THE HEART & SOUL OF CHANGE: DELIVERING WHAT WORKS IN THERAPY.
THE WORLD OF ABNORMAL PSYCHOLOGY COLLEGE COURSE PSYCH 219 Dr. Mary Ann Woodman Rogue Community College.
Categories of Mental Disorders 1 Child and youth mental health problems can be classified into two broad categories: 1Internalizing problems  withdrawal.
Medication Management Toolkit A guide to the use of medication algorithms to guide clinical care.
Chapter 13 INTRODUCTION TO CLINICAL PSYCHOLOGY, THIRD CANADIAN EDITION by John Hunsley and Catherine M. Lee.
Chapter 10 Counseling At Risk Children and Adolescents.
Understanding TASC Marc Harrington, LPC, LCASI Case Developer Region 4 TASC Robin Cuellar, CCJP, CSAC Buncombe County.
June 11, IOM, Reducing Suicide, 2002 Statement of Task w Assess the science base w Evaluate the status of prevention w Consider strategies for studying.
Chapter 11 INTRODUCTION TO CLINICAL PSYCHOLOGY, THIRD CANADIAN EDITION by John Hunsley and Catherine M. Lee.
Evidence-Based Public Health Nancy Allee, MLS, MPH University of Michigan November 6, 2004.
Introduction: Medical Psychology and Border Areas
RHS 303. TRANSITION OF THEORY AND TREATMENT nature of existence and gives meaning to and guides the action Philosophical Base: Philosophy of occupational.
Evidence-Based Psychotherapies for Managing PTSD in the Primary Care Setting Kyle Possemato, Ph.D. Clinical Research Psychologist Collaborative Family.
Interventions with Adults Chapter 6. Background Psychosocial treatment is an inclusive term and is defined as: a relationship that occurs between a professional.
Chapter 10: Depressive Disorders in Adolescents Megan Jeffreys V. Robin Weersing.
Evidence Based Practice RCS /9/05. Definitions  Rosenthal and Donald (1996) defined evidence-based medicine as a process of turning clinical problems.
An Expanded Model of Evidence-based Practice in Special Education Randy Keyworth Jack States Ronnie Detrich Wing Institute.
Is the conscientious explicit and judicious use of current best evidence in making decision about the care of the individual patient (Dr. David Sackett)
5 Ways to achieve parity in mental health Karen Turner Director of Mental Health, NHS England 9 th December.
Chapter 14 INTRODUCTION TO CLINICAL PSYCHOLOGY, THIRD CANADIAN EDITION by John Hunsley and Catherine M. Lee.
Group members Gurpreet kaur Amritpal kaur Arshdeep singh uppal Sandeep kaur bhullar.
Cognitive Behaviour Therapy (CBT) For Anxiety And Depression.
Ethical Issues in Treatment Selection Northern Arizona University Timothy C. Thomason.
Psychodynamic Psychotherapy: A Systematic Review of Techniques, Indications and Empirical Evidence Falk Leichsenring & Eric Leibing University of Goettingen,
1 Psychology Service 2012 Louis Stokes Cleveland VA Medical Center.
EVIDENCE BASED PRACTICE ATHANASIA KOSTOPOULOU ERASMUS IPs
Evidence-Based Mental Health PSYC 377. Structure of the Presentation 1. Describe EBP issues 2. Categorize EBP issues 3. Assess the quality of ‘evidence’
Algorithms for Mental Health Conditions
Psychological treatments for mental disorders in adults: A review of the evidence of leading international organizations Juan Antonio Moriana, Mario Gálvez-Lara.
CHAPTER 1: Introduction
Evidence Based Practice Process
Review of Evidence-Based Practice and determining clinical questions to address This group of 17 slides provides a nice review of evidence-based.
Quality Health Care Nursing 870
24/04/2012 NICE guidance and best practice in psychological care for “bipolar disorder” Dr Graeme Reid, Consultant Clinical Psychologist, Step 5, Central.
Expanding Application of Clinical Significance for Outcome Measures: BAI and R-DAS Zack Hamingson, MA, Matt Multach, Jermaine Dictado, Kenichi Shimokawa,
MeOTa fall conference October 22, 2016
CHAPTER 9 MULTICULTURAL EVIDENCE-BASED PRACTICE
Evidence-Based Public Health
Evidence-based Practice in Psychology (EBPP)
Presentation transcript:

How To Be An Evidence-Based Psychologist John Hunsley hunch@uottawa.ca

OVERVIEW What Is Evidence-Based Psychological Practice (EBPP) Evidence-base practice, empirically supported treatments, clinical practice guidelines Examining the research evidence Evidence-based treatments, therapeutic relationships, and assessment Dissemination and implementation of EBPP Implications, challenges, and opportunities

SOME DEFINITIONS Evidence-Based Practice (and Evidence-Based Practice in Psychology) Empirically Supported Treatments Practice Guidelines Randomized Controlled Trials (RCTs) Efficacy Studies & Effectiveness Studies Treatment As Usual (TAU)

EVIDENCE-BASED PRACTICE The use of systematically collected data, clinical expertise, and patient preferences by decision-makers (including clinicians, administrators, and policy makers) when considering service options. Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. British Medical Journal, 312, 71-72. Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285.

EVIDENCE-BASED PRACTICE Providing the right health care services—services that have been demonstrated to work—for each client’s needs Services are based on empirical evidence but are individually tailored to take into account client characteristics, needs, and resources Services may also need to be adjusted in order to fit the demands and constraints of real world clinical practice

IMPLEMENTATION OF EVIDENCE-BASED SCURVY PREVENTION PROGRAMS: AN EXAMPLE WE DO NOT WANT TO REPEAT 1497 Vasco da Gama sailed around Cape of Good Hope: 100 of 160 sailors died of scurvy (due to deficiency of vitamin C) 1601 James Lancaster RCT (4 ships England to India, 1 on which sailors received 3 tsps of lemon juice each day): Halfway, 110/278 died of scurvy on 3 ships; 0 on the “lemon” ship 1747 James Lind RCT of 6 treatments (including citrus juice) replicates Lancaster finding 1768-1780 James Cook (early adopter), 3 voyages, required sauerkraut in the rations Only 3 sailors died of scurvy 1795 British Navy required citrus fruit as part of diet (limey) 1865 British Board of Trade required citrus fruit as part of diet Berwick. (2003). Disseminating innovations in health care. Journal of the American Medical Association

Best Available Evidence Client/Patient Preferences THE EBP MODEL Best Available Evidence EBP Client/Patient Preferences Clinical Expertise 7

BEST AVAILABLE RESEARCH EVIDENCE (TREATMENT)

EVIDENCE-BASED PSYCHOLOGICAL PRACTICE Influenced by multiple factors in North America Evidence-Based Medicine Scientist-Practitioner Model Accountability/Quality Assurance Empirically Supported Treatments

EBP IN PSYCHOLOGY: THE CONTROVERSY Concerns: loss of professional autonomy takeover of professional psychology by specific interest groups dehumanization of psychological services inadequacy of the research base impossibility of basing care on research evidence Similar concerns raised in other health professions

APA DIVISION 12 EMPIRICALLY SUPPORTED TREATMENTS: CRITERIA FOR WELL-ESTABLISHED TREATMENTS I. At least 2 good between group design experiments demonstrating efficacy in one or more of the following ways: A. Superior (stat. sign.) to pill or psychological placebo or to another treatment B. Equivalent to an already established treatment in experiments with adequate sample sizes OR II. A large series of single case design experiments (n > 9) demonstrating efficacy. These experiments must have: A. Used good experimental designs and B. Compared the intervention to another treatment. --highlight experiments—RCT as the gold standard --if a large series of studies with mixed findings, go with the preponderance of findings

CRITERIA FOR WELL-ESTABLISHED TREATMENTS FURTHER CRITERIA FOR BOTH I AND II: III. Experiments must be conducted with treatment manuals or equivalent clear descriptions of treatment IV. Characteristics of the client samples must be clearly specified V. Effects must have been demonstrated by at least 2 different investigators or teams

CAUTION: ESTs ≠ EBT Difference between (a) research on a treatment meeting a pre-established set of criteria and (b) determining which treatments have strongest support for a specific condition Despite this, terms are now being used almost interchangeably can be confusing, especially as trend in professional psychology has been to use lists rather than encourage the use of practice guidelines or having individual clinicians conducting their own literature searches

EBTs FOR CHILDREN & ADOLESCENTS (Multiple Single Case Designs or Higher) Autistic Disorder Attention-Deficit/Hyperactivity Disorder Anxiety Disorders Chronic Pain Conduct Problems & Oppositional Defiant Disorder Major Depressive Disorder Eating Disorders Elimination Disorders Obesity Tic Disorders

EBTs FOR ADULTS Anxiety Disorders Specific Phobias Social Phobia Panic Disorder (with/without Agoraphobia) GAD OCD PTSD Major Depressive Disorder Bipolar Disorder

EBTs FOR ADULTS Eating Disorders Anorexia Nervosa Bulimia Nervosa Binge-Eating Disorder Sleep Disorders Substance-Related Disorders Alcohol Abuse Cocaine Abuse Opiate Abuse

EBTs FOR ADULTS Tic Disorders Sexual Disorders Schizophrenia Marital/Couple Conflict Personality Disorders Avoidant PD Borderline PD Somatoform Disorders Pain Disorders Body Dysmorphic Disorder Hypochondriasis

SOME ADDITIONAL EBTs Anger Management Anxiety/fear associated with medical/dental procedures Assertiveness Skills Parent Training Social Skills Stress Management Many also available for specific illnesses/chronic health conditions (e.g., Chronic Fatigue Syndrome, Irritable Bowel Syndrome) Psychometrically strong assessment measures available for assessing outcomes from all EBTs

CLINICAL PRACTICE GUIDELINES Common in many health professions; in professional psychology, they are almost totally absent (but coming soon from APA) In general, they are consensus statements from experts/professional organizations/healthcare organizations that present best clinical practices (screening, assessment, consultation, treatment, referral, etc.) Not the same as guidelines for reimbursement or other administrative purposes Usually encourage appropriate initial assessment, without any further guidance on how to do this (more on this later)

CLINICAL PRACTICE GUIDELINES National Institute for Health and Clinical Excellence (NICE) sponsored by National Health Service in England and Wales Use of explicit evidence hierarchy Extensive consultations undertaken with stakeholder organizations (both consumer and professional groups) Each guideline has a limited “life,” ensuring review in near future http://www.nice.org.uk/

CLINICAL PRACTICE GUIDELINES: NICE RECOMMENDATIONS Some current guidelines recommending use of a psychological intervention ADHD Anxiety Disorders (Panic Disorder, Agoraphobia, Generalized Anxiety Disorder) Bipolar Disorder Chronic Fatigue Syndrome Conduct Disorder Depression (child, adolescent, & adult) Eating Disorders Obsessive-Compulsive Disorder & Body Dysmorphic Disorder Personality Disorders (Antisocial, Borderline) Posttraumatic Stress Disorder Schizophrenia

BUT AREN’T ALL THERAPIES EQUIVALENT? Most meta-analyses suggest that this is not the case for most disorders/conditions Wampold et al. (1997) meta-analysis of bona fide comparative treatment studies Mean effect size = .19 Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes.” Psychological Bulletin, 122, 203-215. But, over ¾ of comparisons within types of CBT NNT was 9 (for comparison, interferon vs. placebo to slow progression of multiple sclerosis has NNT of 9) For depression, marital conflict, PTSD, & bipolar disorder, however, there do appear to be multiple treatments with similar results

EBT COMPARED WITH TAU Meta-analysis of 32 studies of youth interventions found majority of EBTs yielded outcomes superior to TAU (most studies focused on externalizing disorders) Mean effect size for EBT versus TAU was 0.30 This indicates that the average “EBT” youth was better off after treatment than 62% of “TAU” youth Alternatively, this can be represented as a NNT of 6 (similar NNT for average effects of adding radiation treatment to chemotherapy for a range of cancers) Weisz, Jensen-Doss, & Hawley (2006). Psychological Bulletin.

EBT COMPARED WITH TAU Data on treatment outcome for adults, in real world clinics: Of over 6000 clients, 35% rate of improvement/recovery (Hansen, Lambert, & Forman, 2002, Clinical Psychology: Science and Practice, 9, 329-343) Of over 6100 clients, 29% rate of improvement/recovery (Wampold & Brown, 2005, Journal of Consulting and Clinical Psychology, 73, 914-923) Data on treatment outcome for adults, in RCTs with EBTs Of over 2100 clients, 67% rate of improvement/recovery (Hansen, Lambert, & Forman, 2002, Clinical Psychology: Science and Practice, 9, 329-343)

BUT ARE EBTs CLINICALLY RELEVANT? Are the participants in treatment studies similar enough to patients routinely seen in practice settings to warrant generalizing research results to clinical practice? Is the research literature on psychological treatments sufficiently developed to be applicable to the broad range of conditions encountered in practice?

BUT ARE EBTs CLINICALLY RELEVANT? Participants in RCTs/efficacy trials are unlike “real” clients because ……….. They are only the “worried well” They are only “pure” cases

BUT ARE EBTs CLINICALLY RELEVANT? Patients included in RCTs for cocaine dependence had symptoms comparable to, but more severe than, those found reported by patients receiving outpatient services Carroll et al. (1999, Drug and Alcohol Dependence) Compared patient files from a managed behavioral health care network to the inclusion and exclusion criteria used in numerous RCTs for adults with mental disorders Over ½ of patients would have been ineligible for RCTs because symptoms were not severe enough to warrant inclusion (in most cases, due to patient diagnosis of adjustment disorder) Stirman et al. (2003, Journal of Consulting and Clinical Psychology)

BUT ARE EBTs CLINICALLY RELEVANT? For youth RCTs, approx. half used no exclusion criteria related to comorbidity Weisz et al. (2004, Child and Adolescent Psychiatric Clinics of North America) In adult disorder efficacy RCTs, only “clinical appropriate” exclusion criteria used for many years now Can see this even almost 20 years ago in US NIMH Collaborative Depression Treatment study, with approximately ¾ having personality disorders

BUT ARE EBTs CLINICALLY RELEVANT? Single RCTs, or more, available for most DSM IV Axis I and Axis II disorders, and many nondiagnosable conditions Hawaii study of 2,200 youth receiving services 89% had a primary diagnosis for which an EBT was available In terms of treatment targets, 90% had 1 or more problems for which an EBT was available On the other hand, for only 3% were there EBTs for all treatment targets Schiffman et al. (2006). Evidence-based services in a statewide public mental health system: Do the services fit the problems? Journal of Clinical Child and Adolescent Psychology, 35, 13-19.

EFFECTIVENESS RESEARCH Reviewed EBT effectiveness studies published prior to April 2006 21 studies of adult treatment and 13 of child/adolescent treatment met criteria (including at least 2 effectiveness trials) Compared results to benchmarks from reviews of efficacy studies (mainly meta-analysis) Hunsley & Lee (2007). Research-informed benchmarks for psychological treatments: Efficacy studies, effectiveness studies, and beyond. Professional Psychology: Research and Practice, 38, 21-33.

EFFECTIVENESS RESEARCH Completion rates higher than usually reported in studies of “real world” psychotherapy For both adult and youth disorders, the average improvement rates were similar to efficacy benchmarks Some examples: Adult Depression: 74% completed, 51% improved Adult OCD: 88% completed, 64% improved Youth Anxiety Disorders: 87% completed, 63% improved

EFFECTIVENESS RESEARCH PTSD Treatments Omagh Bombing Gillespie et al. (2002). Behaviour Research and Therapy, 40, 345-357. 9/11 World Trade Center Levitt et al. (2007). Behaviour Research and Therapy, 45, 1419-1433. Vast majority completed treatment, with results very similar to efficacy RCTs Up to 25-30 sessions; most clinicians had limited CBT background, received training over several days, weekly/monthly supervision

EBTs: CONCLUSIONS Does treatment research generalize to practice? Compelling evidence that it does, but always need to exercise caution in applying results to a particular individual Much more research on “mild” conditions and Axis II conditions needed How effective are evidence-based treatments (EBT) in clinical practice? More evidence needed, but EBTs are usually as effective as in efficacy trials Hunsley, J. (2007). Addressing key challenges in evidence-based practice in psychology. Professional Psychology: Research and Practice, 39, 113-121. Hunsley, J. (2007). Training psychologists for evidence-based practice. Canadian Psychology, 47, 32-42.

EBPP: “FLEXIBILITY WITHIN FIDELITY” Treatment manuals now focus attention on key elements of treatments, not just session by session list of activities/strategies Increased recognition of importance of tailoring treatment to clients, especially aspects related to cultural diversity and presence of multiple disorders/problems components Call for attention to principles of change and commonly used techniques/strategies, not “trademarked” therapies Some examples: David Barlow: Unified treatment for mood and anxiety disorders Bruce Chorpita: Modular CBT for youth disorders

EBPP: INCLUDING CLINICAL EXPERTISE Evidence-Based Therapy Relationships Norcross, J. C. (Ed.). (2011). Psychotherapy relationships that work: Evidence-based responsiveness (2nd ed.). New York: Oxford University Press. There is more to EBT than RCTs Client expectations Therapist empathy Therapeutic alliance Repairing therapeutic ruptures Culturally adapted treatments

EVIDENCE-BASED ASSESSMENT (EBA) Use of research and theory to guide: The selection of constructs to be assessed for a specific assessment purpose The methods and measures to be used in the assessment The manner in which the assessment process unfolds (including integration and interpretation of assessment data) Hunsley, J., & Mash, E. J. (2007). Evidence-based assessment. Annual Review of Clinical Psychology, 3, 29-51.

WHY IS EBA NEEDED? “…blanket recommendations to use reliable and valid measures when evaluating treatments are tantamount to writing a recipe for baking hippopotamus cookies that begins with the instruction “use one hippopotamus,” without directions for securing the main ingredient.” Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362-379.

WHY IS EBA NEEDED? Almost no overlap among the psychological measures found in surveys (USA, UK) to be commonly used by psychologists (and those commonly taught to clinical graduate students) and the measures necessary to implement and monitor EBTs adapt treatments based on the consideration of Evidence-Based Therapy Relationship (EBTR) elements How can we use EBT and EBTR information without using appropriate assessment tools in practice?

A GUIDE TO ASSESSMENTS THAT WORK Hunsley and Mash (2008) “good enough” criteria for use of instruments Must balance psychometric ideals with clinical realities Must keep in mind issues such as age, gender, ethnicity in determining relevance of instruments and supporting data Presented criteria used in rating norms, reliability indices (internal consistency, inter-rater reliability, test-retest reliability), validity indices (content validity, construct validity, validity generalization, treatment sensitivity), and clinical utility

A GUIDE TO ASSESSMENTS THAT WORK Youth: ADHD, Conduct Problems, Depression, Self-Injurious Thoughts and Behaviors, Anxiety Disorders, Pain Adults: Depression, Bipolar Disorder, Self-Injurious Thoughts and Behaviors, Anxiety Disorders, Substance Abuse Disorders, Alcohol Use Disorders, Gambling Disorders, Schizophrenia, Personality Disorders, Couple Distress, Sexual Dysfunction, Paraphilias, Eating Disorders, Sleep Disorders, Pain Hunsley, J., & Mash, E. J. (Eds.). (2008). A guide to assessments that work. New York: Oxford University Press.

WHY IS EBA NEEDED? When asked to rate 3 most important problems to address in treatment, 77% of child-parent-therapist triads failed to agree on a single problem Correlations between clinician and youth symptom and self-esteem measures <.23 Hawley, K. M., & Weisz, J. R. (2003). Child, parent, and therapist (dis)agreement on target problems in outpatient therapy: The therapist’s dilemma and its implications. Journal of Consulting and Clinical Psychology, 71, 62-70. Love et al. (2007). Meeting the challenges of evidence-based practice: Can mental health therapists evaluate their practice? Brief Treatment and Crisis Intervention, 7, 184-193. .

WHY IS EBA NEEDED? In a population-based study of Canadian adults who received psychotherapy in past year, 43% terminated services because they felt better But, 14% terminated because they felt therapy was not helping, and 7% were not comfortable with the therapist’s approach Clinicians identified <50% of treatment successes (as rated by patients) and failed to identify the 10% of patients who terminated because they felt treatment was worsening their problems Westmacott & Hunsley. (2010). Reasons for terminating psychotherapy: A general population study. Journal of Clinical Psychology, 66, 965-977 Hunsley et al. (1999). Comparing therapist and client perspectives on reasons for psychotherapy termination. Psychotherapy, 36, 380-388. .

WHY IS EBA NEEDED? Limited evidence of clinical utility for commonly used instruments MMPI-2 completed by all clients prior to treatment Half of clinicians (randomly assigned) received test results; half did not All clients received an appropriate EBT (based on diagnosis) Having MMPI-2 information had no impact on the number of sessions patients attended whether therapy ended prematurely overall patient improvement in functioning assessed in the end of treatment. Lima et al. (2005). The incremental validity of the MMPI-2: When does therapist access not enhance treatment outcome? Psychological Assessment, 17, 462-468.

A GUIDE TO ASSESSMENTS THAT WORK Focus on specific assessment purposes directly pertinent to clinical interventions: diagnosis (including screening issues and the importance of addressing comorbidity) case conceptualization and treatment planning treatment monitoring and treatment evaluation

DIAGNOSIS Diagnostic information allows access to relevant research on psychopathology, epidemiology, prognosis, and treatment Client characteristics (common comorbid conditions, likely health concerns) and social/interpersonal characteristics (common problems or limitations associated with social networks and intimate relationships, work functioning, and healthcare utilization) that are likely to merit further evaluation or consideration in treatment planning e.g., clients meeting criteria for a substance abuse disorder are likely to abuse additional substances, and those who abuse multiple substances are least likely to benefit from treatment Rohsenow, D. (2008). Substance use disorders. In J. Hunsley & E. J. Mash (Eds.), A guide to assessments that work (pp. 319-338). New York: Oxford University Press.

DIAGNOSIS Diagnosis has utility for EBTs i.e., points in the direction of treatment options based on research evidence (e.g., NICE guidelines) Can give directions for treatment planning with respect to comorbidity e.g., presence of depression among people with OCD can diminish the effectiveness of exposure and response prevention Abramowitz, J. S., Franklin, M. E., Kozak, M. J., Street, G. P., & Foa, E. B. (2000). The effects of pre-treatment depression on cognitive-behavioral treatment outcome in OCD clinic patients. Behavior Therapy, 31, 517-528.

DIAGNOSIS Symptom profile can be used to guide treatment selection e.g., bipolar disorder treatments to reduce manic symptoms should address medication adherence and recognition of mood changes; treatments focusing on cognitive and interpersonal coping strategies reduce depressive symptoms Miklowitz, D. J. (2008). Adjunctive psychotherapy for bipolar disorder: State of the evidence. American Journal of Psychiatry, 165, 1408-1419. Symptom profile can also lead to emphases in treatment strategies or addition of treatment strategies Chorpita’s modular treatment approach Chorpita, B. F. (2006). Modular cognitive– behavioral therapy for childhood anxiety disorders. New York: Guilford Press. Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments for 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566-579..

DIAGNOSIS Study of community based services for adolescents Disagreement between clinician-generated and research-based diagnoses were associated with a host of treatment implementation problems including: Increased number of client “no-shows” Cancelled treatment appointments Treatment drop-outs Inaccurate clinician-generated diagnoses were also associated with smaller treatment gains Jensen-Doss, A., & Weisz, J. R. (2008). Diagnostic agreement predicts treatment process and outcomes in youth mental health clinics. Journal of Consulting and Clinical Psychology, 76, 711-722.

TREATMENT MONITORING AND EVALUATION What do all RCTs, used as evidence for EBTs, have in common? Routine collection of monitoring data, typically reviewed by clinicians during the supervision of services being provided Hypothesis: The fact that treatment is repeatedly monitored, and information provided to clinicians, is one of the most important contributors to successful treatment outcome in EBTs Collection of assessment data as treatment unfolds allows for making any needed adjustments as required Does this apply to routine practice?

TREATMENT MONITORING AND EVALUATION The potential benefits of treatment monitoring Outcome Questionnaire – 45 http://www.oqmeasures.com Over 2,500 clients from a range of clinics All completed OQ weekly, half of clinicians received feedback, namely RED, YELLOW, & GREEN dots No feedback: 21% improved, 21% worsened Feedback: 35% improved (i.e., 66% increase), 13% worsened (i.e., 33% decrease) Lambert et al. (2003). Is it time to track patient outcome on a routine basis? A meta-analysis. Clinical Psychology: Science and Practice, 10, 288-301.

TREATMENT MONITORING AND EVALUATION Most recent meta-analyses Lambert & Shimokawa. (2011), Collecting client feedback. In Norcross (Ed.). Psychotherapy relationships that work (2nd ed.). Outcome Questionnaire (4 studies, over 6100 clients) 2.6X greater likelihood of improvement Less than half the likelihood of deterioration Partners for Change Outcome Management System (3 studies, over 550 clients) 3.5X greater likelihood of improvement

TREATMENT MONITORING AND EVALUATION Three specific categories of client and treatment variables that should be given particular consideration: Treatment targets and goals (intermediate and ultimate) Causal mechanisms believed to be maintaining client problems Therapeutic context or process variables are particularly relevant to enhancing treatment services In all instances, validity evidence for sensitivity to treatment change is particularly relevant in selecting measures

TREATMENT MONITORING AND EVALUATION To practice in an evidence-based manner, it is inappropriate to claim that the treatment that a client is receiving is effective simply because it is an EBT—to determine the impact of any treatment, including an EBT, it is essential that data are collected in order to accurately determine its effects for individual clients Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child & Adolescent Psychiatry, 47, 1114-1119.

INTEGRATION AND INTERPRETATION OF ASSESSMENT DATA Clear evidence that clinicians (like people in general) are not very accurate in self-assessment, have limited awareness of biases, and are influenced by numerous heuristics in the decisions they make Ægisdóttir et al. (2006). The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34, 341-382. Davis et al. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. Journal of the American Medical Association, 296, 1094-1102. Garb, H. (2005). Clinical judgment and decision making. Annual Review of Clinical Psychology, 1, 67-89.

INTEGRATION AND INTERPRETATION OF ASSESSMENT DATA Use decision aids (such as decision trees in DSM-IV-TR, research based data, norms, practice guidelines) Use standardized, psychometrically strong instruments to collect data Collect data from multiple sources (remembering that different informants have differing experiences, perspectives, & attributions) Ensure all aspects of assessment processes are sensitive to diversity factors Continue to evaluate and adjust hypotheses based on data Treat all formulations as tentative Remember heuristics (e.g., primacy effect, availability heuristic) and biases (e.g., attributional biases, orientation influence) and actively address them

EBA INITIATIVES Antony, M. M., Orsillo, S. M., & Roemer, L. (Eds.). (2001). Practitioner's guide to empirically based measures of anxiety. New York: Plenum. Nezu, A. M., Ronan, G. F., Meadows, E. A., & McClure, K. S. (Eds.). (2000). Practitioner’s guide to empirically based measures of depression. New York: Kluwer Academic. Journal of Clinical Child and Adolescent Psychology (2005) special section Psychological Assessment (2005) special section Journal of Pediatric Psychology (2008) special issue A Guide to Assessments That Work (2008) Hunsley & Mash Handbook of Child and Adolescent Diagnostic and Behavioral Assessment (in press) McLeod, Jensen-Doss, & Ollendick

THE PROFESSIONAL CHALLENGES OF EBPP Critical self-evaluation of knowledge and skills Critically evaluate limits of competence Time constraints for learning and skill development What is sufficient training (peer supervision/consultation) Competence + Frequency = ↑ Positive Outcomes implications for specialist vs. generalist? Possible impact on income/job requirements (referring to others vs. time for training vs. client hours) Balancing the ethical/professional issue of competence with reality of availability of appropriately trained mental health professionals

EBPP: KEEPING UP TO DATE Example of CPA Code of Ethics (2000) II.9 (Psychologists) Keep themselves up to date with a broad range of relevant knowledge, research methods, and techniques, and their impact on persons and society, through the reading of relevant literature, peer consultation, and continuing education activities, in order that their service or research activities and conclusions will benefit and not harm others. Information overload: Primary care physicians need 21 hours/day to read all relevant articles that are published! Alper et al. (2004). Journal of the Medical Library Association. Clearly some strategies and summaries are necessary

OXFORD UNIVERSITY PRESS SERIES

EBPP RESOURCES Antony & Barlow (Eds.), (2010). Handbook of Assessment and Treatment Planning for Psychological Disorders (2nd ed.). Guilford Press. Clinician’s Research Digest “Clinicians don't have to read all the journals publishing research of interest to them—the Editor and staff of Clinician's Research Digest do it for them. CRD reviews over 100 journals each month and highlights the most relevant articles in this 6-page monthly newsletter.” Evidence-Based Mental Health “Evidence-Based Mental Health journal surveys a wide range of international medical journals applying strict criteria for the quality and validity of research. Practising clinicians assess the clinical relevance of the best studies. The key details of these essential studies are presented in a succinct, informative abstract with an expert commentary on its clinical application.”

TREATMENTS THAT WORK SERIES http://www.oup.com/us/companion.websites/umbrella/treatments/?view=usa Some examples: Stress Management Hoarding Coping with Chronic Illness Social Anxiety Adult ADHD Fears and Phobias Panic Worry Eating Disorders Pathological Gambling Insomnia School Refusal

ADVANCES IN PSYCHOTHERAPY: EVIDENCE-BASED PRACTICE SERIES http://www.hogrefe.com/program/books/book-series/advances-in-psychotherapy-evidence-based-practice.html Some examples: Bipolar Disorder Heart Disease Obsessive-Compulsive Disorder Childhood Maltreatment Schizophrenia Treating Victims of Mass Disaster and Terrorism Attention-Deficit/Hyperactivity Disorder in Children and Adults Problem and Pathological Gambling Chronic Illness in Children and Adolescents Alcohol Use Disorders Borderline Disorder Prostate Cancer Diabetes

EBPP RESOURCES: WEB SITES The Cochrane Collaboration http://www.cochrane.org/ The Cochrane Library http://www.thecochranelibrary.com/view/0/index.html “The Cochrane Library contains high-quality, independent evidence to inform healthcare decision-making. It includes reliable evidence from Cochrane and other systematic reviews, clinical trials, and more. Cochrane reviews bring you the combined results of the world’s best medical research studies, and are recognised as the gold standard in evidence-based health care.”

EBPP RESOURCES: WEB SITES TRIP Database http://www.tripdatabase.com/index.html “Welcome to the TRIP Database, the Internet's leading resources for Evidence-Based Medicine. Allowing users to rapidly identify the highest quality clinical evidence for clinical practice.” Enter a term (e.g., adolescent anxiety, postpartum depression, psychosis) and search engine will list systematic reviews, evidence based synopses, guidelines (international), e-textbooks, etc.

EBPP RESOURCES: WEB SITES National Registry of Evidence-Based Programs and Practices http://www.nrepp.samhsa.gov/ “Welcome to the National Registry of Evidence-based Programs and Practices (NREPP), a service of the Substance Abuse and Mental Health Services Administration (SAMHSA). NREPP is a searchable database of interventions for the prevention and treatment of mental and substance use disorders. SAMHSA has developed this resource to help people, agencies, and organizations implement programs and practices in their communities.”

EBPP RESOURCES: WEB SITES Society of Clinical Psychology (APA Division 12) http://www.div12.org/PsychologicalTreatments/index.html “The purpose of this website is to provide information about effective treatments for psychological disorders. The website is meant for a wide audience, including the general public, practitioners, researchers, and students. Basic descriptions are provided for each psychological disorder and treatment. In addition, for each treatment, the website lists key references, clinical resources, and training opportunities. The American Psychological Association has identified "best research evidence" as a major component of evidence-based practice (APA Presidential Task Force on Evidence-Based Practice, 2006). This website describes the research evidence for psychological treatments, which will necessarily be combined with clinician expertise and patient values and characteristics in determining optimum approaches to treatment.”

EBPP RESOURCES: WEB SITES Evidence-Based Mental Health Treatment for Children and Adolescents http://effectivechildtherapy.com/sccap/ “The information on this website is offered as a completely free service to families and mental health professionals to help ensure that children and adolescents benefit from the most up-to-date information about mental health treatment. We request absolutely no information from visitors to this site, and hope this service will help all learn more about important differences in mental health treatments. Families want their children to get the best possible treatment, and this site maintains an updated list of treatments with strong scientific support.”

HOW TO BE AN EVIDENCE-BASED PSYCHOLOGIST Accurately assess and diagnose clients, paying close attention to: Comorbidity Dominant client concerns Cultural factors Collaborate with clients to develop treatment goals, paying close attention to client expectations and barriers to treatment involvement Provide the most appropriate evidence-based treatment possible, flexibly tailoring to client circumstances

HOW TO BE AN EVIDENCE-BASED PSYCHOLOGIST Work to ensure a strong therapeutic relationship, and move quickly to repair any relationship difficulties that arise Monitor treatment progress, both in terms of symptoms/problems and the therapeutic relationship Regularly review treatment progress with clients, modifying treatment as needed If treatment progress is suboptimal, be prepared to provide a different EBT or refer to another professional who can offer a different EBT

HOW TO BE AN EVIDENCE-BASED PSYCHOLOGIST Questions or comments? For copies of the presentation, contact hunch@uottawa.ca