1. Evidence-Based Practice in Clinical Psychology: What It Is, Why It Matters, What You Need to Know Bonnie Spring, Ph.D., ABPP Northwestern University

2. Why it matters: EBBP Rationale improve quality and accountability for health care practice (IOM, 2001, Crossing the Quality Chasm) shared vocabulary and concepts for transdisciplinary, biopsychosocial research, practice, health care policy stimulate development of evidence base for behavioral treatments

3. Why it matters: Potentionally Useful Infrastructure Clinical Practice Guidelines: Clinical Practice Guidelines: Increasingly based on ongoing systematic review of research (esp. RCTs) (e.g., USPTF, Cochrane, CDC/AHRQ) Research reporting guidelines (CONSORT, TREND, QUOROM) Evidence grading & knowledge synthesis systems (e.g., GRADE, AHRQ) Policy, often coverage/reimbursement implications (VA/DOD, CMS, NICE) (P4P?) Evidence-Based Practice: Evidence-Based Practice: (life-long learning) Question formulation, search strategies, critical appraisal SUMSEARCH Clinical Evidence, First Consult, BMJ updates, Best Evidence Topics, CATCRAWLER, CATBANK – clinical scenario & bottom line

4. Overview History of evidence-based practice (EBP) History of evidence-based practice (EBP) Core elements of EBP Core elements of EBP EBP pedagogy in psychology EBP pedagogy in psychology EBP pedagogy in other health disciplines EBP pedagogy in other health disciplines Useful infrastructure and potential opportunities for synergy Useful infrastructure and potential opportunities for synergy

5. Origins of Evidence-Based Practice

6. Emergence of Evidence- Based Medicine 1910 - Flexner report :155(31!) 96 (1915) 76(1930) 1972 - Archie Cochrane – epidemiology, health services research - Effectiveness and Efficiency: Random Reflections on Health Services 1973 – John Wennberg – widespread practice variation 1982 - clinical epidemiology determinants and consequences of health care decisions (McMaster U – David Sackett, Gordon Guyatt) 1985 – IOM: 15% medical practices evidence-based [2001 Crossing the Quality Chasm] 1990 - Evidence-based medicine, Brian Haynes & Ann McKibbon – search strategies 1992-3 -Cochrane Collaboration 2000 - How to Practice and Teach EBM 2000 - Sackett - How to Practice and Teach EBM

7. What do we mean by “evidence-based practice?”

8. Alternative Definitions of Evidence-Based Practice Guidelines: public health, medicine) – Guidelines: (public health, medicine) – focus on problem/disorder & level of evidence for practices (based on systematic review) (e.g., NICE, VA, apa) ESTs: (psychology)ESTs: (psychology) focus on intervention (& disorder) EBP: EBP: (psychology, medicine, nursing, social work) focus on decision-making about individual patients Idiographic……….. Lifelong Learning Nomothetic

9. APA Policy Statement adopted August 2005 “Evidence-based practice in psychology is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” “Evidence-based practice in psychology is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” -adapted from IOM, 2001 & Sackett, 2000

10. Best available research evidence Patient’s values, characteristics, and circumstances Clinical Expertise Clinical Decision-Making

11. Syllabus Project Prompt: Does anyone on the list teach a course on evidence-based practice (EBP)? Specifically, I am searching for syllabi that cover one or more "legs" of the three- legged EBP stool: a) research evidence, b) clinical expertise, c) patient values, preferences, characteristics. Prompt: Does anyone on the list teach a course on evidence-based practice (EBP)? Specifically, I am searching for syllabi that cover one or more "legs" of the three- legged EBP stool: a) research evidence, b) clinical expertise, c) patient values, preferences, characteristics. November, 2006

12. Listservs Sampled ABCT ABCT APA Division 12 APA Division 12 SSCPNET (Section III, Div 12) SSCPNET (Section III, Div 12) CUDCP CUDCP APA Division 38 APA Division 38 ABMR ABMR SBM EBBM, MRBC, Obesity, CA SIGs SBM EBBM, MRBC, Obesity, CA SIGs

13. Outcome 39 syllabi 39 syllabi 17 additional recommended articles and books 17 additional recommended articles and books 273 page document 273 page document Discipline: Discipline: 30 psychology3 public health 3 medicine 1 nursing 3 medicine 1 nursing 1 PE/health/sport studies 1 PE/health/sport studies 140 requests 140 requests November, 2006

14. Evidence-Based Practice Modal Course Title: CBT, EST, EVT, Psychological Interventions, Psychotherapy Research Course Title: CBT, EST, EVT, Psychological Interventions, Psychotherapy Research Texts: Barlow, Handbook Psychologic Disorders, Bergen & Garfield Handbook of Psychotherapy and Behavior Change Texts: Barlow, Handbook Psychologic Disorders, Bergen & Garfield Handbook of Psychotherapy and Behavior Change Content: ESTs Content: ESTsAdditional Additional Texts: Additional Texts: -Persons, Case Conceptualization -Dawes, House of Cards Additional Content: Additional Content:-Assessment -Case formulation, functional analysis -Clinical judgment -Diversity -Iatrogenic effects -Research methods

17. courtesy of Barbara Walker, Indiana University, 2006

18. Best available research evidence Patient’s values, characteristics, and circumstances Clinical Expertise Clinical Decision-Making Researcher Design Conduct Analysis Reporting Synthesizer Locate Critically appraise Meta-analysis Consumer Locate Appraise quality & relevance Integrate Clinician Communicate Assess patient Deliver EBP Patient Understanding Preferences Access

19. Researcher Training in Psychology versus Medicine Psychology Psychology DesignDesign Correlational (convenience classes) Correlational (convenience classes) Experimental (from animal studies) Experimental (from animal studies) ConductConduct Brief, tight control Brief, tight control Little missing data; replace cases Little missing data; replace cases Analysis - completerAnalysis - completer ReportingReporting Clinical Medicine Clinical Medicine Design Observational (population) Clinical Trial –test of policy applied to population Conduct Long, intercurrent events Missing data; Analysis – ITT Reporting – CONSORT

20. Researcher, Synthesizer, Consumer Training in Analysis Psychology Psychology ANOVA/regressionANOVA/regression Clinical Medicine Clinical Medicine Odds Ratios Epidemiology Terminology Absolute risk (p[disease] in a particular population) Relative risk (p[disease/exposed]/p[disease/unexposed) Attributable risk (p[disease/exposed] -p[disease/unexposed) Number needed to harm (1/attributable risk) Odds ratio (odds[disease/exposed]/odds[disease/unexposed])

21. Clinical Significance NNH = 5. If 5 patients treated with TX1, 1 would be more likely to have AE than if all had received TX0 NNH = 5. If 5 patients treated with TX1, 1 would be more likely to have AE than if all had received TX0 NNT = 13. 13 patients would need to be treated with TX1 to see one success not seen with TX0 NNT = 13. 13 patients would need to be treated with TX1 to see one success not seen with TX0

22. Reporting: Consort Flow Diagram Consolidated Standards of Reporting Trials (CONSORT) www.consort- statement.org www.consort- statement.org

23. Excerpt from CONSORT checklist METHODS Participants 3Eligibility criteria for participantsEligibility criteria for participants and the settings and locations where the data were collected.settings and locations where the data were collected Interventions4Precise details of the interventions intended for each group and how and when they were actually administered. Objectives5Specific objectives and hypothesesSpecific objectives and hypotheses. Outcomes 6Clearly defined primary and secondary outcome measuresClearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors).methods used to enhance the quality of measurements Sample size 7How sample size was determinedHow sample size was determined and, when applicable, explanation of any interim analyses and stopping rules.explanation of any interim analyses and stopping rules Randomization -- Sequence generation 8Method used to generate the random allocation sequence, including details of any restrictionsMethod used to generate the random allocation sequence, including details of any restrictions (e.g., blocking, stratification) Randomization -- Allocation concealment 9Method used to implement the random allocation sequenceMethod used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned. Randomization -- Implementation 10Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups. Blinding (masking)11Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment.Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment. When relevant, how the success of blinding was evaluated.how the success of blinding was evaluated

24. Evidence Synthesizer and Consumer Skills

25. Best available research evidence Patient’s values, characteristics, and circumstances Clinical Expertise Clinical Decision-Making Researcher Design Conduct Analysis Reporting Synthesizer Locate Critically appraise Meta-analysis Evidence User Locate Appraise quality & relevance Integrate Clinician Communicate Assess patient Deliver EBP Patient Understanding Preferences Access

26. Synthesizer: Systematic Reviewer- Synthesizer: Systematic Reviewer- explicit, systematic, transparent to avoid bias  Specific research question (PICO)  Search protocol to select papers – key words systematic search of the literature (EMBASE, CINAHL, Cochrane Controlled Trial register, DARE) systematic search of the literature (EMBASE, CINAHL, Cochrane Controlled Trial register, DARE) explicit inclusion and exclusion criteria explicit inclusion and exclusion criteria  Explicit, transparent rating of methodological quality  Data extraction  Analysis: qualitative or quantitative  Conclusion  Discussion of strengths and limitations

27. The 5 Step EBM Model for Evidence Users (Consumers) formulate the question Ask: formulate the question Acquire: evidence - search for answers Acquire: evidence - search for answers the evidence for quality and relevance Appraise: the evidence for quality and relevance the results Apply the results the outcome Assess the outcome

28. Asking: Well-Built Clinical Questions Background: What are effective treatments for bulimia nervosa? Background: What are effective treatments for bulimia nervosa? Foreground: In patients with Foreground: In patients with P: binge eating disorder Patient: binge eating disorder Intervention: does interpersonal therapy Comparison: compared to CBT reduce Outcome: frequency of binge episodes

29. Critically appraising the evidence Use of standardized a priori appraisal methods to answer: Is the evidence valid? Is the evidence valid? Internal validityInternal validity Is the evidence applicable/relevant? Is the evidence applicable/relevant? External validityExternal validity Is the evidence clinically significant? Is the evidence clinically significant?

30. Clinical Decision-Making Clinical epidemiology discipline Clinical epidemiology discipline study of determinants and consequences of clinical decisionsstudy of determinants and consequences of clinical decisions apply EBP/5A’s/critical appraisal at clinical encounter to overcome automatic, unconscious decision-making biases (aka bad clinical intuition)apply EBP/5A’s/critical appraisal at clinical encounter to overcome automatic, unconscious decision-making biases (aka bad clinical intuition)

31. barriers between research and practice 30 kg of guidelines per family doctor per year 25000 biomedical journals in print 8000 articles published per day 95% of studies cannot reliably guide clinical decisions  2001 Bazian Ltd

32. Clinical Decision-Making Health Informatics discipline Health Informatics discipline infrastructure, resources, devices, structures (e.g., algorithms, guidelines) needed to store, retrieve, manage and use health information and the time and place that a decision needs to be made. -Decision support.

33. Secondary Synthesized Evidence (AKA “evidence-based capitulation”) Research proliferates rapidly. Clinical performance demands increase. Practicing clinicians too busy to use all EBM steps will all patients. Research proliferates rapidly. Clinical performance demands increase. Practicing clinicians too busy to use all EBM steps will all patients. Increased focus on pithy clinical practice guidelines, synopses, and structured abstracts Increased focus on pithy clinical practice guidelines, synopses, and structured abstracts MD ConsultMD Consult ACP Journal ClubACP Journal Club Cochrane Database of Systematic ReviewsCochrane Database of Systematic Reviews “Up-to-date”“Up-to-date” InfoPOEMS (Patient Oriented Evidence that Matters)InfoPOEMS (Patient Oriented Evidence that Matters)

34. Best available research evidence Patient’s values, characteristics, and circumstances Clinical Expertise Clinical Decision-Making Researcher Design Conduct Analysis Reporting Synthesizer Locate Critically appraise Meta-analysis Consumer Locate Appraise quality & relevance Integrate Clinician Communicate Assess patient Deliver EBP Patient Understanding Preferences Access

35. Clinically Supervised Training in Evidence-Based Treatment Needs work: 2005-2006 papers by Woody and by Weissman Needs work: 2005-2006 papers by Woody and by Weissman

36. Best available research evidence Patient’s values, characteristics, and circumstances Clinical Expertise Clinical Decision-Making Researcher Design Conduct Analysis Reporting Synthesizer Locate Critically appraise Meta-analysis Consumer Locate Appraise quality & relevance Integrate Clinician Communicate Assess patient Deliver EBP Patient Understanding Preferences Access

37. Patient Preferences requires information only available to patient (e.g., valuation of harms/hassles, alternative outcomes & treatments) Shared decision-making requires information only available to patient (e.g., valuation of harms/hassles, alternative outcomes & treatments) : All possible outcomes assigned a value between 0 (death) and 1 (perfect health). Utility assessment: All possible outcomes assigned a value between 0 (death) and 1 (perfect health). Time trade-off approach Time trade-off approach The proportion of life in a particular health state (e.g., severe depression) that you would give up to attain perfect health (e.g., 30%). Utility of that health state is 1-(30%) =.70The proportion of life in a particular health state (e.g., severe depression) that you would give up to attain perfect health (e.g., 30%). Utility of that health state is 1-(30%) =.70 Standard gamble approach Standard gamble approach The point where you are indifferent to the choice between spending the rest of your life in the health state in question and a gamble between perfect health and instant death where the probability of perfect health represents the utility of the health state.The point where you are indifferent to the choice between spending the rest of your life in the health state in question and a gamble between perfect health and instant death where the probability of perfect health represents the utility of the health state.

38. Teaching evidence-based practice = teaching a process Didactics Didactics Small groups, problem-based learning Small groups, problem-based learning Preceptorships/clinical supervision Preceptorships/clinical supervision Standardized patients and evidence stations Standardized patients and evidence stations Embedded throughout curriculum Embedded throughout curriculum

39. Medical Decision Making in the NU-FSM curriculum MDM-I (first week of medical school) MDM-I (first week of medical school) Sensitivity, specificity, pre- and post-test probabilities, innumeracy, uncertainty in medicineSensitivity, specificity, pre- and post-test probabilities, innumeracy, uncertainty in medicine MDM-II (last two weeks of M1 year) MDM-II (last two weeks of M1 year) EpidemiologyEpidemiology StatisticsStatistics MDM-III (beginning of M2 Spring Quarter) MDM-III (beginning of M2 Spring Quarter) Decision analysisDecision analysis Meta-analysisMeta-analysis Cost-effectiveness analysisCost-effectiveness analysis Clinical guidelinesClinical guidelines M3 MDM (once a month in M3 year) M3 MDM (once a month in M3 year) Review papers pertaining to clinical casesReview papers pertaining to clinical cases Use of CATUse of CAT

40. Evidence-Based Behavioral Practice (EBBP) NIH Office of Behavioral and Social Sciences Research contract N01-LM-6-3512: Resources for Training in Evidence- Based Behavioral Practice, 2006 - 2011

41. OBSSR 5-Year Plan Year 1: develop training website, Council, Scientific Advisory Board, white paper on training, skills, competencies reflecting education in evidence-based behavioral practice (EBBP) Year 1: develop training website, Council, Scientific Advisory Board, white paper on training, skills, competencies reflecting education in evidence-based behavioral practice (EBBP) Year 2: d evelop, implement a web-based, research-focused training module(s) on EBBP; field test in graduate curricula Year 2: d evelop, implement a web-based, research-focused training module(s) on EBBP; field test in graduate curricula Year 3: l aunch interactive web-based training courses; establish practice network, develop first EBBP clinical practice training module Year 3: l aunch interactive web-based training courses; establish practice network, develop first EBBP clinical practice training module

42. OBSSR 5-Year Plan Year 4: With practice network, develop modules on application of evidence-based clinical decision- making to intervention with specific cases. Field test in internship/residency/post-doctoral training programs and practice network. Year 4: With practice network, develop modules on application of evidence-based clinical decision- making to intervention with specific cases. Field test in internship/residency/post-doctoral training programs and practice network. Year 5: Link website to systematic reviews of behavioral interventions, treatment manuals, outcome assessments. Develop and field test clinical decision-making modules that integrate patient preference and clinical competency assessments. Year 5: Link website to systematic reviews of behavioral interventions, treatment manuals, outcome assessments. Develop and field test clinical decision-making modules that integrate patient preference and clinical competency assessments.

43. Suggestions To enhance the evidence base for psychological treatments and support lifelong learning, clinical psychology training might benefit from enhanced coverage of: To enhance the evidence base for psychological treatments and support lifelong learning, clinical psychology training might benefit from enhanced coverage of: Researcher skills in methods: clinical trial design, analysis, reporting, synthesisResearcher skills in methods: clinical trial design, analysis, reporting, synthesis Clinician training in 5-step (5A’s) EBP model – cover 2 A’sClinician training in 5-step (5A’s) EBP model – cover 2 A’s

44. Suggestions Psychology informatics could use infrastructure development (PSYCinfo & Cochrane; ; coverage in secondary synthesized sources like Up-to-Date; practice-based research networks)Psychology informatics could use infrastructure development (PSYCinfo & Cochrane; library access; coverage in secondary synthesized sources like Up-to-Date; practice-based research networks) Psychology could use appropriate patient preference measures that support shared decision-makingPsychology could use appropriate patient preference measures that support shared decision-making A discipline of clinical psychology decision-making needs to develop to systematize integration of research evidence, clinical expertise, and patient clinical data and preferencesA discipline of clinical psychology decision-making needs to develop to systematize integration of research evidence, clinical expertise, and patient clinical data and preferences

45. Concluding Questions What training modules and materials would be helpful? What training modules and materials would be helpful? Will you partner with us to help develop and try these out? Will you partner with us to help develop and try these out?

48. The Evidence Pyramid for Treatment Effectiveness Questions ***USE THE BEST EVIDENCE AVAILABLE***

49. Alternatives to evidence-based medicine Eminence based medicine Eminence based medicine Eloquence based medicine Eloquence based medicine Vehemence based medicine Vehemence based medicine Nervousness based medicine Nervousness based medicine (Isaacs and Fitzgerald, 1999, BMJ)

50. Levels of Clinical Evidence in the Primary Literature (psycINFO, MEDLINE) RCT, Practice Guideline Consensus Development Conference Randomized Controlled Trial Quality Improvement Cohort Studies, Risk Cohort Studies Etiology Cohort Studies, Prognosis, Survival Analysis Cohort Studies, Case Control, Case Series Prognosis Randomized Controlled Trial, Double Blind, Clinical Trials Double-Blind Randomized Controlled Trial Therapy Search Filters Methodology Type of Question

51. EBM Resources Pocket guides with web-linked updates (Sackett; Guyatt & Rennie) Pocket guides with web-linked updates (Sackett; Guyatt & Rennie) Cochrane Library Cochrane Library BMJ: www.clinicalevidence.com BMJ: www.clinicalevidence.comwww.clinicalevidence.com Centre for EBM: http://minerva.minervation.com Centre for EBM: http://minerva.minervation.com http://minerva.minervation.com Centre for Evidence-based mental health: www.cebmh.com Centre for Evidence-based mental health: www.cebmh.comwww.cebmh.com

52. Evidence Pyramid

53. Evidence Pyramid - OVID

54. Evidence Pyramid - SUMsearch