The Generation and Use of Evidence in Child Welfare Aron Shlonsky Visiting Scholar, Haruv Institute Associate Professor Factor-Inwentash Chair in Child Welfare University of Toronto Factor-Inwentash Faculty of Social Work

Evidence-Informed Social Work Is a Concept Awaiting Implementation Much discussed Required in most jurisdictions Not clearly Defined Not yet implemented in pure form “--- the conscientious, explicit and judicious use of current best evidence in making decisions regarding the welfare of service-users and carers” (Sheldon, 2003, p. 1).

The Need for EBP in Social Work Overwhelming and unreliable sources of information Changing state of knowledge Potential to cause harm Opportunity costs Social justice / informed choice Systematic rubric

What is EIP?

EBP, EBM, EIP, EST’s, EBP’s, PG’s, and BP’s – huh? Evidence-based Practice (EBP) is based on Evidence-based Medicine (EBM) Sackett et al (1997) at McMaster Evidence-informed practice –Describes EBP and EBM Empirically Supported Treatments (EST’s or EBP’s) – Use of interventions with evidence of their efficacy and/or effectiveness Practice guidelines (PG’s) and Best Practices (BP’s) are lists of (hopefully) validated interventions

Clinical state and circumstances Client Preferences and actions Current Best Evidence Clinical Expertise Haynes, Devereaux, and Guyatt (2002) The EBP Model: A Philosophical and Practical Approach

Steps of EBP (based on Sackett et al., 2000) Become Motivated to Apply EBP 1.Convert information need (prevention, assessment, treatment, risk) into an answerable question. 2.Track down the best evidence to answer the question. 3.Critically appraise the evidence for its validity (closeness to the truth) impact (size of the effect) and applicability (usefulness in our practice). 4.Integrate critical appraisal with our practice experience, client’s strengths, values circumstances. 5.Evaluate effectiveness and efficiency in exercising steps 1- 4 and seek ways to improve them next time. 6.Teach others to follow the same process Gibbs, 2003

Challenges to EBP Philosophical straw men Lack of evidence Access to evidence Complexity of social services Lack of time Translation across different contexts Decision-making in an uncertain world Cognitive dissonance No manual for integrating evidence with context

Organizational Mandate Organizational Resources / Constraints Community Training / Supervision Political Context Socio / Historical Context Economic Context Professional Context and Client Preferences and Actions Research Evidence Expertise and Client State and Circumstances Client Preferences Client Preferences Research Evidence Research Evidence Expertise Professional Regehr, Stern & Shlonsky (2007)

Evidence in NOT Deterministic Evidence-informed Systematic process InclusiveCriticalFlexible D NO DETERMINISM

Tensions Within the North American Child Welfare System Child welfare or child protection? Child welfare worker - friend or foe? System - should it be residual or preventive? How do you balance safety, culture, community, family rights, and the individual needs of children? How do you assess the risk the system poses to the child?

Decision Points in Child Welfare Report / Screening investigateInvestigationVerifyclose Open for ongoing Ongoing Services (in-home) Close Case Out-of-home care Ongoing Services (out-of-home) ReunificationLTFCGuardianshipAdoption Risk / Prognostic Tools Diagnostic Tools Effective Treatment Monitoring Evaluation

Proper blend: Clinical Skills and Evidence Requires a structure that teaches clinical skills and constantly creates and evaluates evidence. Ongoing struggle Responsibility to both find and evaluate evidence Find ways to integrate No set recipe

Types of Evidence in CW Clinical Experience Research literature Administrative Data SurveyEvaluation –Quantitative –Qualitative

The Cycle of EBP Clinical state and circumstances Client Preferences and actions Current Best Evidence Clinical Expertise Actuarial risk assessment Contextual assessment Other valid assessment measures? Effective services Appropriate for this client? Barriers (e.g., cultural conflict) Client’s preferred course or at least willing to try? Adapated from Haynes, Devereaux, and Guyatt, 2002; as presented in Shlonsky and Wagner, 2005.

Campbell Collaboration Evolution Systematic Reviews of Evidence Importance of question Importance of non-RCT’s –Econometric –Observational –Risk/diagnosis Increasing number of reviews Equity Group User Abstracts –Translation

California Clearinghouse Scientific Ratings of 1 Cognitive Therapy Coping Cat Coping Power Coping with Depression for Adolescents Interpersonal Psychotherapy (Adults & Children of Depressed Adults) Mindfulness Based Cognitive Therapy (Adults) Motivational Interviewing (MI) Multidimensional Treatment Foster Care - Adolescents (MTFC-A) Multisystemic Therapy?? Nurse-Family Partnership (NFP) Parent-Child Interaction Therapy (PCIT) The Incredible Years Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) Triple P-Positive Parenting Program Slide courtesy of Richard P. Barth

Critical Thinking is Key Eight Critical Thinking Guidelines Ask Questions: Be willing to wonder. Define Your Terms: Key to Research Examine the Evidence Analyze Assumptions and Biases Avoid Emotional Reasoning Don’t Oversimplify Consider Other Interpretations Tolerate Uncertainty

A Well Cultivated Critical Thinker Raises vital questions and problems; Formulates them clearly and precisely; Gathers and assesses relevant information Uses abstract ideas to interpret information; Comes to well-reasoned conclusions and solutions; Tests them against relevant criteria and standards; Thinks openmindedly; Recognizes and assesses assumptions, implications, and practical consequences; Communicates effectively Elder, 2007

Knowledge needs to be managed much more effectively than it has been in the past The challenge is to rethink our view of what knowledge is & how best to facilitate its rapid generation, sharing, and application in a manner that closes the policy- research-practice gap

What to do? Teach process of EBP Fund systematic reviews Systematic reviews lead to targeted primary studies Create expertise in EST’s and train to common factors Learn how to effectively work in teams Implement and evaluate KTE strategies Develop and sustain a culture of critical thinking

Discussion