Presentation on theme: "None of the interventions are evidence-based but think positive – you get a new and untested intervention."— Presentation transcript:
None of the interventions are evidence-based but think positive – you get a new and untested intervention
IMS APPROACH TO GOOD PRACTICE Knut Sundell Jenny Rehnman Mari Forslund www.evidens.nu
Development & Evaluation (assessment tools and interventions) 20 employees 30 projects lasting 3-4 years Systematic reviews (assessment tools and interventions) 10 employees 15 projects lasting 1-2 years Additional external researchers Support & Dissemination (administrative support & dissemination of all that is produced within IMS) 10 employees 15 projects lasting less than 1 year IMS director Board of IMS (12 stake-holders) Director- general NBHW
Sweden is the third largest country in Western Europe with nine million inhabitants Social care service delivery involve agencies of the 290 municipalities, and 20 regional county councils. Median size of municipalities = 15.250 inhabitants National agencies and research councils, deals with research and development, supervision of service quality and safety, and the delivery of institutional care. The voluntary sector is considerable Social welfare states (e.g., low rate of unemployment, poverty, drug use, violence) General trust among Swedes in collective solutions and in government authorities Sweden
Research on interventions for women with experience of partner violence (Anttila et al, 2007)
Swedish outcome studies on social work Two articles in Sweden’s largest morning paper by the Director-General of the Swedish National Board of Health and Welfare claiming that social work agency managers did not have any idea whether social care services made any difference to clients’ lives. Important dates National action plan for preventing alcohol related harm & National action plan on drugs (lasting to 2007). 100 million SEK on research Training of local prevention workers IMS is inagurated General election and a new government The first national guidelines on social work (substance abuse treatment) Official reports of the Swedish government – “Evidence-based social work practice – favouring clients” Agreement between the government and the Swedish Association of Local Authorities and regions on implementing the national guidelines on substance abuse treatment An inquiry initiated at the NBHW on how to support EBP by National guidelines, research reviews National and regional knowledge transfer Developing criteria for local-follow up and open comparisons Initiative to clarify the concept of EBP among governmental authorities Addiction Severity Index is introduced in Sweden
PREVALENCE OF EVIDENCE-BASED METHODS (SOMETIMES) 24 per cent
PREVALENCE OF EVIDENCE-BASED METHODS (MOSTLY) 7 per cent
PREVALENCE OF EVIDENCE-BASED METHODS (SOMETIMES)
PREVALENCE OF STANDARDISED ASSESSMENT TOOLS (SOMETIMES) 63 per cent
14 per cent PREVALENCE OF STANDARDISED ASSESSMENT TOOLS (REGUAL)
Sweden: What are the political issues that need to be addressed? Keywords: Ethics – Evidence – Transparency not the technical issues concerning internal / external validity Knowledge transfer / dissemination of innovations need extensive support at a regional level Bring forward good examples
Stockholm evidence-based Clearinghouse for social work ”Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett et al., 1996) The general aim of “the Stockholm evidence-based Clearinghouse for social work” is to build a bridge between professionals and research on “what works” It was launched on 1st September, 2008 18
Stockholm evidence-based Clearinghouse for social work Web-based service (at IMS’ homepage): http://www.socialstyrelsen.se/Amnesord/socialt_arbete/IMS/Metodguiden_index.htm Provides information on interventions, assessment tools and general knowledge (e.g., mechanisms) Target group: Professionals, politicians and policy makers Not recommendations Arranged in a simple, straightforward format reducing the need to conduct literature searches Methodology of the systematic review ( e.g., Higgins & Green, 2008)
Topics Currently Available on the Website Target groups Child/family Social assistance (for welfare recipients) Disability Addiction Elderly Subheadings Interventions (18 descriptions/5 final evaluations) Assessment tools (46 descriptions/13 final evaluations) General knowledge IMS projects 20
The evaluation process Selection of interventions to evaluate – starts with discussion in IMS research council: Which interventions should we evaluate? Which outcomes are important? What databases should be searched? All relevant studies with a RCT or QE (including data at baseline) are reviewed Use of a guide/protocol to assist the review process Studies are evaluated for to their internal validity (i.e., can we trust the result?) Two independent reviewers evaluate each study. Any disagreements are settled based on consensus with help from a coordinator 22
23 Judging of internal validity Selection bias e.g., Are there any important differences between the groups? Performance bias e.g., Are researchers, participants and data collectors ”blind”? Attrition bias e.g., Any differences in size and type of attrition between the groups? Detection bias e.g., Are the outcome measures measured in the same way in all groups? Potential bias is evaluated within and across each domain of bias Low risk: plausible bias unlikely seriously alter the results Unclear risk: plausible bias that raises some doubt about the results High risk: plausible bias that seriously weakens confidence in the results (from Higgins & Green, 2008)
Grading of evidence The scientific raiting scale is a modified version of a scale developed at California Evidence-Based Clearinghouse for Child Welfare (http://www.cachildwelfareclearinghouse.org/scientific-rating/scale) Based on statements in Flay et al., 2005 (Society for Prevention Research, SPR) The scale is devided into five grades - A lower score indicates effectiveness and a higher level of research support Grading of evidence is based on: –Number of studies with a certain degree of internal validity (i.e., risk of bias) –If the practice is evaluated in usual care –If the effect are sustained over time –If the practice may cause harm –If the practice is replicable 24
1.Effective practice with well-supported research evidence At least two studies with low risk of bias, in different usual care or practice settings, have found the practice to be superior to treatment as usual (TAU). If multiple effectiveness studies have been conducted, the overall weight of the evidence supports the benefit of the practice. In at least one study with low risk of bias, the practice has shown to have a sustained effect at least one year beyond the end of treatment. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify components of the service and describes how to administer it.
2. Effective practice supported by research evidence At least one study with low risk of bias has found the practice to be superior to treatment as usual (TAU). If multiple outcome studies (at least with medium risk of bias) have been conducted, the overall weight of evidence supports the benefit of the practice. In at least one study with low risk of bias, the practice has shown to have a sustained effect of at least six months beyond the end of treatment. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specifies the components of the practice protocol and describes how to administer it.
3. Practice with promising research evidence At least one study with medium risk of bias has established the practice's benefit over no intervention (or placebo or waiting list) or is found to be comparable to or better than treatment as usual (TAU). If multiple effectiveness studies with at least medium risk of bias have been conducted, the overall weight of evidence supports the benefit of the practice. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.
4. Practice where the evidence fails to demonstrate effect At least two studies with low risk of bias have found that the practice has not resulted in improved outcomes compared to no intervention (e.g., placebo or waiting list), or that the practice is shown to be less effective when compared to treatment as usual. If multiple effectiveness studies have been conducted, the overall weight of evidence does not support the benefit of the practice. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.
5. Concerning practice At least one study of low or medium risk of bias shows that the intervention can cause serious harm, and/or there is a reasonable theoretical basis suggesting that the practice constitutes a risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.
Beyond the five grades the scientific scale includes an additional category (not rated - no number is given). Practice with unknown effect There is lack of studies with a medium or low risk of bias. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specifies the components of the practice protocol and describes how to administer it.
Conclusions and summary - Is there a sufficiently robust evidence base to identify good practice? What are the strength and weaknesses? Are there significant gaps that should be addressed? Is there an agreed approach to deciding what counts as evidence? Is there an agreed approach for judging the quality of the evidence base? Is there a scope for working internationally to strengthen the evidence-base?
IMS practical delivery mechanisms to promote the adoption of good practice
IMS want to reach: Social workers Politicians and decision makers Authorities Universities and students Researchers Clients and others who are interested
Target groups Social work agency managers Government officials
Parts of our communication strategy Value of direct, face-to-face communication Web-based services Tailored products Short courses for social work agency managers about EBP Start kit for municipalities for working with EBP
IMS STRATEGI ETICS INFORMATION TOOLS FOR CHANGE NOT COACHING SPECIFIC METHODS Target groups Social workers Politicians and decision makers Governments - - - - Universitys Goals Increase the interest for EBP Provide support for decision-making Implementation knowledge Support to managers Long-term partnership Activities Lectures Clearinghouse Readiness for change Handbook in leading EBP Pilot-project Train the trainers Ask a researcher Start-up-package IMS network for EBP
Organizational Readiness for Change Dwayne Simpson Anonymous survey to co-workers, managers and clients
Pilot-project Södermalm - Trying different mechanisms to promote EBP A steering group and 24 IMS-coodinatiors with special drive Seminars about EBP and implementation ORC-survey Focusing on the managers with seminars about leading EBP –the importance of the managers –management research –tools from the handbook Local seminars about how to find research and assessment tools
Training trainers in ASI Addiction Severity Index Big difference when local support in the new method is given and when the managers get’s support and owns the implementation 2 persons at IMS, 25 local trainers, 12 county administrative boards and 194 municipalities' Combination of IMS knowledge in implementation research and the method ASI and the local knowledge about the practice and support to the managers Success factors: –National support –Local competence –Managers in focus –Networking
Start-up- package Content: EBP – concepts and implications Evidence, ethics and affectivity Risk- and protective factors Assessment tools Research reviews Dissemination EBP and cost effectiveness Including: DVD-films The book Evidence based practice in social work The book To change social work PowerPoint material Glossary Information about IMS website Focused on Managers Study material as an introduction to EBP
Publications Collaboration with a publishing house (Gothia förlag) Adapted to our target group Reading guidances IMS-nytt New layout + more pages 4 numbers annually Special feature issues Research and interviews with professionals and clients
Challenges 1.Lack of interest among the Swedish social work academia to support an EBP, and train the future generations of social workers accordingly 2.Lack of (inter)national agreement on how to grade evidence 3.Questions on the transportability of evidence-based interventions between countries 4.Lack of reliable and valid measures of quality of services at a local level (in order to motivate change) 5.Lack of an infrastructure for diffusion of innovations, and knowledge transfer
AIDA – acronym used in marketing A Attention I Interest D Desire A Action