1. What does this mean for you?
3/28/2017
Meeting the Challenges in Implementing Evidence-Based Practices and Treatments in Ontario
What does this mean for you?

2. Presentation Objectives
3/28/2017
Presentation Objectives
Introduction to Evidence-Based Practice
What is it?
Why use it?
What is needed?
Evidence to practice process
Role of the Centre of Excellence in the uptake of Evidence-Based Practice

3. 3/28/2017
Truth or Dare …
If you’ve been doing something for twenty years and we tell you that what you’re doing is not effective – or may even cause harm – would you change your practice?
WHY or why not?
What if we can show you strong evidence?
What kind of Evidence would convince you?
Would you then?
Those who would not change?
What’s preventing you?

4. Critical Decision-Making:
3/28/2017
Critical Decision-Making:
How do you make decisions in your practice?
Think about a recent client (child, young person, family) you’ve worked with where you made a significant decision about their care/support.
What criteria did you use to make this decision?
Your intuition (gut feel) about what will work for them.
What you have heard from other professionals in informal exchanges.
Your previous experience with similar cases.
Data about outcomes from alternative care / support options.
What is usually offered by your service / agency.
Feedback from other similar clients about what is helpful.
What you are most familiar / comfortable with.
What you know about cases like this from reading research / theory.
What you learned to do in your own training (no matter how long ago)
© 2005, Research in Practice: Teamwise: Using Research Evidence

5. Critical Decision-Making:
3/28/2017
Critical Decision-Making:
How do you make decisions in your practice?
NOW… imagine you have a potentially serious medical problem and you seek help from a doctor to recommend a treatment.
What criteria would you like the doctor to use to make this decision?
His/her intuition (gut feel) about what will work for you.
What he/she has heard from other professionals in informal exchanges.
His/her previous experience with similar cases.
Data about outcomes from alternative treatment options.
What is usually offered by the surgery / hospital / clinic.
Feedback from other similar patients about what is helpful.
What he/she is most familiar / comfortable with.
What he/she knows about cases like this from reading research / theory.
What he/she learned to do in his/her training (no matter how long ago).
© 2005, Research in Practice: Teamwise: Using Research Evidence

6. How do we *know that what we do works?
3/28/2017
Evidence-Based Practice
(*not just believe)
How do we *know that what we do works?
Really!
© 2005, Research in Practice: Teamwise: Using Research Evidence

7. …Do we have the evidence to back our beliefs?
3/28/2017
…Do we have the evidence to back our beliefs?

8. Evidence-Based Practice It’s not simple, either…
3/28/2017
Evidence-Based Practice It’s not simple, either…
What is evidence?
Would you recognize evidence if it was right in front of you?
What exactly does “evidence” mean to you?
How can you possibly know when you are using it?

9. 3/28/2017
You are not alone…
At the Centre, we are also grappling with defining models of construct relating to evidence-based practice. There is no universal definition when it comes to evidence as it relates to child & youth mental health. What do we mean by…
Effective
Model
Evidence- informed
Promising
Evidence- based
Evidence
Emerging

10. 3/28/2017
Every child and youth has the right to demand that anyone involved in practice decisions knows what is most likely to work for them, their families and their caregivers.
“That’s the best I can do. If you’d like to see another child psychologist …”

11. Limitations of Evidence-Based Practice
3/28/2017
Limitations of Evidence-Based Practice
Evidence does not always exist
Evidence derived from a lab in one place does not necessarily apply elsewhere
Many promising practices have never had the opportunity to be evaluated in such a way as to elevate them to EBP
Evidence can be developed/tested on homogeneous samples that do not reflect real life
An effective intervention may not be feasible/appropriate in certain communities and with certain populations

12. Culturally Sensitive Approaches
3/28/2017
Culturally Sensitive Approaches
How best do you approach the treatment of clients whose characteristics and problems may differ from those of samples studied in research?
(Especially those from minority or marginalized populations.)
gender  gender identity  ethnicity  race  social class  disability status  sexual orientation  religion  disability status  rural  urban  group association (i.e. deaf culture)  family context 

13. Definition of Evidence: CHSRF
3/28/2017
Definition of Evidence: CHSRF
Evidence is information that comes closest to the facts of a matter. The form it takes depends on context. The findings of high-quality, methodologically appropriate research are the most accurate evidence. Because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to or stand-ins for research. The evidence base for a decision is the multiple forms of evidence combined to balance rigour with expedience—while privileging the former over the latter.
WEIGHING UP THE EVIDENCE
Making evidence-informed guidance accurate,
achievable, and acceptable
A summary of the workshop held on September 29, 2005
JANUARY 2006

14. Evidence Based Programs U.S. Perspective
3/28/2017
Evidence Based Programs U.S. Perspective
SAMHSA defines evidence-based programs according to three categories

15. What is Evidence-Based Practice? U.K. Perspective
3/28/2017
What is Evidence-Based Practice? U.K. Perspective
In these models:
Awareness of research findings alone is not enough
Research evidence complements – not replaces – the many other forms of knowledge
It’s not a magic bullet
And not simply about searching for the ‘right’ answer
It’s about understanding options available and, with children and youth or their caregivers, choosing the course of action most likely to lead to a good outcome
Sources: © 2003, Evidence Network: What Works for Children?
© 2005, Research in Practice: Teamwise: Using Research Evidence

16. Most Common Evidence-Based Practices?*
3/28/2017
Most Common Evidence-Based Practices?*
According to Children’s Mental Health Ontario
Cognitive behaviour therapy (65%)
COPE (42.7%)
Wraparound (42.5%)
Behavioural parent training (41.2%)
Brief strategic family therapy (39.2%)
Narrative therapy (38.8%)
“The Incredible Years” (36.4%)
Multisystemic therapy (35.9%)
“Stop Now and Plan” (32.4%)
“Right from the Start” (29.3%)
What has been your motivation for using these interventions or therapies?
* Defined by CMHO as “a treatment that has been developed through research, is supported by the results of controlled treatment studies, and has guidelines and procedures related to its implementation”

17. Beyond the Evidence What else is Needed?
3/28/2017
Beyond the Evidence What else is Needed?
Practitioners need to be willing and able to challenge themselves and others
Organizations need to provide professional development opportunities and validate the need to read, reflect and debate
Leaders need to promote critical thinking, improve access to research, identify and spread good practice across their agencies
Teams should be able to tackle real practice dilemmas collectively
© 2005, Research in Practice: Teamwise: Using Research Evidence

18. individual, organizational, systemic
3/28/2017
Breakout Evidence-based Practice: Why or Why Not?
In groups discuss the pro’s and cons of integrating Evidence-Based Practices into your practice:
If you are using an EBP in your organization name it
If you are using EBP or not
What are the barriers?
What are the facilitators?
What do you think are the key elements in creating a culture that encourages and supports Evidence-Based Practice?
Consider 3 levels:
individual, organizational, systemic

19. Barriers to Evidence-Based Practice
3/28/2017
Barriers to Evidence-Based Practice
Limited access to relevant information/models
Evidence does not fit for my community
Research done on pure samples
Research done on different samples
Intervention as researched is not feasible in my community
Innovation does not equal evidence-based practice

20. Barriers to Evidence-Based Practice
3/28/2017
Barriers to Evidence-Based Practice
Culture may not support this approach
Change is difficult at the best of times
Often viewed as an add-on that takes time away from direct provision of service
Resources/expertise is often lacking
Access to training and ongoing support
What are we missing?

21. Interviewed 12 experts in implementation science
3/28/2017
Surveyed leaders and practitioners in 80 associated Child and Youth Mental Health Clinics across Ontario about utilization of research-based information and readiness for change
Interviewed 12 experts in implementation science
Made recommendations for system of care improvements
© 2005, Children’s Mental Health Ontario

22. 3/28/2017
Conclusion:
To implement, transfer, deploy evidence-based best practices to the field, we need:
Resources
Leadership
Training
Practice Change Readiness Multiple methods of support Culture of professional development
© 2005, Children’s Mental Health Ontario

23. What does this look like for you?
3/28/2017
Literature Review Highlights:
Active strategies, collaboration, leadership are important
Sharing tacit knowledge face-to-face is powerful
Resistance to change – from the system, leaders, and practitioners – needs to be recognized and addressed
Change is complex and requires buy-in from a critical mass – create a tipping point!
Distillation of research knowledge into practice guidelines is insufficient to create practice change
Implementation requires a collaborative effort
Strategize on multiple levels: practice, organization, system, nature of evidence, the support plan
What does this look like for you?
© 2005, Children’s Mental Health Ontario

24. Take It One Step at a time…
3/28/2017
Take It One Step at a time…
Formulate an answerable question
Identify search terms for each component of the question
Appraise the evidence
Apply the evidence
Evaluate your practice
© 2003, Evidence Network: What Works for Children?

25. The Evidence to Practice Process Evaluate your practice
Start With a Plan…
3/28/2017
What is the problem we need to solve?
What is the question we need answered?
What might be the role of the Centre or others in helping you do this?
Select first choice resource where an answer may be found (e.g. the Cochrane library)
Select second choice resource
Design search strategy and carry out search
Appraise and summarize the evidence
Good Evidence
Poor or insufficient evidence
Assess the relevance of the evidence to your local situation and target groups
The Evidence to Practice Process
Consider implications for your practice and resources, and those of the organizations you work with
Improvement
Apply the evidence to practice
Evaluate your practice
No improvement
© 2003, Evidence Network: What Works for Children?

26. Evidence Source Checklist
3/28/2017
Evidence Source Checklist
There’s no simple way to gather evidence:
Look to credible sources
What are peers saying?
Look for reliable methods
large samples have power to generalize results
control groups – where possible and ethical
Where has this been published?
Is there a consensus?
Look for results over time
Is the entire process transparent and re-producible?
Is there a meta-analysis or systematic review?
Is there open discussion of results and gaps?

27. Evaluating What You Find
3/28/2017
Evaluating What You Find
Once you’ve gathered evidence you need to evaluate it:
Can it be implemented?
Was random assignment applied?
Is it based on effective principles?
What was the longitudinal impact?
Was there multiple site replication?
Has dosage analysis been conducted?
Were comparison groups used?
What about meta-analysis, expert review and consensus?
Were clients satisfied?
Did the program change something?
D. Andrews, C. Buettner. Evidence-based Practice: Evaluating Supporting Evidence

28. Evaluating Feasibility: Can it be implemented?
3/28/2017
Evaluating Feasibility: Can it be implemented?
Is a particular intervention or therapy a good fit for your organization based on:
Availability
Affordability
Feasibility
D. Andrews, C. Buettner. Evidence-based Practice: Evaluating Supporting Evidence

29. How Can We Best Meet Your Needs?
3/28/2017
What you told us:
Research and Development
Relevant, evidence-based information, geared to a variety of audiences
Best/promising practices
Practical information (e.g., about funding sources)
Funding creative, relevant research and building research capacity
“Facilitate knowledge dissemination”
Access to information and research was the number one need identified by regional conference participants.
When it came to what’s needed in research and development, groups told us they want relevant, evidence-based information that can be understood by a variety of audiences. In addition, they want to be kept abreast of promising practices that may not have yet been clinically proven. They want practical information about where and how they can secure funding for research, and help in developing research capacity.

30. EBP at the Centre of Excellence
3/28/2017
EBP at the Centre of Excellence

31. How Can We Best Meet Your Needs?
3/28/2017
What you told us:
Consulting services (Intervention)
Connecting people with experts
Consulting on projects
Advising on program development
Providing training
Helping “natural leaders” develop to their full potential
Consulting and mentoring was the second most common need identified by regional conference participants.
“Provide leadership”
Through our consulting services they want expert advice on projects, program development and training – and support for natural leaders to help them develop to their full potential.

32. How Can We Best Meet Your Needs?
3/28/2017
What you told us:
Partnerships and Networks
Facilitate and enhance existing and new partnerships and networks
Create opportunities to connect and network
Over one half of regional conference participants identified the need to develop linking, networking and partnerships.
“Play a proactive role in partnership building”
They want the centre to play a proactive role in partnership building by facilitating and enhancing existing and new partnerships and networks and by creating opportunities for agencies and individuals involved in child and youth mental health to get together.

33. Researchers Directory
3/28/2017
*Datalife…Finding that perfect partner!
Researchers Directory
Launched October 19, 2005
Open to CYMH researchers in Ontario
Must have, or be working on, a PhD
Over 100 entries
Service Provider Directory
Launched February 20, 2006
Open to CYMH service providers in Ontario
Over 145 entries
* With apologies to LavaLife!

34. How Can We Best Meet Your Needs?
3/28/2017
What you told us:
Education and Training
Best practices
More CYMH in more university programs
More training for frontline CYMH workers
More CYMH training in other sectors (e.g., justice)
More opportunities to learn from each other
“Enhance training all the way down the line”
One third of regional conference participants identified education and training needs.
When it comes to education and training, participants want more courses relating to child and youth mental health in university programs. They need additional training for “frontline” child and youth mental health-care workers such as therapists, counselors and social workers, and increased awareness-raising and training for professionals such as lawyers, judges, physicians, ministry officials and teachers.
They also want the Centre to support training for parents to help them meet the needs of their children.

35. You can still support good research by:
3/28/2017
Don’t have research capacity? (… or have better things to do?)
You can still support good research by:
Consuming research
Facilitating research
Partnering around research
Identifying relevant applied questions
The Centre seeks to increase capacity in order to increase Evidence-Based Practice

36. 3/28/2017
Where do we go from here?
The process of implementing evidence-based practice starts 1 step at a time – today is that first step.
You’re not alone – this is why developing a community of practice around this issue is so important. The Centre is part of this – as well as other organizations.
Working together, we can create a tipping point and make things happen.