1. DTES Second Generation Strategy
Evaluation Framework
DRAFT – NOT FOR DISTRIBUTION

2. Purpose of the evaluation framework
Give an overview of the strategy at a level that can be measured
System performance measurement
Quality improvement
Demonstrates logical link:
Identify gaps in measurement
Prioritize activities that will most likely get us to our ultimate goal
Inputs
Activities
Outputs
Outcomes

3. Current state of health in DTES
Poor health outcomes for DTES population
High volumes of ED visits and hospitalizations of at-risk people from DTES
Care is fragmented, uncoordinated, inaccessible for some
Housing is unstable
Access to nutritious food is challenging

4. Goal of DTES SGS
Improving health outcomes for the population in the DTES

5. DTES Second Generation Strategy Theory of Change
Long term outcomes
Ultimate goal
Intermediate outcomes
Indirect result of outputs
Immediate outcomes
Direct results of outputs
Outputs
The type, volume, and quality of products and services
Activities
The work we are doing to produce the outputs
Inputs
The resources required to undertake the activities

6. DTES Second Generation Strategy Theory of Change
Client-reported quality of life
Overall health improvement
Long Term
Health issues/risk identified and addressed before complications occur
Outcomes
Intermediate
Social Determinants of Health
People get access to nutrition
People are appropriately housed
At-Risk people engaged into care
Immediate
Unbroken attachment to care and treatment retention
Patient experience of care:
Trauma-informed
Culturally competent
Harm reduction
Recovery-orientation
Coordination
Outputs
Appropriate housing services
Nutritious food services
Access & Accessibility
Integrated care
Capacity building:
Trauma-informed care
Cultural competency
Best pain management practices
Harm reduction
Recovery-orientation
Activities
Strengthen relationships and partnerships
New models of care:
Low barrier addiction care model
MH substance use drop-in model
Integrated care model
Strategies:
Food & nutrition strategy
Housing strategy
Peers strategy
Low Threshold Addiction Clinic
New clinic sites
Integrated care teams
Housing contracts
Inputs
Embedded peers
Food contracts

7. DTES Second Generation Strategy Theory of Change
Focus of System Performance Measurement
Client-reported quality of life
Overall health improvement
Long Term
Health issues/risk identified and addressed before complications occur
Outcomes
Intermediate
Social Determinants of Health
People get access to nutrition
People are appropriately housed
At-Risk people engaged into care
Immediate
Unbroken attachment to care and treatment retention
Focus of Quality Improvement
Patient experience of care:
Trauma-informed
Culturally competent
Harm reduction
Recovery-orientation
Coordination
Outputs
Appropriate housing services
Nutritious food services
Access & Accessibility
Integrated care
Capacity building:
Trauma-informed care
Cultural competency
Best pain management practices
Harm reduction
Recovery-orientation
Activities
Strengthen relationships and partnerships
New models of care:
Low barrier addiction care model
MH substance use drop-in model
Integrated care model
Strategies:
Food & nutrition strategy
Housing strategy
Peers strategy
Integrated care teams
Housing contracts
Inputs
Low Threshold Addiction Clinic
New clinic sites
Embedded peers
Food contracts

8. DTES Second Generation Strategy Theory of Change
Focus of System Performance Measurement
Client-reported quality of life
Overall health improvement
Long Term
Health issues/risk identified and addressed before complications occur
Outcomes
Intermediate
People are appropriately housed/ increased housing tenure
At-Risk people engaged into care
Unbroken attachment to care and treatment retention
People get access to food
Immediate
Patient experience of care:
Trauma-informed
Culturally competent
Harm reduction
Recovery-based
Coordination
Outputs
Appropriate housing services
Food services
Access & Accessibility
Integrated care
Training:
Trauma-informed care
Cultural competency
Best pain management practices
Harm reduction
Recovery-based
Activities
Strengthen relationships and partnerships
New models of care:
Mental health & addiction care
MH substance use drop-in model
Integrated care model
Pain management best practices
Strategies:
Food & nutrition strategy
Housing strategy
Peers strategy
Low Threshold Addiction Clinic
New clinic sites
Integrated care teams
Housing contracts
Inputs
Embedded Peers
Food contracts

9. Performance Measurement
Peer reviewed, validated methods, generalizability of results
Give validity to the innovations of the SGS
VCH is partnering with external researchers at UBC and SFU to conduct population-based analysis of key health outcomes and some outputs of the SGS

10. Long Term Outcome Measure Indicator definition Data source
Reporting format/ Time frame
SGS Innovation
Overall health improvement
Increase in number of clients with HONOS score in target range
EMR HONOS, PARIS HONOS
Integrated care teams (ICTs) and clinics, and expanded mobile care.
Peer navigators
Shared treatment continuums
Dedicated ICT for women
Enhanced partnerships with private clinics
Cultural competence and trauma-informed practice
Peers at drop-ins, tenant support workers linking to care teams
Connect with private clinics
Address service gaps for women and children
Managed alcohol
Strategic plan for harm reduction
Best practices for pain management
Overdose training
Address social determinants of health: housing & food
Washrooms policy
Clinically significant improvement from baseline
VCHRI project
Oct 2017 – March 2018
Reduction in the rate of ED admissions, hospital admissions, death
Improved functional assessment in 4 key areas: physical health, substance use, mental health hospitalization; adherence to medications
TBD by data reference group
Chronic disease indicators per guideline care
EMR – TBD by data reference group
Dashboard / Quarterly
Client-reported quality of life
Clients with a Q-LES-Q score in target range
EMR HONOS, PARIS HONOS - TBD

11. Intermediate Outcome Measure Indicator definition Data source
Reporting format / Time frame
SGS Innovation
Health issues/risk identified and addressed before complications occur
Preventable hospitalizations: Number of in-patient acute care hospitalizations for conditions where appropriate ambulatory care may prevent or reduce the need for admission to hospital within VCH care.
ED and Acute data in Decision Support
Integrated care teams (ICTs) and clinics, and expanded mobile care.
Peer navigators
Shared treatment continuums
Dedicated ICT for women
Enhanced partnerships with private clinics
Cultural competence and trauma-informed practice
Peers at drop-ins, tenant support workers linking to care teams
Connect with private clinics
Address service gaps for women and children
Managed alcohol
Strategic plan for harm reduction
Best practices for pain management
Overdose training
Address social determinants of health: housing & food
Washrooms policy
Ambulatory care sensitive condition acute admissions
Acute data in DS
Dashboard / Quarterly
CTAS 4 and 5 level ED visits by known clients
ED visit rate
Hospitalization rate
Acute length of stay
# who stabilized on adequate methadone
LTAC - TBD
# who stabilized on adequate methadone/ suboxone
# days reduced illicit opiate use

12. Immediate Outcome Measure Indicator definition Data source
Reporting format / Time frame
SGS Innovation
At-Risk people engaged into care
Counts of both total volume and unique clients
VCHRI project
Oct 2017 – March 2018
Peer navigation
Drop-ins
Trauma-informed practice
Cultural competence
Dedicated ICT for women
Overdose training
Washrooms policy
Service engagement: No. of individuals known to have specified conditions I = (1,…n) who are engaged in optimal care.
Service engagement: # of new patients in program who initiated on methadone or suboxone
Unbroken attachment to care and treatment retention
Retention / Internal attachment: Clients who in the past 14 months have had at least 4 visits to the clinic/program (regular appointments – evenly distributed) and have a Care Plan documented, especially patients with certain health conditions
EMR
Dashboard / Quarterly
Integrated care teams
Mobile health services
Address gap in care for women and children
Connect with private clinics
People get access to nutritious food
Food access/nutrition: TBD
Service contracts that align with SGS food strategy
Appropriate housing
Appropriateness / stability of housing (tenure): To be developed
Housing models designed for appropriateness
Create efficient / effective system for access to housing

13. DTES Second Generation Strategy Theory of Change
Client-reported quality of life
Overall health improvement
Long Term
Health issues/risk identified and addressed before complications occur
Outcomes
Intermediate
Social Determinants of Health
People get access to nutrition
People are appropriately housed
At-Risk people engaged into care
Immediate
Unbroken attachment to care and treatment retention
Focus of Quality Improvement
Patient experience of care:
Trauma-informed
Culturally competent
Harm reduction
Recovery-orientation
Coordination
Outputs
Appropriate housing services
Nutritious food services
Access & Accessibility
Integrated care
Capacity building:
Trauma-informed care
Cultural competency
Best pain management practices
Harm reduction
Recovery-orientation
Activities
Strengthen relationships and partnerships
New models of care:
Low barrier addiction care model
MH substance use drop-in model
Integrated care model
Strategies:
Food & nutrition strategy
Housing strategy
Peers strategy
Integrated care teams
Housing contracts
Inputs
Low Threshold Addiction Clinic
New clinic sites
Embedded peers
Food contracts

14. Quality Improvement - Output measurement
Changes in outputs happen sooner than outcomes
Making sure we are on track to achieve improved health outcomes
Opportunity to fail fast and make course corrections

15. Output Measure: Quality of care Indicator Definition Data source
Reporting format / time frame
SGS innovation
Accessibility
Same day service: Clients who were referred to a certain service/team/program and were engaged in treatment/service that same day (clinical assessment, case management assessment, intervention, etc.)
EMR and PARIS
Dashboard
Same day service at point of request
Wait times to 1st contact: Clients with an open referral during the reporting period and who have been contacted by the treatment team/program
Wait times from service request to first contact
Cultural competence
Patient experience of culturally competent care
Core competencies survey
Core competency training
Trauma-informed care
Patient experience of trauma-informed care
Harm reduction
Patient experience of harm reduction care
Recovery-based care
Patient experience of recovery-based care

16. Output Measure Indicator definition Data source Reporting format
SGS Innovation
Access
Client volumes: Clients with an open referral during the reporting period
EMR and PARIS
Dashboard
Low Threshold Addiction Clinic
Integrated Care Teams
Number of clients turned away
TBD
Care coordination
Care coordination: % of clients with a new care plan or care plan update in last month
EMR and PARIS - TBD
Care coordinator
Integration
Integration: To what extent do you think that the array of services offered to DTES clients, at this time, is integrated in a way that best serves client needs?
VCHRI project
Data should be available Jan 2017
Peer navigators,
ICTs
Integration: # peers connected to drop-ins
Contract deliverable
Food services
Food contract (Schedule A)
Housing spaces
Contract deliverable - TBD

17. Gaps Measurement of access to nutritious food
Measures appropriate housing
Patient experience of attachment, engagement, self-management of health
Patient experience of accessibility, coordination & integration
Benchmarks
Program evaluation of some work streams
Process evaluation of the overarching project

18. Balanced scorecard Area Measures System Yes! Neighbourhood No Staff
Clients
Some

19. Questions for the group
Given the gaps – where would you prioritize?
Appetite for program evaluation?
Appetite for process evaluation?
Setting benchmarks?
Where do we want to be 18 months from now?
What do we want to know 18 months from now?
Where are our opportunities for shared monitoring and evaluation with the community?