What Did We Say We Were Going to Do? 1. Overview of Measurement and Change Paradigms 2. Audience Review and Discussion of Full Service Partnership Strategies.

What Did We Say We Were Going to Do? 1. Overview of Measurement and Change Paradigms 2. Audience Review and Discussion of Full Service Partnership Strategies Full Service Partnerships - January 30 – 31, 2007 Afternoon Session: 1:00pm – 3:00 pm T. Stephanie Oprendek, Ph.D. Acting Chief, Evaluation, Statistics and Support Branch California Department of Mental Health

Tri-Level Performance Measurement Paradigm

Client and Services Tracking Individual Client Outcomes Tracking Monitoring / Quality Assurance / Oversight (multi- stakeholder process) Monitoring / Quality Assurance / Oversight (multi- stakeholder process) Staff / Provider Evaluation / Satisfaction with regard to mental health system Client / Family Satisfaction / Evaluation of Services and Supports TRI-LEVEL PERFORMANCE MEASUREMENT PARADIGM Mental Health Promotion and Awareness Community Reaction / Evaluation / Satisfaction with regard to mental health system Large-Scale Community Indicators Mental Health System Structure / Capacity in Community PUBLIC / COMMUNITY- IMPACT LEVEL (Evaluation of Global Impacts and Community-Focused Strategies) MENTAL HEALTH SYSTEM ACCOUNTABILITY LEVEL (Evaluation of Community Integrated Services and Supports – Program/System-Based Measurement) INDIVIDUAL CLIENT LEVEL (Evaluation of Community Integrated Services and Supports – Individual Client Tracking) Client and Services Tracking Individual Client Outcomes Tracking Monitoring / Quality Assurance / Oversight (multi- stakeholder process) Monitoring / Quality Assurance / Oversight (multi- stakeholder process) Client / Family Satisfaction / Evaluation of Services and Supports Staff / Provider Evaluation / Satisfaction with regard to mental health system Mental Health Promotion and Awareness Community Reaction / Evaluation / Satisfaction with regard to mental health system Large-Scale Community Indicators Mental Health System Structure / Capacity in Community

PERFORMANCE MEASUREMENT Client and Services Tracking (Examples) Client-specific information, e.g., contact, demographic information, reason for system disengagement, etc. Services / supports information, e.g., new services/programs/supports pertinent to the MHSA, evidence-based practices, levels of care, partnering agency/provider services, etc. (Client and services/supports data capture is envisioned to be achieved through interoperable information systems residing at both the state and local levels. A phased-in approach will be used to achieve this long-term goal of full interoperability.) INDIVIDUAL CLIENT LEVEL (Evaluation of Community Integrated Services and Supports – Individual Client Tracking) Individual Client Outcomes Tracking (Examples) Initial and periodic assessments Ongoing assessments of core outcomes. The following are examples (State and local information systems interoperability, based on statewide standards, will be the mechanism by which this client outcome information is captured. DMH will work with counties/providers to provide flexible system options with regard to measurement of outcome indicators.) Recovery & Wellness Oriented Client Outcome Indicators : (These are examples; indicators and measures to be determined though stakeholder and committee recommendations.) Hopefulness Wellness Empowerment Self-efficacy, Etc… Etc.Suicide Physical healthSymptoms / Suffering Individual service plan goalsFamily preservation Social / community connectednessIncome / Entitlements Illness self-management Culture-specific indicators Hospitalization (acute//long term restrictive levels of care) Quality of LifeEmployment / Education Substance AbuseCriminal justice system involvement FunctioningHousing

PERFORMANCE MEASUREMENT Monitoring / Quality Assurance / Oversight (multi-stakeholder process) (Examples) Local / county plans and performance with respect to: Cultural competency / no disparities Recovery / Resilience philosophy and promotion Full participation of clients / family members in service delivery system processes Fidelity to evidence-based practice guidelines or model programs Adherence to budget / timelines Staff / provider competencies Adherence to appropriate client-to-staff ratios Quality (performance) improvement projects Service partnerships - Comprehensive / inter-agency / coordinated service delivery Supportive services (e.g., housing, employment, peer-delivered supportive services) Coordinated services for co-occurring disorders Costs, cost-effectiveness of services Etc. (Measured with standardized review criteria, monitoring tools, electronic data entry / reporting interfaces, etc. Cost information to be associated with client, service, and outcomes tracking information to determine costs per client, cost-effectiveness and cost-benefit analyses of programs, etc.) Staff / Provider Evaluation / Satisfaction with regard to mental health system (Examples) Perceived effectiveness of the structure of system, inter-agency issues, effectiveness of service models, etc. Interviews / surveys/ focus groups Etc. Client / Family Satisfaction / Evaluation of Services and Supports (Examples) Mental Health Statistics Improvement Program (MHSIP) indicators and surveys Surveys / assessments targeting specific services / supports appraisal by clients / families / caregivers Focus groups / multiple means of eliciting client / family / caregiver input Etc. MENTAL HEALTH SYSTEM ACCOUNTABILITY LEVEL (Evaluation of Community Integrated Services and Supports – Program/System-Based Measurement)

PERFORMANCE MEASUREMENT Mental Health Promotion and Awareness (Examples) Outreach services (e.g., homeless, rural communities, ethnic/culture-specific outreach, Tele-health, etc.) Community Emergency Response Team Services Community Mental Health / Depression Screenings Educational Seminars (e.g., general public, primary care settings, schools, etc.) Anti-Stigma and Anti-Discrimination Campaigns Prevention and Early Intervention Efforts Workforce Recruitment and Development (e.g., university, licensing board collaborations, continuing education) Community Support Groups Media, public awareness announcements, (e.g., Recovery & Resiliency) Access and educational enhancements (e.g., Network of Care website, promotion of recovery philosophy) Etc (Typically measured by counts of individuals reached, screened, informed, etc.) Community Reaction / Evaluation / Satisfaction with regard to mental health system (Examples) Media reviews Interviews with public officials Assessment of community members Etc. Large-Scale Community Indicators (Examples) Population prevalence of mental illness Community mental health need / unmet need Percents of youth in juvenile justice or Level12-14 group home placements Etc. Mental Health System Structure / Capacity in Community (Examples) Inventory of available services & supports (includes cultural competency and language proficiency) Location of services, including inter-agency, outreach, mobile, natural/community setting, etc (e.g., GIS mapping) Etc. PUBLIC / COMMUNITY- IMPACT LEVEL (Evaluation of Global Impacts and Community-Focused Strategies)

Accountability and Measurement of Outcomes, Objectives and Process

ACCOUNTABILITY One component of accountability is the effectiveness of services, supports and activities as measured by individual client outcomes and community impact. The other component is the demonstration that the mental health system is performing appropriately in providing services, supports, and activities - that is, doing what it should do & said it would do. These are the two “arms of accountability”:  They hold the MHSA transformational processes up to the light  They demonstrate that the mental health system is reaching out to both individuals and the community in ways that produce positive results. OUTCOMES OBJECTIVES PROCESS / METHODS

We can measure our achievement of particular objectives and interim outcomes on the path toward our ultimate (long term) goals/outcomes. OBJECTIVES: We must measure our process, which includes our methods and tracking our implementation, in order to be able to explain how we achieved our goals/outcomes/objectives. PROCESS / METHODS:

Longitudinal Change Paradigm Determine Objectives Met, Measure Interim Client & Community Outcomes Achieve objectives known/believed to lead to desired ultimate goals/outcomes. A measurement is only an “outcome” because we chose a particular time as “endpoint”. Measure Process: Track Implementation, Fidelity to Plan/Methods/Model Develop Concept/ Implement Intervention or Strategy e.g., Expansion Area Quality Improvement Need Identify Issue or Problem Measure Progress Toward Ultimate Goals/Outcomes Progress toward ultimate goals/outcomes (e.g., impacting MH prevalence) is likely to be modest and slow paced. Engaging multiple strategies and change partners potentially increases pace and magnitude of progress. Start

FSP Strategies Review and Discussion Evaluating and Documenting the Process -------- The “What” and the “How” of FSP Program Implementation

FSP Strategies Review and Discussion  An FSP strategy is not necessarily an EBP or “model”. Counties proposed service strategies reflective of their community planning process.  Not a question of “model adherence” but knowing if process is now consistent with plan and objectives.  Are FSP programs being implemented with respect to the “Key Constructs”?  If so, what is being done, and how?

FSP Strategies Review and Discussion  Review of Exhibit #4s: What was proposed in FSP strategies?  Evaluate the FSP strategies against the Key Constructs.  Review and document the “whats” and the “hows” of implementation to evaluate the strategies.

Areas in which to address the “Hows”:  Operationally  Administratively  Programmatically  Inter-agency  Interpersonally  Etc…

Key Constructs of FSP Programs Community collaboration Cultural competence Client and family driven Wellness/recovery/resiliency focus Integrated services for clients and families (which could include health and substance abuse issues)

Some FSP Strategy Specifics that Reflect the Spirit of the MHSA  Use of flexible funding  Interagency collaboration  24/7 service availability  Single point of responsibility  A “partnership” versus a provision of service to someone  Shared decision making on an individual’s road to recovery.  Linkage to or provision of all needed services  Individualized service plan that is person centered, with individuals and their families given sufficient information to allow them to make informed choices  Peer provided services  Strategies that reflect the CSOC and Wraparound core values and principles have been incorporated for children, youth, young adults and families  And more…..

Report Back to the Group What is being done differently now in FSP programs under MHSA? (Choose one or two promising areas to share.) How are these new things being accomplished? Where is more info/specificity needed to understand the process better? What areas pose the greatest challenges or have barriers to be overcome? What are some action steps to address barriers? (Breakouts may provide some guidance.) This process may help counties in providing information for the annual MHSA implementation update.

What Did We Say We Were Going to Do? 1. Overview of Measurement and Change Paradigms 2. Audience Review and Discussion of Full Service Partnership Strategies.

Similar presentations

Presentation on theme: "What Did We Say We Were Going to Do? 1. Overview of Measurement and Change Paradigms 2. Audience Review and Discussion of Full Service Partnership Strategies."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

What Did We Say We Were Going to Do? 1. Overview of Measurement and Change Paradigms 2. Audience Review and Discussion of Full Service Partnership Strategies.

Similar presentations

Presentation on theme: "What Did We Say We Were Going to Do? 1. Overview of Measurement and Change Paradigms 2. Audience Review and Discussion of Full Service Partnership Strategies."— Presentation transcript:

Similar presentations

About project

Feedback