New England Vocational Rehabilitation Quality Assurance System 7th Annual Summit Performance Management in Vocational Rehabilitation September 8, 2014

The Charge October 2013 New England (NE) VR Directors NE Performance Evaluation and Quality Assurance (NE PEQA) staff Develop a Quality Assurance Model for the Region 1 VR agencies

NE PEQA Working Group Connecticut Maine Massachusetts New Hampshire Rhode Island Vermont NE TACE

Key Resources Institute on Rehabilitation Issues Performance Management: Program Evaluation and Quality Assurance in Vocational Rehabilitation 2011 Continuous Quality Improvement (Deming, 1982) NE PEQA Knowledge and Experience

The Model Seven Components Self-Assessment Definitions Elements Best Practices Interrelationships Self-Assessment

Component #1: Communication of Mission and Values Mission: Primary goal of the organization Values: Principles that mission and vision are based on Communicated via public documents, web sites, intranets and strategic plan; exemplified by consumer success stories

Mission/Values: Best Practices Supported & reinforced through training and aligned with procedures and operational roles Staff, consumers, partners, etc. are able to verbalize mission/vision/values Strategic Plan developed to meet needs of all stakeholders; all staff know how they contribute to mission and organization’s success

Component #2: Organizational Goals, Objectives and Timelines Long-terms goals established and translated into measurable short-term objectives and timelines Planning is ongoing and based on strategic & state plans & stakeholder input Project management, clear roles/responsibilities and regular monitoring are built into organizational structure

Goals/Objectives: Best Practices Multi-level, comprehensive strategic planning defines goals/objectives based on mission/vision/values Strategic Plan is “road map” with objectives/outcomes/timelines and measures and developed with stakeholder input

Component #3: Measures Tools, methods and processes that are used to quantify and assess past, ongoing and future goals and results Outcome measures (rehab rate, average earnings, etc.) Process measures (change in education level, frequency of case notes, etc.)

Measures: Best Practices Performance management system includes broad range of measures Measures aligned with program goals Measures are simple, easily understood, effectively implemented and include consumer characteristics

Component #4: Program Evaluation Data Systems Used by organizations to collect quantitative and qualitative data Allows for the validation and retrieval of data for evaluation, analysis, & reporting Elements include Case Management Systems and Operating Principles

Program Evaluation Data Systems: Best Practices Reliably stores Agency data & allows data preparation for analysis Includes data validation procedures Allows for integrated data sets that enable more extensive analysis Has effective security protocols, roles, and rules Provides data dashboard that relays data at the user level

Component #5: Analysis of the Data Process of transforming data into a usable and reportable form Adds meaning and value to the output of data collection tools and methods Informs quality improvement efforts

Analysis of Data: Best Practices Validate data Develop specific questions to aid in the analysis of data Re-evaluate collection of data periodically to ensure that what is collected is valuable to the program Maintain awareness of developments and best practices as they relate to VR performance management

Component #6: Dissemination The communication of results to specific audiences (e.g., counselors, staff, SRC members, CRPs, policy makers, etc.) Assists stakeholders in making more informed decisions that ultimately lead to improved consumer outcomes

Dissemination: Best Practices Practices should assure that all staff have a clear, consistent understanding of the Agency’s QA approaches There is a clearly defined and shared dissemination strategy Findings are developed & presented in ways that are useful for the intended users

Component #7: Cycle of Continuous Improvement A developmental and routine process It supports an ever-evolving Strategic Plan Is an ongoing effort to improve products, services and/or processes Widely used is Deming Cycle: Plan-Do-Check-Act

Continuous Improvement: Best Practices Expectation and a requirement of everyone in organization Changes not made without stakeholder input Policy and procedural changes are well considered and implemented strategically and systematically Project management mindset and tools Success is celebrated

Interrelationships of Core Components All seven components are closely related to each other Each component impacts every other and the overall quality assurance system A successful QA system requires recognizing and managing the interrelated components

State Self-Assessment NE PEQA found that every state has some pieces of each component All had good practices and those that could be improved None had really worked on identifying and managing them as a whole A tool was created for this purpose

Summary Discussion It’s about continuous improvement It’s not about perfection Everyone in the organization needs to participate A comprehensive QA system prevents the sole focus from being on regulatory compliance

Thank You! Susan Wells, Vermont DVR susan.wells@state.vt.us Ella McAllister, New Hampshire VR ella.mcallister@doe.nh.gov Karen Fraser, Maine BRS karen.d.fraser@maine.gov