1. for quality and accountability
Measuring Outcomes
for quality and accountability

2. The changing landscape in mental health and rehabilitation
Increasing demand for accountability i wide range of service settings
Funding agencies demand that providers examine service effectiveness and measure consumer outcomes
Indications of quality improvement and capacity of management
Meet core values and get input from service consumers

3. Planning a program evaluation
Individual accounts and anecdotes
Concrete and standardized ways to measure service effectiveness
What is “best practice”?

4. Planning a program evaluation
Program administrators are often not well informed about outcome measurement
… and how best to link outcome measures to questions that they want to address related to services

5. The best starting point is to determine:
What outcomes need to be addressed?
Who is the evaluation information being developed for?
Who will receive information about the results?
Example:
Results can be disseminated to stakeholders such as funding and accreditation bodies, a board of directors, primary consumers, or family members.

6. A program evaluation effort can be started at any time – when a service starts or added later to an existing service

7. Clarifying outcomes to be measured
Conducting an evaluation, study, or quality assurance effort begins with understanding which outcomes need to be addressed

8. Steps towards clarification:
Get an understanding of your agency’s intended areas of effectiveness
Example: The agency’s stated goal is “Increase in employment”
Ask whether the services your agency offers are likely to go beyond the intended outcomes to other unintended outcomes
Example: A bonus goal is “Increase in self-esteem among service recipients”

9. Examine your agency’s mission statement, strategic plan, or results of former program evaluation efforts to get closer to your agency’s intended effects and thus the outcomes you should measure

10. Determine what domains need to be measured from these efforts (housing, employment, social inclusion, satisfaction, etc.)
Develop clear definitions or criteria for these domains so that the appropriate measures or instruments can be located

11. Selecting an outcome measure or instrument
Providers must have confidence in the measures
Choose high quality and appropriate measures
Look for existing measures that match the outcomes needing to be addressed, rather than using home-grown surveys or instruments developed just for the purpose at hand

12. Ensure that the measures have been tested and are known to be reliable and valid, so …
that you can trust what they measure, and …
that they measure what they say they should

13. Two important paradigm shifts
The need for reliable and valid measures of the subjective aspects of recovery
The need for measures that were developed from a consumer perspective

14. “The times they are a-changing”
Providers, practitioners, managers, administrators, and policy makers can ease pressures they face in the present and in the future by …
making use of established, high quality measures and instruments, and
clarifying which effects and outcomes ought to be measured

15. The results from evaluations
… may be used not only to satisfy funding bodies and other stakeholders
- they will also provide useful information that can guide the direction of changes needed to create more effective ways of helping the people being served

16. Evaluation is an integrated component in program development
Procedures
Record keeping
Activities
Supervision structure
Policy statements
Job descriptions
Mission
Evaluation mechanisms