Recovery Oriented System Indicators (ROSI) Survey FY 2015 ROSI Survey Results Virginia Department of Behavioral Health and Developmental Services September, 2015

FY 2015 ROSI Survey Method In Fiscal Year 2015, 40 CSBs conducted the ROSI survey with 4,084 adult recipients of mental health services. 16 CSBs – used online SurveyMonkey ® 6 CSBs – used paper version and local data entry 18 CSBs – used combination of paper version and online version

Respondents by Category (n=4,084)

Survey Analysis Using MS Excel, Access, and SPSS –Average scores from 1 to 4 1 = strongly disagree 2 = disagree 3 = agree 4 = strongly agree –% responding Yes = agree or strongly agree No = disagree or strongly disagree N/A = does not apply to me Blank = no response

8 ROSI Recovery Domains Ranked Average Score / 4 = “Grade” Formal Service Staff / 4 = 84.9 Formal Services / 4 = 83.8 Social Relationships / 4 = 83.7 Choice / 4 = 81.5 Basic Material Resources / 4 = 76.2 Meaningful Activities / 4 = 76.1 Peer Support / 4 = 75.1 Self/Holism / 4 = 73.6

Findings: Formal Service Staff Formal Service Staff – Avg. Score = –the critical roles formal service staff play in helping or hindering the recovery process roles % ‘Yes’: Survey Item (average score) 90%: Staff respect me as a whole person. (3.4) 85%: Staff treat me with respect regarding my cultural background. (3.6) 84%: Staff believe that I can grow, change and recover. (3.5) 83%: Staff listen carefully to what I say. (3.5)

Findings: Formal Service Staff Formal Service Staff continued % ‘Yes’: Survey Item (average score) 79%: Staff see me as an equal partner in my treatment program (3.4) 75%: My treatment plan goals are stated in my own words. (3.4) 18%: Staff do not understand my experience as a person with mental health problems. (1.9) 21%: Staff lack up-to-date knowledge on the most effective treatments. (1.8) = Increase of 2 or more percentage points from FY 2013

Findings: Formal Services Formal Services – Avg. Score = –the systems’ culture, organization, structure, funding, access, choice, quality, range, continuity and other characteristics can help or hinder the process of recovery % ‘Yes’: Survey Item (average score) 81%: Mental health staff support my self-care or wellness. (3.5) 78%: Mental health staff help me build on my strengths. (3.4) 77%: The doctor worked with me to get on medications that were most helpful for me. (3.5) 75%: I have information and/or guidance to get the services and supports I need. (3.3) 68%: I can see a therapist when I need to. (3.2)

Findings: Formal Services Formal Services continued % ‘Yes’: Survey Item (average score) 47%: My family gets the education or supports they need to be helpful to me. (2.9) 5%: Staff use pressure, threats or force in my treatment. (1.2) 10%: Mental health services have caused me emotional or physical harm. (1.7) 12%: The mental health staff ignore my physical health. (1.8) 16%: I cannot get the services I need when I need them. (1.9)

Findings: Social Relationships Social Relationships – Avg. Score = –the roles social and personal relationships play in facilitating recovery % ‘Yes’: Survey Item (average score) 92%: There is at least one person who believes in me. (3.4) 9%: Mental health staff interfere with my personal relationships. (1.4) 21%: I do not have the support I need to function in the roles I want in my community. (2.0)

Findings : Choice Choice – Avg. Score = –having choices, as well as support in the process of making choices, regarding housing, work, social, service, treatment as well as other areas of life facilitate recovery % ‘Yes’: Survey Item (average score) 81%: Staff give me complete information in words I understand before I consent to treatment or medication. (3. 5) 69%: I have a say in what happens to me when I am in crisis. (3.2) 68%: My right to refuse treatment is respected. (3.3) 22%: I do not have enough good service options to choose from. (2.0) = Decline of 2 or more percentage points from FY 2013

Findings: Basic Material Resources Basic Material Resources – Avg. Score = –(recovery from mental illness is incumbent on basic material resource needs being met). % ‘Yes’: Survey Item (average score) 84%: I have a place to live that feels like a comfortable home to me (3.2) 78%: Staff stood up for me to get the services and resources I needed (3.4) 72%: Mental health services helped me get medical benefits that meet my needs. (3.1)

Findings: Basic Material Resources Basic Material Resources continued % ‘Yes’: Survey Item (average score) 65%: I have reliable transportation to get where I need to go. (3.1) 60%: I have housing that I can afford. (3.1) 49%: Mental health services helped me get housing in a place I feel safe. (2.9) 47%: I have enough income to live on. (2.4) = Decline of 2 or more percentage points from FY 2013

Findings: Meaningful Activities Meaningful Activities – Avg. Score = –(work, education, voluntary and/or group advocacy activities that are meaningful to the individual facilitate recovery). %‘Yes’: Survey Item (average score) 82%: Staff encourage me to do things that are meaningful to me. (3.5) 49%: I have a chance to advance my education if I want to. (2.9) 25%: Mental health services helped me get or keep employment. (2.5)

Findings: Peer Support Peer Support – Avg. Score = –peer support and consumer operated services in a myriad of forms facilitate recovery % ‘Yes’: Survey Item (average score) 77%: I am encouraged to use consumer-run programs (e.g., support groups, drop-in centers, etc.) (3.2) 48%: There was a consumer peer advocate to turn to when I needed one. (2.9) 44%: There are consumers working as paid employees in the mental health agency where I receive services. (2.9) =Decline of 2 or more percentage points from FY 2013

Findings: Self/Holism Self/Holism – Avg. Score = –characteristics that relate to one’s sense of self, such as self- reliance, as well as having a holistic and human rights focus can facilitate recovery and other such characteristics, such as low self-esteem, can hinder recovery % ‘Yes’: Survey Item (average score) 81%: Services help me develop the skills I need. (3.1) 36%: Mental health services led me to be more dependent, not independent. (2.3) 20%: I lack the information or resources I need to uphold my client and basic human rights. (2.0)

ROSI Score Comparisons 53% scored their CSB’s recovery orientation above the average score of 3.19 What are the characteristics of respondents who rate their CSB above average in the overall score? Recovery Oriented System Indicators (ROSI) Survey Score Comparisons

Above Average Score of 3.19 (n=1,999) % Scoring above average: significant differences by gender, race, and education –Gender: Male 51% Female 57% –Race: White 59%Black 50% = Increase of 2 or more percentage points from FY 2013= Decline of 2 or more percentage points from FY 2013 Education Level: < High School High School College Grad. School % Scoring Above Average 52%55%56%49%

Rated Below Rated Above Above or Below Average by Gender

Above or Below Average by Race Rated Below Rated Above

Rated Below Rated Above Above or Below Average by Education Level

Above Average Score of 3.19 (n=1,999) % Scoring above average: significant differences by CSB region, community, and housing type –Community: Urban 47% Rural 59% Housing Type: homeless boarding home residential facility supervised apartment own home % Scoring Above Average 24%35%55%49%60% = Increase of 2 or more percentage points from FY 2013= Decline of 2 or more percentage points from FY 2013 Region% AboveRegion% Above 156%550% 245%651% 376%751% 445%All53%

Above or Below Average by Partnership Planning Region Rated Below Rated Above

Above or Below Average by Community Type Rural vs. Urban CSBs Rated Below Rated Above

Rated Below Rated Above Above or Below Average by Housing Type

Above Average Score of 3.19 (n=1,999) % Scoring above average: significant differences by years in service, number and type of services –In service five years or less:50% –In service more than five years:56% –Number of services received: One:43%Two: 53% Three to Four: 57%More than Four: 58% –Service received: Medication:58% Counseling: 55% Case Management: 58% = Increase of 2 or more percentage points from FY 2013= Decline of 2 or more percentage points from FY 2013

Above or Below Average by Time in CSB Services > 5 Years Rated Below Rated Above NoYes

Above or Below Average by Number of CSB Services Received Rated Below Rated Above

Above or Below Average by Receiving Medication Service Rated Below Rated Above NoYes

Above or Below Average by Receiving Counseling Service Rated Below Rated Above NoYes

Above or Below Average by Receiving Case Management Service Rated Below Rated Above NoYes

ROSI Score Comparisons Which survey items showed the biggest difference between above and below overall average scores? Recovery Oriented System Indicators (ROSI) Survey Score Comparisons

Which items make the most difference? The largest difference in responses between the percent of respondents who scored their CSB above average vs. those who scored their CSB below average: % Above – Below by Survey Item –73% - 33% Help getting/keeping employment –82% - 45%: Having Peer Advocates to turn to –82% - 46%: Chance to advance my education –81% - 47% Family gets education or supports –92% - 60%: Having a say in what happens in crisis

Above or Below Avg. by Employment Assistance Rated Below Rated Above Strongly Disagree Strongly Agree DisagreeAgree

Above or Below Average by Peer Advocate Rated Below Rated Above Strongly Disagree Strongly Agree DisagreeAgree

Above or Below Average by Chance to Advance Education Rated Below Rated Above Strongly Disagree Strongly Agree DisagreeAgree

Above or Below Average by Family Education and Supports Rated Below Rated Above Strongly Disagree Strongly Agree DisagreeAgree

Above or Below Average by Having a Say when in Crisis Rated Below Rated Above Strongly Agree DisagreeAgreeStrongly Disagree

CSB Provider Survey CSBs that reported sharing the results of last year’s ROSI survey with staff and the individuals receiving services had a higher percentage of respondents rating them above average. And CSBs that reported making changes based on their ROSI survey responses had a higher percentage of respondents rating them above average in their Recovery Orientation.

Above or Below Average by Whether ROSI Results are Shared * Rated Below Rated Above NoYes * With staff and individuals receiving services

Examples of How FY 15 Survey Results Were Shared  “Results were initially shared with program managers and staff, who then shared the results with individuals at the times they were at the programs receiving services.”  “Survey results were shared with the consumer advisory council and sent to a mailing list of over 300 individuals, which includes service recipients, peer-providers, family members and service providers.”  “Results were reviewed with program staff for discussion and planning as well as with individuals receiving services through psychosocial rehabilitation.”  “A copy of the ROSI survey results was distributed to each staff member…Also, a copy of the ROSI survey was left in the reception area for general population perusal of results.”  “We reviewed the results of the last survey with individuals at our psychosocial rehabilitation program during a group meeting.”

Above or Below Average by Made Changes based on Survey* Rated Below Rated Above NoYes * Based on ROSI survey results

Examples of Actions Taken Based on Survey Results “We began offering WRAP seminars this year.” “Created and filled a Peer Support Specialist position.” “Developed and distributed recovery-oriented handbook for consumers regarding services.” “A peer mentor program was developed to assist individuals on pass from the state hospital to help in adjusting to the clubhouse environment.” “Greater emphasis was placed on developing priorities, setting program benchmarks and formalizing Peer Recovery Services (PRS) as a value-added service. Some examples of this work include finalizing the PRS brochure, scheduling structured monthly peer supervision meetings, setting-up in-service trainings about PRS to market services internally and filling the need for direct PRS services.” “MH and SA staff have increased the focus on improving medical health of individuals and assisted consumers in understanding insurance changes.” “Developed strategies to make improvements and followed up on those in smaller teams. This included the general training and reminders about language used with clients, the need to remind clients of rights and not assume they are aware, etc.”