Rose Krebill-Prather, PhD 8/2/2018 Respondent Interaction with Survey Mode in an Assessment of Mental Health Services Outcomes Rose Krebill-Prather, PhD Danna Moore, PhD Social and Economic Sciences Research Center (SESRC), Washington State University Template-WSU Hrz 431 gradation.ppt

Introduction and Background 8/2/2018 Introduction and Background 2015 Survey Mental Health Statistics Improvement Program (MHSIP) Statewide Consumers of outpatient mental health services Telephone adapted to include web mode Template-WSU Hrz 431 gradation.ppt

8/2/2018 Key Questions Does offering another survey mode enrich responses to an outcome assessment of mental health services? Do web respondents have different characteristics? Do web respondents report different experiences and outcomes for mental health services?   Template-WSU Hrz 431 gradation.ppt

Study Variables of interest 8/2/2018 Study Variables of interest Demographics Age Gender Employment status Quality of life rating  Experience with mental health service provider Staff helped me obtain information I needed so that I could take charge of managing my illness Outcomes of the services received I am better able to control my life My symptoms are not bothering me as much I am better able to handle things when they go wrong Template-WSU Hrz 431 gradation.ppt

Survey Data Collection Strategy 2015 MHSIP Survey: telephone/internet Statewide random sample of consumers of state funded mental health services Data collection period extended from August - September 2015 Introductory postal letter sent one week prior to calling

Survey Response A 35% response rate with 1686 completed surveys 91% by telephone 9% by internet  

Results are based on bivariate chi-square analysis 8/2/2018 Analyses Results are based on bivariate chi-square analysis All statistically significant relationships are significant at a p-value < 0.01 Template-WSU Hrz 431 gradation.ppt

NO statistically significant differences by survey mode for: gender Results NO statistically significant differences by survey mode for: gender age employment status.  

Quality of Life Rating: Significantly different by survey mode Results Quality of Life Rating: Significantly different by survey mode

Experience with Mental Health Services Provider: Results Experience with Mental Health Services Provider: Significantly different by survey mode

Mental Health Outcomes: I am better able to control my life Results Mental Health Outcomes: I am better able to control my life Significantly different by survey mode

Mental Health Outcomes: My symptoms are not bothering me as much Results Mental Health Outcomes: My symptoms are not bothering me as much Significantly different by survey mode

Results Mental Health Outcomes: I am better able to handle things when they go wrong Significantly different by survey mode

Discussion These results suggest there is an interaction effect with telephone interviewers Satisficing Social desirability bias

Discussion ♦ Explore use indirect question techniques ♦ Are respondents with less positive experience with mental health services and fewer positive mental health outcomes more likely to participate in the survey when the web option is offered?

Conclusions ♦ Offering an alternative mode for completing the annual MHSIP Survey, via the web, revealed the possibility that telephone respondents are responding to the survey questions in a more socially desirable way—thereby giving slightly inflated positive ratings on mental health services experiences and outcomes. ♦ These results challenge researchers to explore ways to reduce the social desirability bias and to further examine whether offering the web brings in respondents who have truly had a less positive experience and have fewer positive outcomes with mental health services.

8/2/2018 Thank you! Template-WSU Hrz 431 gradation.ppt