1. Rose Krebill-Prather, PhD
8/2/2018
Respondent Interaction with Survey Mode in an Assessment of Mental Health Services Outcomes
Rose Krebill-Prather, PhD
Danna Moore, PhD
Social and Economic Sciences Research Center (SESRC), Washington State University
Template-WSU Hrz 431 gradation.ppt

2. Introduction and Background
8/2/2018
Introduction and Background
2015 Survey
Mental Health Statistics Improvement Program (MHSIP)
Statewide
Consumers of outpatient mental health services
Telephone adapted to include web mode
Template-WSU Hrz 431 gradation.ppt

3. 8/2/2018
Key Questions
Does offering another survey mode enrich responses to an outcome assessment of mental health services?
Do web respondents have different characteristics?
Do web respondents report different experiences and outcomes for mental health services?
Template-WSU Hrz 431 gradation.ppt

4. Study Variables of interest
8/2/2018
Study Variables of interest
Demographics
Age
Gender
Employment status
Quality of life rating
 Experience with mental health service provider
Staff helped me obtain information I needed so that I could take charge of managing my illness
Outcomes of the services received
I am better able to control my life
My symptoms are not bothering me as much
I am better able to handle things when they go wrong
Template-WSU Hrz 431 gradation.ppt

5. Survey Data Collection Strategy
2015 MHSIP Survey: telephone/internet
Statewide random sample of consumers of state funded mental health services
Data collection period extended from August - September
Introductory postal letter sent one week prior to calling

6. Survey Response
A 35% response rate with 1686 completed surveys
91% by telephone
9% by internet

7. Results are based on bivariate chi-square analysis
8/2/2018
Analyses
Results are based on bivariate chi-square analysis
All statistically significant relationships are significant at a p-value < 0.01
Template-WSU Hrz 431 gradation.ppt

8. NO statistically significant differences by survey mode for: gender
Results
NO statistically significant differences by
survey mode for:
gender
age
employment status.

9. Quality of Life Rating: Significantly different by survey mode
Results
Quality of Life Rating:
Significantly different by survey mode

10. Experience with Mental Health Services Provider:
Results
Experience with Mental Health Services Provider:
Significantly different by survey mode

11. Mental Health Outcomes: I am better able to control my life
Results
Mental Health Outcomes:
I am better able to control my life
Significantly different by survey mode

12. Mental Health Outcomes: My symptoms are not bothering me as much
Results
Mental Health Outcomes:
My symptoms are not bothering me as much
Significantly different by survey mode

13. Results Mental Health Outcomes:
I am better able to handle things when they go wrong
Significantly different by survey mode

14. Discussion
These results suggest there is an interaction effect with telephone interviewers
Satisficing
Social desirability bias

15. Discussion
♦ Explore use indirect question techniques
♦ Are respondents with less positive experience with mental health services and fewer positive mental health outcomes more likely to participate in the survey when the web option is offered?

16. Conclusions
♦ Offering an alternative mode for completing the annual MHSIP Survey, via the web, revealed the possibility that telephone respondents are responding to the survey questions in a more socially desirable way—thereby giving slightly inflated positive ratings on mental health services experiences and outcomes.
♦ These results challenge researchers to explore ways to reduce the social desirability bias and to further examine whether offering the web brings in respondents who have truly had a less positive experience and have fewer positive outcomes with mental health services.

17. 8/2/2018
Thank you!
Template-WSU Hrz 431 gradation.ppt