Presentation on theme: "Effects of Survey Mode on Responses: An Experiment of Social Desirability Biases in Telephone, Interviewer, and Online Surveys Indeok Song Joongbu University,"— Presentation transcript:
Effects of Survey Mode on Responses: An Experiment of Social Desirability Biases in Telephone, Interviewer, and Online Surveys Indeok Song Joongbu University, Korea Sung Kyum Cho Chungnam National University
Contents Background: 3 Surveys on Gambling AddictionBackground: 3 Surveys on Gambling Addiction Research Question & PurposeResearch Question & Purpose Review of Previous ResearchReview of Previous Research MethodsMethods ResultsResults Discussions & SuggestionsDiscussions & Suggestions
Research Question What is the reason for the inconsistencies among 3 surveys results? Which mode does produce the most accurate estimates?Sampling? Survey Mode?
Previous Research Survey modes can be equivalent but are not always identical (Riva et al., 2003, p. 79) –Sensitivity & Social desirability (Booth-Kewley, Larson, & Miyoshi, 2007; Couper, 2000; Huang, 2006; Joinson, 1999; Kays, Gathercoal, & Buhrow, 2011; Tourangeau & Yan, 2007, 2011 and so on) Mode of admin.: Self- vs. Interviewer- administered –Self-administration increases reporting of socially undesirable behaviors (Corkrey & Parkinson, 2002; Currivan et al., 2004; Hochstim, 1967; Moskowitz, 2004; Mott, 1985; Richman et al., 1999, and so on) Paper-Pencil, Online > Telephone, Interviewer
Previous Research Interviewer or 3 rd party presence –Social desirability bias is worse in telephone survey than in face-to-face interviews (Aquilino, 1994; Groves & Kahn, 1979; Holbrook et al., 2003; Johnson, Hougland, & Clayton, 1989; Leeuw & van der Zouwen, 1988, and so on) –The presence of others affects responses to sensitive questions (Aquilino, Wright, & Supple, 2000; Harrison, 2001; Lombard & Ditton, 1997; Sudman, 2001; Tourangeau & Yan, 201, and so on ) –Online survey yields the lowest level of other peoples presence Online > Interviewer > Telephone
Research Purpose Controlling for variances from sampling and different groups of samples, to examine the effects of survey modes on the responses to sensitive questions.
Methods Pretest-posttest 3 between-group experimental design –93 undergraduate students 66 analyzed Out of 66, male 38 (57.6%), female 28 (42.4%), mean age 20.61 (SD=1.97) Paper-pencil (N=93) 2 weeks Random Assignment Online (n=21) Telephone (n=24) Interview (n=21)
Methods –Sexual experience Age of the first experience –n=27, M=18.59, SD=1.67 Number of sexual partners –n=26, M=10.81, SD=3.68 Sexual behaviors (physical contact, cuddle, kiss, caress, intercourse) experienced in current relationship –n=19, M=3.68, SD=1.42 –Demographics Gender, age, household-income
Results Homogeneity among groups (pretest: paper- pencil)Homogeneity among groups (pretest: paper- pencil) SourcedfFp Gambling Addiction(2, 63)1.29.28 Internet Addiction(2, 63).09.91 Game Addiction(2, 63)2.13.13 Dissatisfaction with Self(2, 63).31.73 Sexual Values(2, 63).28.76 Age of 1 st Experience(2, 24)1.89.19 No. of Partners(2, 23).59.56 Sexual Behaviors(2, 16).24.79
Responses for Dissatisfaction with Self-Appearance
2 x 3 Factorial ANOVAs Independent Variables –2 groups (Low & High) by median split –3 modes (Online, Telephone, Interview) Dependent Variables –Changes in responses = Pretest – Posttest DVsNMSD Gambling Addiction66.272.30 Internet Addiction661.086.78 Game Addiction661.148.05 Dissatisfaction with Self661.183.13 Sexual Values661.444.65 Age of 1 st Experience22.681.49 No. of Partners22-3.9516.76 Sexual Behaviors16-.131.46
Changes in Responses for Age of the 1 st Intercourse
Summary of the Results Responses for Game Addiction & Dissatisfaction with Self-Appearance –Increased in the online survey mode –Decreased in the telephone & interview survey modes Those who scored higher on the sensitive questions are more likely to change their responses in a socially desirable direction –Less addicted (gambling, Internet, game) –Less dissatisfied with self-appearance –Less liberal (more conservative) sexual values –Younger in the age of the 1 st sexual intercourse
Discussions & Suggestions Survey responses to sensitive (privacy-related) questions are affected by different survey modes. –Online survey in which respondents are less concerned about their privacies is more likely to elicit franker (more accurate) responses. Paper-pencil (group administration), telephone, interview modes lead respondents to privacy concern & misreport. Suggestion: Dual-mode survey Telephone Sampling Online Questionnaire