1. 1 Kenneth E. Wallen a,b,c, Adam C. Landon a,b, Gerard T. Kyle a,b,c, Michael A. Schuett a,c, Jeremy Leitz d, & Ken Kurzawski d a Department of Recreation, Park and Tourism Sciences, Texas A&M University b Human Dimensions of Natural Resources Lab, Texas A&M University c Applied Biodiversity Science NSF-IGERT Program, Texas A&M University d Texas Parks and Wildlife Department Sampling Efficacy & Bias in Mode of Response for Survey-Based Research

2. 2 2 Background ‘’89 ‘90 ‘93 ‘97 ‘04 ‘08 ‘01 ‘12

3. 3 3 Background

4. 4 4

5. 5 5

6. 6 6

7. 7 7

8. 8 8

9. 9 9

10. 10 Background

11. 11 Background

12. 12 Response Rates Why do we care about response rates? Need (desire) to reduce survey error… the difference between the estimate using the collected data and the true value of the variables in the population Non-response error… emerges when those who do not respond differ from those who respond Mail questionnaires are costly… printing, postage & labor

13. 13 Trends in Response Rates All modes of collection area are on the decline Mixed-mode designs (mail/web) have demonstrated success for reducing non-response error Respondents to mail and web-based collection modes are different demographically and with their use of technology Mixed-mode benefit is twofold… opportunity in preferred manner & remind respondent of the opportunity to respond

14. 14 Texas Recreational Anglers Heterogeneous Demographics Motivations Preferred resources Target species Avidity Expenditures Imperative to reduce non- response error to obtain data that is reflective of the angling population to make informed decisions that will impact this population

15. 15 Study Purpose Compare three modes of questionnaire administration in terms of: 1.Response rate 2.Respondent characteristics

16. 16 Survey Methodology Data Collection Sample drawn randomly from TPWD database of licensed fresh/saltwater anglers Three modes of contact commencing the last week of September, 2012 Mixed-mode (n=4,000) Email-Only (n=4,000) Combination (n=1,000) Web-based questionnaire was designed/administered thru Qualtrix with the URL: WWW.TPWD-SURVEY.ORG

17. 17 Mixed-Mode 1.Initial contact: Invitation letter with web push 2.One week follow-up: Thank you/reminder postcard with web push 3.Two week follow-up: Second contact letter to non- respondents with web push 4.Three week follow-up: Survey packet to non-respondents… cover letter (with web push), questionnaire, and postage- paid reply envelope

18. 18 Email-Only Four email invitations, one week apart Similar to the cover letter sent to the Mixed-Mode group, the email invitations outlined the study purpose and invited respondents to complete the questionnaire online Both a URL to be entered into respondents’ browser and hyperlinked icon (“Take Survey”) were provided for respondents to access the questionnaire

19. 19 Combination A combination of contacts replicating the “Mixed Mode” and “Email Only” methods Eight invitations (4 hardcopy, 4 email invitations) were sent to respondents, one week apart The sending of email and postal invitations were synchronized to arrive simultaneously; i.e., email invitations were sent approximately two days following the mailing of hard copies

20. 20 Survey Response Effective Response Rates Email-only 29.9% (784/2685) Mixed-mode 20.0% (697/3486) Combination 63.4% (407/640)

21. 21 Survey Response 53.8% of Mixed-mode respondents completed hard copies of the questionnaire 29.5% of Combination respondents completed hard copies

22. 22 Survey Analysis Multinomial Logistic Regression Model (logit) Dependent variable: survey response mode Null-model Approach Independent Variables Socio-demographics, motivations, & avidity Reference group: Mixed-mode Ø = Mixed-mode 1 = Combination 2 = Email only Age + gender + income + race + ethnicity + motivation + avidity

23. 23 Table 1. Multinomial logistic regression results and item descriptives Mixed-modeCombinationEmail-only Mean% Coeff.s.e%StdXMean%Coeff.s.e.%StdX Socio-demographics Age49.4049.95.001.0061.848.30-.007.005-9.3 Income5.135.9.129**.03836.06.09.162**.03147.2 Female12.99%4.60%-1.012**356-23.44.47%-1.015**.274-23.4 Hispanic6.86%5.44%-.221.360-5.26.15%-.045.284-1.1 Black1.72%.42%-1.5421.085-13.2.37%-1.676*.828-14.2 N=1148, Likelihood-ratio X 2 = 144.17, p > X 2 =.000, Pseudo R 2 =.06, **p<.005, *p<.05 Note: Reference group is Mixed-mode Socio-demographics

24. 24 Table 1. Chi-square comparison of observed and expected income by survey mode Under $20,000 $20,000- $59,999 $60,000- $99,999 $100,000- above Mixed-mode Observed83359139 Expected7.341.665125.2 Percentage3.3513.8124.6958.16 Email-only Observed966150312 Expected16.393.4146281.2 Percentage1.6812.2927.9358.10 Combination Observed19107113169 Expected12.471111213.6 Percentage4.6626.2327.741.42 Notes. χ 2 = 49.90, p < 0.001, Cramér's V = 0.15 Income

25. 25 Table 2. Chi-square comparison of observed and expected income by survey mode Under $20,000 $20,000- $59,999 $60,000- $99,999 $100,000- above Mixed-mode Observed83359139 Expected7.341.665125.2 Percentage3.3513.8124.6958.16 Email-only Observed966150312 Expected16.393.4146281.2 Percentage1.6812.2927.9358.10 Combination Observed19107113169 Expected12.471111213.6 Percentage4.6626.2327.741.42 Notes. χ 2 = 49.90, p < 0.001, Cramér's V = 0.15 Income

26. 26 Table 3. Chi-square comparison of male and female respondents by survey mode FemaleMale Mixed-mode Observed11228 Expected17.8221.2 Percentage4.695.4 Email-only Observed24513 Expected39.9497.1 Percentage4.4795.53 Combination Observed53355 Expected30.3377.7 Percentage12.9987.01 Notes. χ 2 = 27.95, p < 0.001, Cramér's V = 0.15 Gender

27. 27 Within Groups – Socio-demographics Table 4. T-test results between respondents who completed survey by paper or online within mixed-mode and combination groups Mixed-modeCombination PaperOnlinep-valuePaperOnlinep-value Socio-demographics Age 51.347.3.00654.448.8.007

28. 28 Table 1 (continued). Multinomial logistic regression results and item descriptives Mixed-modeCombinationEmail-only Mean% Coeff.s.e%StdXMean%Coeff.s.e.%StdX Motivations/Avidity To be outdoors4.394.48.092.1496.54.56.325*.12524.7 For family recreation3.963.87-.218*.094-20.53.95-.143.079-13.9 To experience new things3.443.38-.154.094-16.33.36-.214*.078-22.0 To be with friends3.753.92.228*.09726.63.95.221**.07925.8 (Not) most important activity35.29%28.87%-.351.191-29.626.26%-.512**.158-40.1 N=1148, Likelihood-ratio X 2 = 144.17, p > X 2 =.001, Pseudo R 2 =.06, **p<.001, *p<.05 Note: Reference group is Mixed-mode Motivation & Avidity

29. 29 Table 5. Results of ANOVA for Motives EffectMeanSDSSF-valuedf To be outdoors 1 Mixed-Mode4.490.67 Email-Only4.56 a 0.64 Combination4.40 a 0.72 Between-groups6.166.76***2 To be with friends 1 Mixed-Mode3.920.99 Email-Only3.95 b 1.01 Combination3.75 b 1.08 Between-groups10.174.78**2 Notes. 1 Mean score value is on a scale ranging from 1 (not important) to 5 (extremely important). Like superscripts indicate significant differences at p ≤ 0.01 (Scheffe post-hoc analysis). ** p-value < 0.01 ***p-value < 0.001 Motives

30. 30 Table 6. Chi-square comparison of observed and expected importance/non-importance of angling as primary recreational activity by survey mode Most importantNot most important Mixed-mode Observed69170 Expected71.5167.5 Percentage28.8771.13 Email-only Observed141396 Expected160.6376.4 Percentage26.2673.74 Combination Observed144264 Expected122286 Percentage35.2964.71 Notes. χ 2 = 9.19, p < 0.01, Cramér's V = 0.09 Avidity

31. 31 Within Groups - Motivation Table 7. T-test results between respondents who completed survey by paper or online within mixed- mode and combination groups Mixed-modeCombination PaperOnlinep-valuePaperOnlinep-value Motivations To be outdoors 4.34.5.0014.14.6.000 To experience new things 3.53.4.5762.93.5.004 To get away from demands 3.63.7.6473.53.7.245 For the experience of the catch 4.0.5384.24.1.408 To experience natural surroundings 3.83.9.3303.54.2.000 To get away from the regular routine 3.94.2.0053.84.2.003 To obtain a trophy fish 2.2.4742.22.3.664 For the challenge or sport 3.43.8.0023.73.8.570 Mixed-mode: N=408, LR X 2 = 66.63, p =.000, Pseudo R 2 =.12 Combination: N=239, LR X 2 = 51.93, p =.000, Pseudo R 2 =.21

32. 32 Within Groups - Motivation Table 7. T-test results between respondents who completed survey by paper or online within mixed-mode and combination groups Mixed-modeCombination PaperOnlinep-valuePaperOnlinep-value Motivations To be outdoors 4.34.5.0014.14.6.000 To get away from the regular routine 3.94.2.0053.84.2.003 For the challenge or sport 3.43.8.0023.73.8.570 Most important/Not most important 35.7%35.0%.862 30.0%28.6%.843 Mixed-mode: N=408, LR X 2 = 66.63, p =.000, Pseudo R 2 =.12 Combination: N=239, LR X 2 = 51.93, p =.000, Pseudo R 2 =.21

33. 33 Discussion – Response Rates Combination (surface mail & email) yielded strongest response rate Lowest non-response error? Respondents (or non-respondents) are reluctant to go from the paper invitation to their computer or device

34. 34 Discussion - Socio-demographics Socio-demographic variation Variations in age Web-based respondents slightly higher household incomes Men more inclined than women to complete online

35. 35 Discussion – Motivation & Avidity Motivation On items where there was significant variation web-based respondents considered these facets more important Avidity Some indication that hard copy respondents most avid

36. 36 Next Steps 2015 Survey of licensed Texas Anglers Mixed-Mode – Mail survey packet with a web push Incentive – “lifetime license”

37. 37 Acknowledgements Texas Parks and Wildlife Department Robin Riechers John Taylor