Presentation is loading. Please wait.

Presentation is loading. Please wait.

Use of Incentives in Surveys Supported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December.

Similar presentations


Presentation on theme: "Use of Incentives in Surveys Supported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December."— Presentation transcript:

1 Use of Incentives in Surveys Supported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December 9, 2008

2 2 5/16/2008 Overview What we know about incentives from the survey methods literature What we know about incentives from the survey methods literature IRB issues in providing incentives IRB issues in providing incentives Results of a survey of NIH grant recipients who planned to do surveys Results of a survey of NIH grant recipients who planned to do surveys Conclusions Conclusions

3 3 5/16/2008 Survey Literature Findings on Use of Financial Incentives Incentives improve response across modes Incentives improve response across modes Effects appear more or less linear, size matters Effects appear more or less linear, size matters Prepaid incentives more effective than promised Prepaid incentives more effective than promised Lotteries less effective than payments Lotteries less effective than payments Lower income respondents more responsive than higher income respondents Lower income respondents more responsive than higher income respondents Incentives can help increase representativeness of the sample Incentives can help increase representativeness of the sample

4 4 5/16/2008 How Financial Incentives Fit Into Survey Response Respondents consider costs of survey participation Respondents consider costs of survey participation Time, inconvenience Time, inconvenience Loss of privacy, concerns about other possible consequences Loss of privacy, concerns about other possible consequences How it will feel to be interviewed or complete the survey How it will feel to be interviewed or complete the survey How others might view them for taking part How others might view them for taking part Whether the purpose of the survey is a potential loss for them Whether the purpose of the survey is a potential loss for them

5 5 5/16/2008 Non-Financial Reasons for Survey Participation Possible benefits other than payment Possible benefits other than payment Belief that the survey will be beneficial to them or to others, desire to help Belief that the survey will be beneficial to them or to others, desire to help Interest in the survey, desire to talk about the topic Interest in the survey, desire to talk about the topic Helping or being associated with a survey sponsor Helping or being associated with a survey sponsor Belief that participating in the survey will be pleasant Belief that participating in the survey will be pleasant Desire to help and/or talk with interviewer Desire to help and/or talk with interviewer Prestige from being a study participant Prestige from being a study participant

6 6 5/16/2008 Institutional Review Boards All federally supported surveys should be reviewed by an IRB All federally supported surveys should be reviewed by an IRB IRBs operate under guidelines from DHHS Office for Human Research Protections IRBs operate under guidelines from DHHS Office for Human Research Protections Requirements for approval: Requirements for approval: Risks are minimized Risks are minimized Subject selection is equitable, vulnerable populations protected Subject selection is equitable, vulnerable populations protected Informed consent Informed consent Protection for subjects privacy and data confidentiality Protection for subjects privacy and data confidentiality Each IRB can and does interpret guidelines in light of local circumstances Each IRB can and does interpret guidelines in light of local circumstances

7 7 5/16/2008 IRBs Wrestle With Incentives Do they coerce people who would not otherwise freely agree to participate in research to do so? Do they coerce people who would not otherwise freely agree to participate in research to do so? How should the level of incentives be determined? How should the level of incentives be determined? As wages for time and effort of participation? As wages for time and effort of participation? Based on the value researchers place on participation? Based on the value researchers place on participation? Equitably for all participants or recognizing different values participants may place on their effort and time? Equitably for all participants or recognizing different values participants may place on their effort and time?

8 8 5/16/2008 Special Problems with Incentives for IRBs Lotteries Lotteries Provide unequal rewards across participants Provide unequal rewards across participants Undermine informed decision since chances of winning are overvalued Undermine informed decision since chances of winning are overvalued Disadvantaged populations Disadvantaged populations Especially vulnerable to coercion Especially vulnerable to coercion May be induced to lie or conceal information in order to participate May be induced to lie or conceal information in order to participate Higher payments for people who refuse to participate Higher payments for people who refuse to participate

9 9 5/16/2008 Survey Literature on Incentives Does Not Address Ethical Issues Focus is on practical concern with effectiveness in terms of response rates and/or MSE Focus is on practical concern with effectiveness in terms of response rates and/or MSE AAPOR Best Practices suggests considering use of incentives to stimulate cooperation AAPOR Best Practices suggests considering use of incentives to stimulate cooperation Ability to enhance participation of reluctant respondents and under-represented groups is a plus Ability to enhance participation of reluctant respondents and under-represented groups is a plus Not part of AAPOR disclosure requirements Not part of AAPOR disclosure requirements Not addressed in AAPOR advice on dealing with IRBs Not addressed in AAPOR advice on dealing with IRBs

10 10 5/16/2008 Web Survey of NIH Grantees Doing Surveys Sampling frame: NIH CRISP database of grants Sampling frame: NIH CRISP database of grants Lists about 250 in each year Lists about 250 in each year 482 unique grants in unique grants in Reviewed abstracts: Reviewed abstracts: Included those with mention of survey data collection Included those with mention of survey data collection Excluded methodological studies Excluded methodological studies 145 grants selected for data collection 145 grants selected for data collection

11 11 5/16/2008 Incentives Web Survey Sent invitation to participate and three follow ups to Principal Investigator listed in CRISP Sent invitation to participate and three follow ups to Principal Investigator listed in CRISP Provided link to COPAFS and Incentives Conference web pages Provided link to COPAFS and Incentives Conference web pages Promised copy of the paper as incentive Promised copy of the paper as incentive Allowed PI or designated proxy to fill out the survey Allowed PI or designated proxy to fill out the survey Asked PI to select one survey, the most important in terms of research goals Asked PI to select one survey, the most important in terms of research goals Received 92 responses - 63% Received 92 responses - 63%

12 12 5/16/2008 Most Surveys Used Incentives

13 13 5/16/2008 Why Didnt Surveys Use Incentives? Shorter surveys Shorter surveys 40% under 15 minutes, nearly all under an hour 40% under 15 minutes, nearly all under an hour Mean time to complete 27 minutes vs. 45 minutes Mean time to complete 27 minutes vs. 45 minutes Reasons: Reasons: 60% expected good response without incentives 60% expected good response without incentives 60% did not have budget for incentives 60% did not have budget for incentives 27% survey team did not want to pay incentives 27% survey team did not want to pay incentives None reported IRB was a factor None reported IRB was a factor

14 14 5/16/2008 How Did Surveys Use Incentives? Mix of expectations and timing Mix of expectations and timing 44% for completed survey 44% for completed survey 32% for partial complete 32% for partial complete 13% for considering participation 13% for considering participation 11% prepaid incentive 11% prepaid incentive About half the surveys included tasks other than the survey About half the surveys included tasks other than the survey Half of those provided a separate payment Half of those provided a separate payment

15 15 5/16/2008 Kinds and Amounts of Incentives Kinds of Incentives Kinds of Incentives Cash - 31% Cash - 31% Gift cards or certificates - 27% Gift cards or certificates - 27% Checks - 25% Checks - 25% Amounts of monetary incentives Amounts of monetary incentives $10 or less - 34% $10 or less - 34% $ % $ % $50 or more - 14% $50 or more - 14% Other kinds of incentives Other kinds of incentives Lottery for an iPod or gift certificates, mugs, bags, water bottles, etc. Lottery for an iPod or gift certificates, mugs, bags, water bottles, etc.

16 16 5/16/2008 Why Are Incentives Used? 73% said to increase response rates was main reason 73% said to increase response rates was main reason Rated as main or very important reason: Rated as main or very important reason: Reduce non-response bias - 71% Reduce non-response bias - 71% Reward participants for research participation - 56% Reward participants for research participation - 56% Not rated as important reasons: Not rated as important reasons: Reduce data collection time or follow up costs Reduce data collection time or follow up costs Help interviewers feel more comfortable Help interviewers feel more comfortable IRB wanted incentives IRB wanted incentives

17 17 5/16/2008 What Kinds of Surveys Were These? Potentially Sensitive Topics Potentially Sensitive Topics Health status or health conditions - 74% Health status or health conditions - 74% Personal financial information - 44% Personal financial information - 44% Sexual behavior - 31% Sexual behavior - 31% Drug use or drug use history - 25% Drug use or drug use history - 25% Immigration status - 14% Immigration status - 14% Special requests Special requests Linkage to other databases (e.g. Medicare) 7% Linkage to other databases (e.g. Medicare) 7%

18 18 5/16/2008 Modes Used With Incentives

19 19 5/16/2008 How Were Amounts Determined? Based on open-ended comments: Reflected actual costs of participation, e.g. travel, lost wages, child care, cell phone charges Reflected actual costs of participation, e.g. travel, lost wages, child care, cell phone charges Time and contribution of personal information Time and contribution of personal information Going rate for surveys of this kind of subject Going rate for surveys of this kind of subject Accounting or safety issues for interviewers or subjects or budget issues as constraints Accounting or safety issues for interviewers or subjects or budget issues as constraints Experiments to determine effective amounts Experiments to determine effective amounts

20 20 5/16/2008 What Was the Role of IRBs? For paying incentives or not: For paying incentives or not: Only one participant cited IRB preferences as a main reason for paying incentives Only one participant cited IRB preferences as a main reason for paying incentives 88% said IRB preferences were not important in making this decision 88% said IRB preferences were not important in making this decision For kind or amount of incentives: For kind or amount of incentives: 23% said IRBs raised questions or placed limitations 23% said IRBs raised questions or placed limitations IRBs capped amount or required same incentive for all IRBs capped amount or required same incentive for all Limited how and when incentives could be mentioned Limited how and when incentives could be mentioned

21 21 5/16/2008 Conclusions At least half the NIH sponsored projects that used surveys paid incentives that were substantial - $10-$50 At least half the NIH sponsored projects that used surveys paid incentives that were substantial - $10-$50 75% of telephone surveys paid incentives, range was $ % of telephone surveys paid incentives, range was $15-20 Surveys that paid incentives were longer (mean=45 minutes), involved sensitive topics, and often included additional kinds of data collection or other requests Surveys that paid incentives were longer (mean=45 minutes), involved sensitive topics, and often included additional kinds of data collection or other requests Most paid incentives to increase response rates Most paid incentives to increase response rates IRBs were a factor, but not a major limitation IRBs were a factor, but not a major limitation

22


Download ppt "Use of Incentives in Surveys Supported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December."

Similar presentations


Ads by Google