Presentation on theme: "Use of Incentives in Surveys Supported by Federal Grants"— Presentation transcript:
1Use of Incentives in Surveys Supported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, andMegan Zander-CotugnoCDC PRAMS National MeetingDecember 9, 2008
2OverviewWhat we know about incentives from the survey methods literatureIRB issues in providing incentivesResults of a survey of NIH grant recipients who planned to do surveysConclusions
3Survey Literature Findings on Use of Financial Incentives Incentives improve response across modesEffects appear more or less linear, size mattersPrepaid incentives more effective than promisedLotteries less effective than paymentsLower income respondents more responsive than higher income respondentsIncentives can help increase representativeness of the sample
4How Financial Incentives Fit Into Survey Response Respondents consider “costs” of survey participationTime, inconvenienceLoss of privacy, concerns about other possible consequencesHow it will feel to be interviewed or complete the surveyHow others might view them for taking partWhether the purpose of the survey is a potential loss for them
5Non-Financial Reasons for Survey Participation Possible “benefits” other than paymentBelief that the survey will be beneficial to them or to others, desire to helpInterest in the survey, desire to talk about the topicHelping or being associated with a survey sponsorBelief that participating in the survey will be pleasantDesire to help and/or talk with interviewerPrestige from being a study participant
6Institutional Review Boards All federally supported surveys should be reviewed by an IRBIRBs operate under guidelines from DHHS Office for Human Research ProtectionsRequirements for approval:Risks are minimizedSubject selection is equitable, vulnerable populations protectedInformed consentProtection for subjects’ privacy and data confidentialityEach IRB can and does interpret guidelines in light of local circumstances
7IRBs Wrestle With Incentives Do they “coerce” people who would not otherwise freely agree to participate in research to do so?How should the level of incentives be determined?As “wages” for time and effort of participation?Based on the value researchers place on participation?Equitably for all participants or recognizing different values participants may place on their effort and time?
8Special Problems with Incentives for IRBs LotteriesProvide unequal rewards across participantsUndermine informed decision since chances of winning are overvaluedDisadvantaged populationsEspecially vulnerable to coercionMay be induced to lie or conceal information in order to participateHigher payments for people who refuse to participate
9Survey Literature on Incentives Does Not Address Ethical Issues Focus is on practical concern with effectiveness in terms of response rates and/or MSE“AAPOR Best Practices” suggests considering use of incentives to stimulate cooperationAbility to enhance participation of reluctant respondents and under-represented groups is a plusNot part of AAPOR disclosure requirementsNot addressed in AAPOR advice on dealing with IRBs
10Web Survey of NIH Grantees Doing Surveys Sampling frame: NIH CRISP database of grantsLists about 250 in each year482 unique grants inReviewed abstracts:Included those with mention of survey data collectionExcluded methodological studies145 grants selected for data collection
11Incentives Web SurveySent invitation to participate and three follow ups to Principal Investigator listed in CRISPProvided link to COPAFS and Incentives Conference web pagesPromised copy of the paper as “incentive”Allowed PI or designated proxy to fill out the surveyAsked PI to select one survey, the “most important” in terms of research goalsReceived 92 responses - 63%
12Most Surveys Used Incentives Reason for no survey did not include anything to do with incentives.
13Why Didn’t Surveys Use Incentives? Shorter surveys40% under 15 minutes, nearly all under an hourMean time to complete 27 minutes vs. 45 minutesReasons:60% expected good response without incentives60% did not have budget for incentives27% survey team did not want to pay incentivesNone reported IRB was a factor
14How Did Surveys Use Incentives? Mix of expectations and timing44% for completed survey32% for partial complete13% for considering participation11% prepaid incentiveAbout half the surveys included tasks other than the surveyHalf of those provided a separate payment
15Kinds and Amounts of Incentives Kinds of IncentivesCash - 31%Gift cards or certificates - 27%Checks - 25%Amounts of monetary incentives$10 or less - 34%$ %$50 or more - 14%Other kinds of incentivesLottery for an iPod or gift certificates, mugs, bags, water bottles, etc.
16Why Are Incentives Used? 73% said to increase response rates was main reasonRated as main or very important reason:Reduce non-response bias - 71%Reward participants for research participation - 56%Not rated as important reasons:Reduce data collection time or follow up costsHelp interviewers feel more comfortableIRB wanted incentives
17What Kinds of Surveys Were These? Potentially Sensitive TopicsHealth status or health conditions - 74%Personal financial information - 44%Sexual behavior - 31%Drug use or drug use history - 25%Immigration status - 14%Special requestsLinkage to other databases (e.g. Medicare) 7%
19How Were Amounts Determined? Based on open-ended comments:Reflected actual costs of participation, e.g. travel, lost wages, child care, cell phone chargesTime and contribution of personal informationGoing rate for surveys of this kind of subjectAccounting or safety issues for interviewers or subjects or budget issues as constraintsExperiments to determine effective amounts
20What Was the Role of IRBs? For paying incentives or not:Only one participant cited IRB preferences as a main reason for paying incentives88% said IRB preferences were not important in making this decisionFor kind or amount of incentives:23% said IRBs raised questions or placed limitationsIRBs capped amount or required same incentive for allLimited how and when incentives could be mentioned
21ConclusionsAt least half the NIH sponsored projects that used surveys paid incentives that were substantial - $10-$5075% of telephone surveys paid incentives, range was $15-20Surveys that paid incentives were longer (mean=45 minutes), involved sensitive topics, and often included additional kinds of data collection or other requestsMost paid incentives to increase response ratesIRBs were a factor, but not a major limitation