Presentation on theme: "Presented at the 2006 American Educational Research Association Annual Meeting April 7 – 11 San Francisco, CA Techniques for Generating Higher Response."— Presentation transcript:
Presented at the 2006 American Educational Research Association Annual Meeting April 7 – 11 San Francisco, CA Techniques for Generating Higher Response Rates Presented at the 2006 American Educational Research Association Annual Meeting April 7 – 11 San Francisco, CA Kevin Kenward, PhD Director of Research National Council of State Boards of Nursing Lynn C. Webb, Ed.D. Testing Consultant
Increasingly, [people] are refusing to participate in consumer surveys. A truly representative survey may be unattainable… The New York Times June 2004
Status of Survey Nonresponse Falling cooperation rates in the developed world (de Leeuw and de Heer, 2002) Greater decline in one-time surveys vs. longitudinal surveys (AAPOR, 2003) Greater decline in unit nonresponse than item nonresponse
An electronic survey was conducted to compare the roles of two types of Advanced Practice Registered Nurses Nurse Practitioners (NP) Clinical Nurse Specialists (CNS) Job Analysis
The professional sophistication of Advanced Practice RNs led us to believe that an electronic survey would be appropriate, and the electronic format offered advantages over paper.
The history of nurse response rates 40% led us to believe we would have strong participation.
The response rate was disappointingly low. 13% In desperation, nurses were randomly selected from the mailing list and asked why they did not participate in the study. TOO BUSY was the recurring theme in the explanations offered.
How do We Get People to Complete Our Surveys and Provide us with Good Information ??
Techniques for Generating Higher Response Rates l Preliminary or Advance Notification l Follow-ups l Appeals l Postage and Mailing Classes l Personalization l Incentives
Decreasing Response Rates QUESTIONNAIRE LENGTH - MAIL 4 PAGES OR LESS No Effect MORE THAN 4 PAGES - 8%
MONETARY INCENTIVES Prepaid - Creates Trust and Obligation Amount - Symbolic Token of Appreciation Understanding the Techniques
Phone Call Reminder Mail Survey Long Form Version 1 Mail Survey Long Form Version 2 Mail Survey Short Form N=1,500 N=1,000
Mail Survey Long Form Version 1 + Photo on Cover Letter Mail Survey Long Form Version 1 + Photo on Cover Letter
Mail Survey + $1 Incentive Mail Survey + $1 Incentive Mail Survey + $2 Incentive Mail Survey + $2 Incentive Mail Survey + $5 Incentive Mail Survey + $5 Incentive Monetary Incentives N=250
Mail Survey + Starbucks $3 Gift Certificate Mail Survey + $250 Drawing N=250 N=1,000
2% 26% Results
Mail Survey Long Form Version 1 Mail Survey Long Form Version 2 Mail Survey Short Form 24% 26% 30% Results
Effective Nonresponse Rate Reduction Methods Incentives reduce nonresponse rates higher effects with increasing incentives prepaid incentives more powerful than promised cash more powerful than in-kind incentives can reduce total survey costs
Sample Size Mailing cost Response Rate Cost per Complete 10,000 $14,000 $5 Incentive No Incentive $64,000 14%54% $10.00$11.85