1 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation ONLINE SURVEYS

2 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Online surveys are relatively new – since the mid 1990’s Why are they so popular? –They are easy to develop and implement –They are relatively low cost –You can access large #s of people at low marginal costs

3 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation But, is it the best method? Shall I use an online survey?

4 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Things to think about before you decide on an online survey Do your potential respondents have easy access to the internet? Are they comfortable using the internet? Do they trust internet communications and interactions? Do they have the skills to open and complete the online survey? Will their hardware and software affect how the questionnaire will be seen?

5 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation What do we know about internet access and use? We know that different audiences (age, cultural background, etc) interact with technology differently 80% of American homes have personal computers but only 57% have access to high-speed internet –Issue of coverage and speed Access and use is affected by –Educational level –Household income (56% of homes with combined incomes of less that $40k do not have internet

6 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation About internet access and use… Access and use is affected by –Age: year olds - 78% have internet –Nonwhites have lower access –Many who use internet do so only at work (there may be restrictions on use); at school Comfort and safety –Lack computer skills; familiarity with internet –Distrust of internet –Fear viruses and other malicious software Source: Thomas, 2009, wwww.thefreelibrary.com

7 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation This means that… Many people may be left out when using an internet survey Those who respond may differ considerably from those who do not respond = bias in results

8 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation So, who do you want to hear from? What do you want to know? Is a survey the best method? Is an online survey the best type of survey to use? Remember, your choice of data collection method depends upon your evaluation purpose and be appropriate for the audience.

9 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Online survey tools – what they can do: Help you define your survey questions Provide possible response options Send your respondents to a link to answer the survey online Allow you to customize the look and feel of the questionnaire Provide for skip logic Help you with analysis: most provide simple reports that summarize answers to each question; some let you download into Excel; more advanced packages allow for more complex analyses

10 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Online survey tools There are many popular tools for conducting online surveys including SurveyMonkey Zoomerang SurveyGizmo Checkbox LimeSurvey Each has its strengths and weaknesses. Do a Google search to learn more about each. For a comparison, see Survey of Survey Tools, the OSU Web Accessibility Center at state.edu/workshops/survey_of_surveys/

11 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Go to CE Technology Services at Follow their instructions; OR, obtain your own online survey tool and implement your surveys yourself. So, you’ve decided that an online survey is best. For UWEX Cooperative Extension employees, Computer Technology Services will help you with

12 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Planning your online survey Planning an online survey is similar to planning any type of survey. There are a series of steps to consider: 1.Involve others – who should be involved in planning and conducting the survey? 2.Define what information you will collect – what do you want to know? 3.Determine respondents – get addresses –Set sampling strategy, if appropriate

13 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Steps continued… 4.Plan for data analysis - What do you want to be able to report? Do you want percentages, comparisons, words? 5.Develop the questionnaire 6.Pilot test the questionnaire, cover letter, etc. 7.Design the analysis plan 8.Plan for communications 9.Set budget, timeline and responsibilities

14 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation A note on anonymity and confidentiality in online surveys Anonymous means that NO ONE can identify who provided the information –This is difficult to assure when the survey is administered online (internet or intranet)…so, don’t promise anonymity! Confidentiality means that you are able to identify the person but you guarantee that the information will not be identified with the person –This applies to all aspects of data collection, analysis and reporting –When reporting and communicating, ensure that no names or other identifying information is used

15 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation A few tips Follow same pre-survey and follow-up processes as for other surveys Use for the pre-survey notification and post-survey follow-up; add other methods (telephone) Personalize the message vs group message or group address

16 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Online survey implementation steps Personalize all contacts as much as possible Consider incentives – improves response rate Use multiple contacts and vary the message Strategically time the contacts Use another mode to contact respondents if possible Keep contacts short and to the point

17 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Online survey implementation steps, cont… Make sure is not flagged as spam Make sure your name and address are clear/reputable; Include an informative subject line Provide clear instructions for how to access the survey Make sure the opening screen informs respondents that they are in the proper place and describes steps to take

18 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Online survey implementation steps, cont… Assign each respondent a unique ID number Keep within the limits of the web server’s capabilities Create a system for dealing with bounced s Establish procedures for dealing with inquiries and issues Monitor progress and early responses

19 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Two things we want to accomplish with our online survey 1.To get the highest response possible 2.To get as accurate information as possible So, you need to pay attention to:  Increasing response rate  Good questionnaire design

20 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Increasing response rate There is a separate powerpoint slide set for “increasing your response rate”. Please find it on the same web site.

21 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Response rate may be affected by how the survey is distributed. In zoomerang, there are two options: 1.Use addresses. you need individual names with current addresses. Send invitation plus other communications - reminders, track respondents, and receive detailed deployment information. –This is the only option that allows you to calculate the response rate 2.Post the url in a public place – when you don’t have individual addresses. E.g., in newsletter, on web site, bulletin board; or send to an list (not individual addresses). –You may not know who sees/received the url – you do not know the total number of people who received the questionnaire.

22 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Questionnaire design There is a separate powerpoint slide set and various materials to help with creating the questionnaire for your online survey. Go to the section on this web site, Common methods for collecting data

23 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Checking in… How would you rate your comfort level NOW in being able to develop and implement a useful online survey? Very poor Poor Fair Good Very good

24 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Resources Dillman, D., Smyth, J., Christian, LM Internet, Mail and Mixed-Mode Surveys: The tailored design method. John Wiley and Sons.