Asking Questions Survey design methods and tools

Where to begin? Identify your audience Who needs the information? How will the information be used?

Where to begin? Define the project scope and the desired product What are you trying to learn more about? How should the information be packaged?

Where to begin? Work backwards from the desired product to identify appropriate tools, techniques, and respondents What is the best way to get the information you need? Are there existing sources of information that you could tap into? If a questionnaire is an appropriate tool to use, who is the most appropriate group of respondents, and what is the best way to reach them?

Some guiding principles of questionnaire design A good questionnaire meets the objectives of the survey Clear focus Well aligned with project objectives Need-to-know, not nice-to-know Every piece of data collected will be used

A good questionnaire is fielded to the right respondents Objectives and intended actions are clearly stated Respondents are engaged Privacy is respected, issues of confidentiality and any other ethical matters are addressed up-front Delivery method is appropriate for reaching the desired respondents Some guiding principles of questionnaire design

A good questionnaire has well-designed questions Not leading or biased Clear, plain-language Comprehensive response options No overlapping categories Tested and validated Written with the analyst’s needs in mind so the right information can easily be extracted and summarized Some guiding principles of questionnaire design

Think it through from various angles to try and identify potential sources of bias, such as: Loaded questions Non-representative sample Biased method of delivery Some guiding principles of questionnaire design

Improving response rates People will not feel motivated to respond if: They feel it is a waste of their time They can’t relate to the topic The questions are confusing or frustrating They are suspicious of your motives for collecting the information

Improving response rates State the value of the research Make it relevant to the respondent Use an appropriate, accessible and appealing format Use reminders Be respectful, not pushy Keep it short, and disclose the time required If appropriate and practical, consider using an incentive

Tips and tricks Use questions from other well-designed questionnaires If you plan to triangulate or cross-reference other data sources, design your questions so the data will be compatible Unless it is a qualitative study, keep open-ended questions to a minimum Always run your objectives and questions by other researchers to ensure you’re on the mark (YBS staff are available to help with this!) Always pilot-test your questionnaires to ensure the questions are easy to interpret and they flow properly

Tips and tricks Use objective language in your report Clearly describe the methodology, sample and respondent group Disclose any shortcomings in the methodology Speak to the audience, as it may not have your level of technical expertise on the subject Use a mixture of text and visuals

Knowledge is Power Inquiries: Rachel Westfall Senior Statistician Yukon Bureau of Statistics