A.N. Oppenheim, Questionnaire Design Questionnaires A.N. Oppenheim, Questionnaire Design

Questionnaires FOCUS What is the focus of your questionnaire?

Questionnaires The focus of your questionnaire is your research question All questions in your questionnaire need to be relevant to your research question.

Questionnaires To help further with the focus identify intended audience for your questionnaire – is it a particular organisation, a particular group of people, or everyone?

Questionnaires ELEMENTS What are the parts of a questionnaire?

Questionnaires PREAMBLE Explain what the survey is for “This survey sets out to ...” Reassure the participants “Please note if you fill in this questionnaire, your answers will be treated in highly confidential way. Neither I, the Dublin Institute of Technology nor any other third part will identify your name, email address or any other personal details, nor will it be possible to identify you in any way in the report I will publish as part of my MSc dissertation. I would like to personally thank you for your time in taking part in this survey.”

Questionnaires TYPES OF QUESTIONS Open-ended Close-ended Combination of both

Questionnaires Open-ended Close-ended

Questionnaires Open-ended Slower to administer Harder to record responses Does not stifle response Answerer can raise new issues Answerer feels they can speak their mind What does a blank answer mean ?

Questionnaires Close-ended Faster to administer Easier to record responses Answerer can only give predefined answers Answerer cannot raise new issues Answerer feels constrained More likely to answer all questions (box tick)

Questionnaires LIKERT SCALES A special type of close-ended question is where you get the respondent to classify their responses on a scale of 1 to 5 (or in any five ordinal categories).

Questionnaires LIKERT SCALES Poor, Weak, O.K., Good, Excellent Very Low, Low, O.K., High, Very High 1, 2, 3, 4, 5

Questionnaires LIKERT SCALES To make life easier for yourself, you don’t need to label each of the five values if you don’t need to. Very Bad Very Good 1 2 3 4 5

Thank them CLOSING REMARKS "Thank you for taking the time to participate in this survey"

Questionnaires DESIGN How will I design the questions?

Questionnaires Think clearly about questions (need to constrain answers as much as possible) Make sure results will answer your research question You can use the Internet for conducting surveys if you need to cover wide geographic reach

Questionnaires Check your grammar, twice It’s a proofreaders' Rule of Thumb that if you read something twice, it gets rid of 95% of errors.

Questionnaires Keep questions short and simple Avoid questions with “not” Avoid questions with bias Avoid sensitive questions (ask indirectly) Do not ask compound questions, just ask one question at a time e.g. "Do you know what services are available to you and how to find out?"

Questionnaires DESIGN How will I design the questions?

Questionnaires Before you deploy to participants, deploy a few pilot versions of the questionnaire to friends, family and fellow students. Correct any complicated, misstated, or ambiguous questions

Questionnaires Deployment Approaches Self-administered Interviewer administered

Questionnaires Self-administered Means you give the questionnaire to the participants and they fill it out in their own time, at their own pace. This approach can allow you to reach large audiences, e.g. by deploying over the Internet. The problem is that if they don’t understand a question, they will probably guess what it means, and that answer with be invalid.

Questionnaires Interviewer administered Means you are present when participant or participants are filling out the questionnaire. If you are dealing with an individual or group you can instruct them how to fill out the questionnaire; “I want everyone to start with question 1, spend about 30 seconds on that ... O.K. now move onto the next question, etc.” If they don’t understand a question, they can ask you, and you can tell the individual or group.

Incentives Survey research suggests that there is a chance that offering an incentive may add bias to a survey e.g. Singer, E., Bossarte, R., 2006, “Incentives for Survey Participation: When Are They “Coercive”?, American Journal of Preventive Medicine, Volume 31, Issue 5, Pages 411-418

Incentives More refs: http://www.ijis.net/ijis1_1/ijis1_1_goeritz.pdf http://www.nyu.edu/gsas/dept/politics/faculty/morton/Incentives_24June06.pdf http://www.iser.sx.ac.uk/publications/working-papers/iser/2007-01.pdf http://www.isr.umich.edu/src/smp/Electronic%20Copies/51-Draft106.pdf http://www.nber.org/chapters/c10111.pdf

Questionnaires EXAMPLES Can I see a few examples of questions?

Questionnaires The following is an example of a questionnaire that was developed by a student who was comparing text-based information with a visual representation of the same information >>>

Questionnaires

Questionnaires What do you think of the questions on the following slide >>>

Questionnaires

Over-Specified Answers Over-Specified Answers Questionnaires Unclear Question and Over-Specified Answers Compound Question and Over-Specified Answers Asymmetric Answers

Other Useful Approaches

Focus groups Take time to arrange, so prepare in advance (use an intermediary to help you if you can) Who will be in your focus group? (e.g. age, gender) Size of focus group (8-10 is typical) Consider whether or not to have separate focus groups for different ages or genders (e.g. discussing sex and sexuality)

Site visits and observation Site visits involve visiting an organization, community project etc Consider using a guide Observation is when you visit a location and observe what is going on, drawing your own conclusions Both facilitate making your research more relevant and concrete

Case studies Method of capturing and presenting concrete details of real or fictional situations in a structured way Good for comparative analysis

Participatory research Allows participation of community being researched in research process (e.g. developing research question; choosing methodology; analysing results) Good way to ensure research does not simply reinforce prejudices and presumptions of researcher Good for raising awareness in community and developing appropriate action plans