Survey Training Pack Session 3 – Questionnaire Design

Introduction In this session we will cover: – Linking questionnaire to the objectives and data analysis plan – Types of questions, wording of questions, sequence of questions – Layout of questionnaire: module approach, informed consent and confirming selection – Key identifiers and data handlers

Link to data analysis plan During questionnaire design keep in mind your analysis plan Will you have all the “ingredients” (i.e. data) to produce your plan? How do you test your questionnaire? For each table and graph in your DAP, list the questions from your questionnaire. Are there questions that are not accounted for by your DAP?

Types of questions Different types of questions can be encountered: Questions on quantitative issues with numerical values (e.g. age) Questions on qualitative issues where answers are coded (e.g. sex of trainee) Numerical answers – make sure that you specify the unit of measurement (e.g. completed years) Think through the range of values which are ‘valid’ – important at a later stage Qualitative answers that are coded – mutually exclusive

Open-Ended Questions Exploratory in nature – respondents less constrained Provide rich qualitative data However, lack statistical significance needed for conclusive research Useful for focus groups or studies with smaller populations

Open vs Closed Questions Open-Ended Respondents less constrained More “richness” & “depth” Help identify response options Complex to code Difficult to compare responses More interpretation needed Closed-Ended Easier to code Allow for statistical summaries of large # of cases Reporting more straight- forward Possible response options may be omitted Choices might be forced

Wording of questions Questions must be clear:  Is it specific?  Is the right terminology being used? Will the respondent understand what you are asking? Has the timeframe been specified? For example, does the question pertain to the last 12 months? Have you exhausted all possible response categories? Do these categories make sense in your context?  Are the categories coherent with the question being asked?

Sequence of questions Key principle is making sure that respondent understands what is being asked:  For adoption, best if questions follow the chronological sequence of technologies applied during the season Use of skips to structure the interview:  Some questions may not be relevant depending on the answers provided to preceding questions  Example – no need to inquire about how a tractor or a power tiller was accessed if no tractor and power tiller were used Use of notes for the enumerator to remind of previous answers

Sequence of questions C23 Question for the enumerator: did the respondent plant directly on any of his lowland fields? 1. Yes2. No If 2, go to C28 |___| C24 Did you plant directly in line on any of your lowland fields? 1. Yes2. No If 2, go to C28 |___| C25 Did you plant directly in square lines on any of your lowland fields? 1. Yes2. No If 2, go to C28 |___|

Tips for tool design  Use simple words  Avoid long and complex questions  No double-barrelled questions  Avoid leading questions  No double negatives  Ensure meaning of questions is appropriate to your study population  E.g. in different contexts, different local terms are used for ‘savings groups’; using those terms is best  Do you expect one answer or are multiple answers to your question possible?

Layout of questionnaire The module approach – different topics discussed in a questionnaire:  One module for each broad topic (e.g. all questions related to adoption) OR  One module for each type of respondent/level of data (e.g. household level questions which are targeted at the household head whereas rice adoption questions are meant for the rice trainee; the rice trainee is not necessarily the head of household) Design of questionnaire can be broken down:  Order of modules should match how the interview will be conducted

Informed consent Obtaining informed consent is an important part of the survey process. From experience, it is usually best to integrate the informed consent form in your questionnaire:  Ensures that enumerators formally ask for informed consent  Also easier from a logistical point of view – already attached to the questionnaire  When interviewing different respondents within a single household, ensures that informed consent is asked from all respondents  Space for recording reasons why a respondent refused to participate in a study is essential

Selection confirmation Even though you have carefully developed your sampling strategy, it is important that you take the time to confirm that you are speaking with the right respondent

Formatting Formatting your questionnaire is key – the better it is, the easier for enumerators to fill it out:  Make sure that response categories and relevant question are on the same page  The last column – response column – should be the only space where the enumerator writes answers C2What is the main reason why you had to replant rice on any of your fields? 1. Flood 2. Extended drought/ insufficient rain 3. Poor seed germination 4. Pest attack/disease 97. Other |__|__| If 97 please specify: _________ _________

Key definitions ElementDefinition Unit of analysis The unit of analysis is the unit on which we conduct the analysis (i.e. individual, household, etc.) Sampling unit Related to the unit of analysis and generally the same. In many instances, it may not be possible to directly sample the unit of analysis (e.g. individuals). We may first need to sample higher-level units (e.g. village and household). Stratification Process of dividing the study population into subsets within each of which an independent sample is created

Unique identifiers Clearly and uniquely identify each sampling unit (e.g. village and trainee) 1 st sampling unit: village; stratified by training approach and village classification 2 nd sampling unit: trainee Identifier is a combination of strata and sampling unit

Example identifier Identifier for village Nangomba = 1-MAtT-09

Identifiers on each page

Data handlers One 1 st page of questionnaire: Enumerator/fieldworker Supervisor Data entry operators

Exercise Task #1: Review the 2013 rice adoption survey questionnaire layout – Give a justification for each question – How is each question linked to the data analysis plan? – Does it include everything needed for the DAP? Task #2: watch video presentation on design and layout Task #3: work through practical exercise

Summary Key points:  Link questionnaire to objectives and analysis plan  Collect the data you need; all the data you need, and only the data you need  Questionnaire should be clear, easy to complete and not too long – not more than an hour to administer  The wording and sequence of your questions is an important part of making your questionnaire user- friendly  Don’t forget about informed consent!