Presentation is loading. Please wait.

Presentation is loading. Please wait.

Questionnaire Design Van Bennekom, Chapter 4.

Similar presentations

Presentation on theme: "Questionnaire Design Van Bennekom, Chapter 4."— Presentation transcript:

1 Questionnaire Design Van Bennekom, Chapter 4

2 Identifying the questions to ask
What are concerns and interests from your Upstream shareholders E.g., monitor service quality to avoid future problems Different managers might have different issues Immediate recipients The direct target, truly reflect their concerns (otherwise, low response rate or wrong answers) Personal interview: interview representatives Focus groups: small group interviews Related literatures, documentations, white papers, reports, etc. Downstream customers Might be the indirect cause, their influence to the immediate recipients Follow your statement of purpose with clear definition of scopes and goals.

3 Identifying the questions to ask
Personal Interviews Defining a set of interview questions that helps elicit the survey purpose Semi-structured questionnaire (creating your questions during the interview) Questionnaire: add more questions, alter the sequences of questions, change wording,

4 Identifying the questions to ask
Personal Interviews How to take the notes Using tape recorder (please check the legal issues) Start with the statement of the interview The best notes are quotes from the interviewee. How many interviewers should we interview? 1 to many Who should conduct the interviews Someone with the interview training Person who will design the questionnaire, conduct the survey

5 Identifying the questions to ask
Personal Interviews How should the interview be sampled Purposive sample (most representative, hard and critical Where should these interviews be conducted Convenient for interviewees Face-to-face is the best How long should these interviews last? 30min-one hour When should we stop When you stop hearing new things

6 Identifying the questions to ask
Personal interview Pros Deep in context Explore unexpected paths Paints the initial picture Cons How structured a questionnaire? How many interviewers By phone or in person length

7 Identifying the questions to ask
Focus group To have a directed discussion with a small group of people The dynamic interaction with groups can bring some issues or ideas that would have been missed or ignored during personal interview. Time efficient How many people should be in a focus group 7-15

8 Identifying the questions to ask
Focus group Who should be invited Purposive sample, attendee covers different fields Who should not be invited Limit the number of observers What is the role of the moderator? Help to bring out all sides Encourage members to discuss/present their ideas Control an overbearing opinion leader What type of questionnaire should be used? Same as personal interview: semi-structured

9 Identifying the questions to ask
Focus group Where should the focus group be held and how should the discussion be captured. A comfortable meeting room Audio taping How many focus groups should be conducted? Until you did not hear new information. How long should it last Two hours, morning is better than afternoon, evening is also good Serving a light snack is always good.

10 Identifying the questions to ask
Focus group Pros Deep in context Explore unexpected paths Interaction among participants Cons Planning & execution critical Good moderator critical Have one central theme How many to invite Whom to invite Time consuming textual analysis

11 Documentary Data Existing documents White papers Related literature

12 Analyzing data Doing content analysis of the textual notes
Boiling down the words into categories, supplemented with a few well-chosen statements Identify patterns: group similar comments to reveal patterns Some free-form comments can address the “why” for a question Literal transcript of every word is not necessary, only capture key points and key focuses. Connect words to interviewee’s background A distillation process

13 Drafting the questionnaire
It is an iterative process: Step 1: Organizing your essential findings from the interviews, group them into logic groups Step 2: Consider three basic types of questions (e.g., demographic, specific, overall) Step 3: Select question formats: scale type (limit the number of different scale types), open-end Step 4: Draft some questions Step 5: Take a break Step 6: Repeat steps 3-5. Step 7: seek other input (look for external reviewers to go through your questionnaire, ask domain experts Step 8: Pilot test your draft questionnaire with a representative sample (critical stage)

14 Elements of a questionnaire
A questionnaire contains several components An introduction Instructions Initial questions Sections and their headings Summary A thank you

15 Elements of a questionnaire
Introduction Personally addressed to the respondent To explain the purpose of the survey Who should complete the survey The expected time to complete the survey Emphasize anonymity, if appropriate What to do with the completed questionnaire When to do the survey (now!) Definition of certain terminology Example: Van Bennekom, p75

16 Elements of a questionnaire
Instructions How to enter answers Explain the scales Example: Van Bennekom, p78

17 Elements of a questionnaire
Initial questions Pay attention to the initial questions (engage them, subject matter, motivation to move ahead, easy to answer) Never open with demographic questions in the beginning

18 Elements of a questionnaire
Section headings Should be short Group 50 questions into 3 section vs. put all 50 questions in one section.

19 Elements of a questionnaire
Summary and Thank you

20 Sequencing questions From general to specific Demographic questions
Initial questions should be general Demographic questions Do not ask questions which data you do not need Question interaction Rating a person, rather than a system Connection of different questions Rotating questions Branching: direct respondents to the right sections

21 Question formats Unstructured Open-ended, free-form
Limited the number of open-ended questions (respondent burden) Post them after certain structured questions about key theme, or ask them to clarify or expand on previous answers

22 Question formats Structured (see page 85)
multiple choice or categorical, ordinal scales, interval-rating scales, ratio scales

23 Question formats Structured: multiple choice or categorical,
For frequency distribution (e.g., job title, company size) Kinds: Multiple choice, multiple response Multiple choice, single response Binary choice Adjective checklist (page 90) Design issues: Range of choices in response sets (include other category) Distinctions among choices Number of choices (max 6-8) Sequencing of choices

24 Question formats Structured: Ordinal scales (page 93)
Basic ordinal scale: age range, or salary range Forced-ranking scale: ask respondent to rank some items Paired comparison Design issues Number of items Clarify of instructions on what the rank means

25 Question formats Structured: Interval-rating scales (page 95)
Such as: level of agreement Verbal scales: scale in text Numeric scales: scale in number Likert-type scale: strongly disagree-disagree-neither agree nor disagree-agree-strongly agree Many others (page ) Design issues Advantages of grouping by type Which step first, developing scale or writing questions? Importance of anchor choice Scale direction (1 is best or 5 is best?) Interval equality Anchor balancing (extremely dissatisfied, vs. very satisfied) Even (1-7) vs. odd (1-6)-numbered scales Number of points on the scale: 10 points are too much. Presenting the scale low to high or high to low Use of multiple scale types creates confusion Avoiding response ruts Dispersion of results

26 Question formats Structured: Ratio scales (page 108)
Fractionation scale: open-ended scale, no upper limit, has better precision.

27 General guidelines Goal Do not ask meaningless questions
Maximizing the information we receive Minimizing response burden Do not ask meaningless questions Keep the instrument focused A survey is not a test (instruction should be clear) Be concerned about the visual and layout of your questionnaire Keep wording and grammar simple Be creative in question design

28 Pilot testing Don’s skip it It is not complicated
It should be conducted as one-on-one or personal interview on purposive sampled targets The purposes are To know whether questions are well designed, difficult to answer, or have proper response choices Engage the respondent?, too long? Clarity of instructions, find awkward wording

Download ppt "Questionnaire Design Van Bennekom, Chapter 4."

Similar presentations

Ads by Google