1. Lecture II Questionnaire Design and Issues Dr. Sutarman, M.Sc
University of Sumatera Utara

2. Four Theoretical Components of a Questionnaire
Words:The researcher must carefully consider which words to use in creating the questions and scales for collecting raw data from respondents.
Questions/Setups:
Unstructured questions:
Open-ended questions formatted to allow respondents to reply in their own words.
Structured questions:
Closed-ended questions that require the respondent to choose from a predetermined set of responses or scale points.

3. Bad questions
Any questions that prevent or distort the fundamental communication between the researcher and the respondents.
Incomprehensible to the respondent because the wording, the concept, or both cannot be understood.
Example: “What is your attitude about the linkage between the 2002 war on the terrorists of al-Quaida in Afghanistan and the Democrats decrying of sexual McCarthyism toward improving the environment in Arizona?”

4. Unanswerable either because the respondent does not have access to the information needed or because none of the answer choices apply to the respondent.
Example: “What was your parents’ exact annual income two years ago?”
Leading (or loaded) in that the respondent is forced or directed into a response that she or he would not ordinarily give if all possible response categories or concepts were provided, or if all the facts of the situation were provided.
Example: “Do you believe that Democrats who loved William Jefferson Clinton agreed he did a good job as president of the United States?”

5. Double-barreled in that they ask the respondent to address more than one issue at a time.
Example: “To what extent do you agree or disagree that Monica Lewinsky and Representative Henry Hyde, R-Ill., were responsible for the impeachment vote against President Clinton?”
Questionnaire Format
The questionnaire’s format should allow for clear communication.

6. Hypothesis: A formalized statement of a testable relationship between two or more constructs or variables.
1. The nature of the respondent.
2. The relationship between the expressed attitudes and behavior of the respondent (e.g., motivation).
3. The sociological structures and their influence on the respondent.
4. The meaning of words and the respondent’s grasp of language and/or concepts.
5. The relationships among a respondent’s knowledge, attitudes, and marketplace behaviors.
6. The descriptive and predictive capabilities of attributes and dimensions with regard to the investigated constructs and concepts (e.g., customer satisfaction, product or service quality, and behavioral intentions)

7. Description versus Prediction
A descriptive design allows the researcher to collect raw data that can be turned into facts about a person or object. For example, the U.S. Census Bureau uses questionnaire designs that collect primarily state-of-being or state-of-behavior data that can be translated into facts about the U.S. population (e.g., income levels, marital status, age, occupation, family size, usage rates, quantities of consumption).
Predictive questionnaires force the researcher to collect a wider range of state-of-mind and state-of-intention data that can be used in predicting changes in attitudes and behaviors as well as in testing hypotheses.

8. Accuracy versus Precision
Accuracy refers to the degree to which the data provide the researcher with a description of the true state of affairs.
precision in questionnaire designs focuses on the reproducibility of the results over repeated usages (i.e., similar to reliability of scale measurements).

9. Examples of Different Types of Hypotheses Used in Information Research

10. Steps in the Development of Survey Instruments

11. Steps in the Development of Survey Instruments (cont’)

12. Steps in the Development of Survey Instruments (cont’)

13. Flowerpot approach A specific framework for integrating sets of
question/scale measurements into a logical, smooth-flowing questionnaire.
Helps the researcher make decisions regarding
construct development,
dimensions and attributes of objects,
Various question/scale measurement formats,
Wording of actual questions, and
Scale points.

14. Flowerpot approach (Cont;)

15. Guidelines for Evaluating the Adequacy of Questions
Questions should be simple and straightforward whenever possible.
Questions should be expressed clearly whenever possible.
Questions should avoid qualifying phrases or extraneous references, unless they are being used as a type of qualifying (screening) factor.
Descriptive words should be avoided, unless absolutely necessary.
The question/setups, attribute statements, and data response categories should be unidimensional, except when there is a need for a multiple-response question.
Raw data response categories (scale points) should be mutually exclusive.
The question/setups and the response categories should be meaningful to the respondent.
Question/scale measurement formats should avoid arrangement of response categories that might bias the respondent’s answer.
Unless called for, question/setups should avoid undue stress of particular words.
Question/setups should avoid double negatives.
Question/scale measurements should avoid technical or sophisticated language, unless necessary.
Where possible, question/setups should be phrased in a realistic setting.
Question/scale measurements should be designed to read logically.
Question/scale measurements should always avoid the use of double-barreled items.

16. Summary of Important Considerations in Questionnaire Designs
Determine the information objectives and the number of information flowerpots required to meet those objectives.
Determine the specific data requirements (i.e., the size) for each information flowerpot, and stack the pots from largest to smallest.
Introduction section should include a general description of what the study is pertaining to; this may well be in a disguised format.
All types of instructions, if necessary, should be given clear expression.
Perhaps most important, the question/scale measurements have to follow some logical order—that is, an order that appears logical to the respondent rather than to the researcher or practitioner.
Begin an interview or questionnaire with simple questions that are easy to respond to, and then gradually lead up to the more difficult questions. Create a general-to-specific data flow.

17. Summary of Important Considerations in Questionnaire Designs (Cont’)
Postpone highly personal questions (state-of-being data) until late in the interview or survey (i.e., place in the base after the last information flowerpot).
Place questions that involve psychological tests (i.e., lifestyle beliefs) toward the end of the interview or survey, but before the identification base.
Do not ask too many questions of the same measurement format (i.e., nominal, ordinal, interval, ratio scale formats) in sequence.
Taper off an interview or survey with a few relatively simple questions that do not require extensive thoughts or expressions of feelings (i.e., the demographic data questions are very appropriate here).
Always end the interview or survey with the appropriate thank-you statement.

18. Guidelines for Developing Cover Letters Factors

19. Example of Supervisor Instructions for a Retail Bank Study Using Personal Interviews

20. Example of Interviewer Instructions for a Retail Bank Study Using Personal Interviews

21. Example of Interviewer Instructions for a Retail Bank Study Using Personal Interviews (cont’)

22. Screening forms
A set of preliminary questions that are used to determine the eligibility of a prospective respondent for inclusion in the survey.

25. Rating cards
Cards used in personal interviews that represent a reproduction of the set of actual scale points and descriptions used to respond to a specific question/setup in the survey. These cards serve as a tool to help the interviewer and respondent speed up the data collection process

26. Call record sheet
A recording document that gathers basic summary information about an interviewer’s performance efficiency (e.g., number of contact attempts, number of completed interviews, length of time of interview).

27. Example of the Question/Scale Format and Rating Card Used in Collecting Raw Data in a Retail Banking Survey