QUESTIONNAIRE DESIGN: TYPE OF QUESTIONS Lu Ann Aday, Ph.D. The University of Texas School of Public Health

MAJOR RESPONDENT TASKS: DIFFERENT TYPES OF Qs Major Tasks/ Types of Qs Comprehen- sion RetrievalJudgmentResponse Health X Demographics XX Behavior- Nonthreatening XX Behavior- Threatening XX Knowledge XX Attitudes XX

DESIGN ELEMENTS & CRITERIA: Health ELEMENTS Words CRITERIA Clearly define the concept Physical, mental, or social dimensions Positive vs. negative health Provider vs. patient judgments

DESIGN ELEMENTS & CRITERIA: Health ELEMENTS Responses CRITERIA Match scale type to design & analyses Design: precision & sensitivity Analyses: typology, Likert, utility, etc. scale types

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Household composition EXAMPLES Ask about relationships to reference person (who owns or rents the home) rather than “head of household”

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Age EXAMPLES What is your date of birth? In what year were you born? Which of the following categories best describes your age?

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Marital status EXAMPLES Are you currently married, not married but living with a sexual partner, separated, divorced, widowed, or never married?

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Ethnicity Race EXAMPLES Are any of the following groups your national origin or ancestry? Which group or groups BEST represents your race?

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Education EXAMPLES What is the highest grade or year of regular school that you completed?

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Employment status EXAMPLES Are you currently employed? IF YES, ASK, Are you self- employed & Is that full-time or part-time? IF NO, ASK, Are you retired, disabled, a student, keeping house, temporarily unemployed, or not looking for paid employment?

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Occupation EXAMPLES What is your job title? What are your most important job activities or duties? What kind of business or industry is this?

DESIGN ELEMENTS & CRITERIA: Demographics VARIABLE Family Income EXAMPLES Which of the following categories best represents your total combined FAMILY income during the last 12 months? Include money from jobs, social security, retirement income, unemployment payments, public assistance and so forth. Also include income from interest, dividends, net income from business, farm, or rent, and any other money income received.

DESIGN ELEMENTS & CRITERIA: Behavior—Nonthreatening ELEMENTS Administration Overreporting (telescoping salient events) Underreporting (omitting non- salient events) CRITERIA Prime R to remember Use bounded recall: ask about events in previous & then current time period Adjust recall period Ask in terms of “usual” behavior Use memory aids, e.g., aided recall, records, diaries

DESIGN ELEMENTS & CRITERIA: Behavior—Threatening ELEMENTS Words Phrases Sentences CRITERIA Use words that are familiar Judiciously consider “loading” questions, e.g., others engage in behavior Make questions longer

DESIGN ELEMENTS & CRITERIA: Behavior—Threatening ELEMENTS Responses CRITERIA Consider open- ended rather and closed-end responses

DESIGN ELEMENTS & CRITERIA: Behavior—Threatening ELEMENTS Questionnaire CRITERIA Build in appropriate explanations & transitions Ask whether they “ever” engaged in behavior before asking about “current” practices Ask questions at end or embed among less threatening items

DESIGN ELEMENTS & CRITERIA: Behavior—Threatening ELEMENTS Administration CRITERIA Consider a self- administered or more anonymous, e.g., computerized, format, to minimize “threat”

DESIGN ELEMENTS & CRITERIA: Knowledge ELEMENTS Questions CRITERIA Consider screening Q first to see if R knows anything about the topic Ask more than one question to find out about knowledge Phrase Qs more like Qs about opinions, e.g., “In your opinion…”

DESIGN ELEMENTS & CRITERIA: Knowledge ELEMENTS Responses CRITERIA Use open-ended rather than closed- end response formats when necessary, to prevent guessing Provide “don’t know” as an alternative

DESIGN ELEMENTS & CRITERIA: Knowledge ELEMENTS Administration CRITERIA Use self- administered quex, esp. in groups Consider inclusion of “sleeper” (fictional) options

DESIGN ELEMENTS & CRITERIA: Attitudes ELEMENTS Words Phrases CRITERIA Clarify the attitude object (or focus of attitude statement) Provide balanced alternatives, e.g., agree or disagree; support or oppose Do not included double- barreled (more than one) referent

DESIGN ELEMENTS & CRITERIA: Attitudes ELEMENTS Sentences CRITERIA Use medium- length, followed by medium-length to long Q Limit the number of Qs included in a battery of similar Qs

DESIGN ELEMENTS & CRITERIA: Attitudes ELEMENTS Responses CRITERIA Minimize “yea-saying” Include positive & negative items Put least socially desirable response first Choose best approach to measuring attitude strength Rating vs. ranking # of points on scale Use of “uncertain”

DESIGN ELEMENTS & CRITERIA: Attitudes ELEMENTS Administration CRITERIA Ask more general attitudinal items before specific ones

SURVEY ERRORS: Formulating Specific Types of Questions Systematic ErrorsVariable Errors Questionnaire effects: under/ overreporting Respondent effects: social desirability bias Respondent effects: yea saying (acquiescent response set) Mode effects: social desirability bias Questionnaire effects: order & context Solutions to errors Employ cognitive question & questionnaire design and evaluation procedures, and use bounded recall and memory aids, to improve the accuracy in answering nonthreatening questions about behavior. Employ cognitive question & questionnaire design and evaluation procedures, to enhance the honesty in answering threatening questions about behavior. Reduce the length of a series of questions (or use a short-form of scales) measuring knowledge or attitudes. Consider using more anonymous modes of data collection. Conduct split- ballot experiments to evaluate how answers to questions vary when they appear in a different order in the questionnaire.

REFERENCES Dillman, Don A. (2000). Mail and Internet Surveys: The Tailored Design Method. Second Edition. New York: John Wiley & Sons, Inc. Ware, J.E., Jr., & Gandek, B., for the IQOLA Project (1998). Overview of the SF-36 Health Survey and the International Quality of Life Assessment (IQOLA) Project. J. Clinical Epidemiology, 51 (11),