Research Design and Instrument Development Ensuring what you want is what you get

Objectives to review some key research design issues to highlight key elements of instrument design

Research Technique Technique is determined by objectives Ensure that the data collected is the right type and quality to answer the research question

Research Techniques Experiments Surveys Secondary data Observation

Selecting the Appropriate Survey Design Personal interviewing Telephone interviewing Speed, cost, impersonal, representative sample Greater tendency to get “no answer” and “incomplete answers” No longer than ten minutes; uniform administration Mail survey No longer than six pages Electronic survey

Personal Interviews Open-ended Focused Survey ask for the facts of a matter as well as for opinions about events or insights Focused interviews may still be open-ended and assume a conversational manner, but you’re likely following a set of questions Survey structured questions

General Design Considerations Time frame Explicit and consistent frame of reference Standardization (for comparability) Descriptive and/or evaluative data

Designing Survey Instruments Collect only the data needed to address the research question Be clear about what concepts you want to collect data on and how you are going to ensure that this data is what you think it is Ask the “right” questions about the concepts in the study

Major Decisions What should be asked? How should each question be phrased? In what sequence should the questions be arranged? What questionnaire layout will best serve the research objectives? How should the questionnaire be pretested? Does the questionnaire need to be revised?

Item Phrasing Dependent upon method of administration Nonpersonal administration requires items and instructions to be succinct All methods require items to be clearly stated, unambiguous, easily understood

When Constructing Items, Ask: Is the item necessary? How will it be asked? Should it be open-or closed-ended? How should the item be worded?

When Constructing Response Formats, Ask: What response alternatives should be used? In what order should alternatives be stated? How will “don’t know”, “no answer” and “neutral” responses be handled?

Item Sequencing and Layout Are the instructions clear and easy to read? Are instructions provided to explain the purpose of the questions, with directions on how to complete the scales, if necessary? Does the questionnaire begin with simple and interesting questions? Are sensitive and classificatory items provided last? The questionnaire has a good flow to it. The questionnaire is well-laid out and aesthetically appealing.

Invitation to Participate Explicit invitation to participate An acknowledgement that participation is voluntary A statement of the purpose of the project A statement of the time demands involved in participation Contact information for the researcher (or supervisor).