STAT 472 Survey Design

Measurement and Research Design There are five suggestion for coming up with a measure There are five suggestion for coming up with a measure

Suggestion.1 Remember the conceptual Definition: Remember the conceptual Definition: The underlying principle for measure is to match it to the specific conceptual definition of the construct that will be used in the study The underlying principle for measure is to match it to the specific conceptual definition of the construct that will be used in the study

Suggestion. 2 Keep an Open Mind Keep an Open Mind Do not get locked into a single measure or type of measure. Be creative and constantly look for better measures, Avoid using one measurement instrument for all problems.

Suggestion. 3 Borrow from others Borrow from others Do not be afraid to borrow from other researchers, as long as credit is given. Good ideas for measure can be found in other studies or modified from other measures.

Suggestion. 4 Anticipate Difficulties Anticipate Difficulties Logical and practical problems often arise when trying to measure variables of interest. Sometimes a problem can be anticipated and avoided with careful forethought and planning

Suggestion. 5 Do not forget your units of analysis Do not forget your units of analysis Your measure should fit the units of analysis of the study and permit you to generalize to the universe of interest

Reliability and Validity Reliability and validity are central issues in all scientific measurement. Reliability and validity are central issues in all scientific measurement. Perfect reliability and validity are virtually impossible to achieve Perfect reliability and validity are virtually impossible to achieve They are ideals researchers strive for. They are ideals researchers strive for. Researchers want to maximize the reliability and validity indicator’s. Researchers want to maximize the reliability and validity indicator’s.

Reliability tells us about an indicator’s dependability and consistency Reliability tells us about an indicator’s dependability and consistency Validity tells us whether an indicator actually captures the meaning of the construct in which we are interested. Validity tells us whether an indicator actually captures the meaning of the construct in which we are interested.

Reliability It means the information provided by indicators ( e.g. questionnaire) does not vary as a result of characteristics of the indicator, instrument, or measurement device itself. It means the information provided by indicators ( e.g. questionnaire) does not vary as a result of characteristics of the indicator, instrument, or measurement device itself.

For Example: Car Speedometer For Example: Car Speedometer If I am driving at a constant slow speed on a level surface, but the speedometer needle jumps from one end to the other, then my speedometer is not a reliable indicator of how fast I am traveling

Types of reliability Stability Reliability: is reliability across time Stability Reliability: is reliability across time It addresses the question: Does the measure or indicator deliver the same answer when applied in different time periods? Use  Test-Retest method

Representative reliability Is reliability across Subpopulations or groups of people Is reliability across Subpopulations or groups of people It addresses the question: Does the indicator deliver the same answer when applied to different people? Use  Independent knowledge about subpopulation or subgroups.

Equivalence Reliability Is reliability across Indicators Is reliability across Indicators It addresses the question: Does measure yield results across different indicators? It addresses the question: Does measure yield results across different indicators? Is used when researchers use multiple indicators (several items in a questionnaire all measure the same construct) Is used when researchers use multiple indicators (several items in a questionnaire all measure the same construct) Use  Split-Half Method or Cronbach’s Alpha