Measurement  The process whereby individual instances within a defined population are scored on an attribute according to rules Usually given a numeric score  Measurement is meant to make comparisons among individual cases easier, more precise and more accurate

Measurement  The easiest examples to understand involve the measurement of physical properties of objects Height Weight Volume  Measuring intangibles such as opinion, intent, beliefs, etc. is difficult and open to error

Measurement issues  Validity: does the measure actually reflect the underlying concept? Are you measuring what you intend to measure? Accuracy  Reliability: does the measure perform the consistently from one occasion to another?  Precision/sensitivity: how large are the differences between adjacent categories/scores?  Efficiency: cost versus value of information

Self-report measures  Memory Forgetting Bias  Social desirability Respondent may believe a given answer is more acceptable to the researcher  Knowledge Respondents may not know or understand the ideas Often, respondents will answer questions without really knowing what they are about

Self-report measures  Sensitive to mood of the respondent  Sensitive to interview situation  Sensitive to data collection method Internet Interview Paper and pencil

Tests as measures  Pretty much all tests of your knowledge would fall under the category of self-report measures  All tests are prone to some level and type of error  Many types of tests exist, each having its strengths and weaknesses Multiple-choice Fill in the answer Essay Apply the concept

The GRE  FairTest analysis FairTest analysis  ETS analysis ETS analysis

Observation  Internal states are only partially reflected in observable behavior, etc.  Behaviors are influenced by the situation, which may not be evaluated  May know they are being watched and change their behavior  Observer may engage in biased perception, interpretation, etc.

Mechanical observation  Manifest behavior may not really reflect the underlying concept you think it does Skin conductance Website ‘hits’  Often intrusive to the point of being unnerving Eye tracking

Eye Tracking Source: Max Planck Institute at:

Measurement levels  Differences in ‘measurement level’ refer to the kind of information conveyed in the scores individual objects receive Four levels: nominal, ordinal, interval, ratio More advanced levels provide a greater amount of information with the score they assign More advanced levels allow for more powerful statistical analysis of the data

Nominal-level measurement  Numbers are assigned to individual objects simply as a means to distinguish among them Distinction without order

A nominal-level measure

Ordinal-level measurement  Scores indicate order, but not distance along some dimension The difference between a one and a two may be greater or less than the difference between a two and a three

An ordinal-level measure Large dog Score: 3 Small dog Score: 1 Medium size dog Score: 2

Interval-level measurement  Scores indicate direction and distance Intervals are of equal size—the difference between 1 and 2 is equal to the difference between 3 and 4 The zero point is arbitrary—does not indicate complete absence of the attribute Many statistical analyses assume this level of measurement

An interval-level measure

Ratio-level measurement  Scores indicate order and distance from a true zero point The units along the scale are equal Allows for calculation of the ratio of one point on the scale compared to another

Ratio-level measures

Scales  When measuring attitudes, behaviors, etc. there is bound to be a significant amount of measurement error  For reasons we will look at later, using multiple items/measures to create scores for individual objects improves the measurement of each one  We call measures combining multiple items to measure a single concept ‘scales’

Scales  Each item in a scale is supposed to measure the construct of interest, but it is possible that either: The concept has more than one dimension, or The items tap into more than one concept  To test for multidimensionality, statistical techniques are available Factor analysis Interitem correlations

Scale development  To improve the reliability/performance of the scale, a researcher may remove items that reduce reliability, etc.  May weight items according to their factor loadings

Humor orientation scale Response categories: 1=strongly agree, 2=agree, 3=neutral, 4=disagree, 5=strongly disagree 1. I regularly tell jokes and funny stories when I am in a group 2. People usually laugh when I tell a joke or story 3. I have no memory for jokes or funny stories 4. I can be funny without having to rehearse a joke 5. Being funny is a natural communication style with me 6. I cannot tell a joke well 7. People seldom ask me to tell stories 8. My friends would say that I am a funny person 9. People don’t seem to pay close attention when I tell a joke 10. Even funny jokes seem flat when I tell them