1. CHAPTER 5 MEASUREMENT CONCEPTS © 2007 The McGraw-Hill Companies, Inc.

2. LEARNING OBJECTIVES
Describe the problem of reactivity of a measure of behavior and discuss ways to minimize reactivity
Describe the properties of the four scales of measurement: nominal, ordinal, interval, and ratio
© 2012 The McGraw-Hill Companies, Inc.

3. LEARNING OBJECTIVES
Define reliability of a measure of behavior and describe the difference between test-retest, internal consistency, and interrater reliability
Discuss ways to establish construct validity, including predictive validity, concurrent validity, convergent validity, and discriminant validity
© 2012 The McGraw-Hill Companies, Inc.

4. RELIABILITY OF MEASURES
Consistency or Stability of a Measure of Behavior
True Score
Measurement Error
Pearson Product-Moment Correlation Coefficient
© 2012 The McGraw-Hill Companies, Inc.

5. If the scale is always off by 10 pounds, does it have reliability and/or validity?
© 2012 The McGraw-Hill Companies, Inc.

6. Pearson’s product moment correlation coefficient (r)
- 1.00
+ 1.00
0.00
Variables covary in opposite directions
Variables covary in the same direction
© 2012 The McGraw-Hill Companies, Inc.

7. Measurement Concepts Chp 5
Reliability refers to the consistency, precision or stability of a measure of behavior pg96 Are the results the same or very similar each time you measure a variable?
Measures that change or fluctuate are not reliable (assuming change is not due to the variable changing)
Any measure has two parts ) true score- real value of the variable and 2) measurement error-is shown by the greater variability
Reliability is increased when we increase the number of items in our measure, survey or test (pg97)

8. How reliable are personality quizzes in magazines?
© 2012 The McGraw-Hill Companies, Inc.

9. Measuring Reliability
We can measure reliability using the Pearson product moment correlation coefficient pg98
To calculate reliability we must have at least two scores on the measure across individuals. If the measure is reliable the two scores should be similar for each of the individuals studied (high positive correlation For most measures coefficient should be at least .80) pg 98
Types of Reliability
1) Test-Retest –Measures the same individuals at least for two points in time then calculate the Pearson product moment r between the scores. Test-Retest reliability is sometimes called a coefficient of stability in that it measures how stable is the trait being measured (Discuss some threats to validity for this measure) This is not a good measurement for traits that are considered to be in a state of flux or events occurring between the two administrations of the test

10. Measuring Reliability
2) Equivalent Form-Can avoid problems associated with Test-Retest by giving equivalent forms of the same test to the same set of people, calculating the correlation between the two scores. You can administer the two tests close in time (something you cannot do with Test-Retest).
However to the extent that the two forms are not totally equivalent a new source of error is introduced. Equivalent forms usually yield lower estimates of reliability

11. Measuring Reliability
Split-Half Reliability-Test is administered once, then the test is split in half, scored separately and a Pearson r is calculated for each score
Split-Half-correlation between the first and second half of the measurement
Odd-Even correlation between the even items & odd items of a measurement
In either case only one administration is required and the coefficient is determined by the internal components of the test (aka internal consistency reliability)
Split-half not meaningful in speed tests (in which most items are not difficult and score depends on how many items answered correctly e.g. algebra test) Coefficient of reliability is inflated*
Item-Total correlations-Look at the correlation between each item score with the total score, based on all items (also measures internal consistency)
Cronbach’s alpha -is a coefficient of internal consistency Cronbach's alpha generally increases when the correlations between the items increase. For this reason the coefficient is also called the internal consistency or the internal consistency reliability of the test pg99-100

12. Interrater Reliability
In research in which raters observe behaviors and make ratings or judgments (and then those judgments are compared and agree determines interrater reliability)
Bandura (1961) conducted a study to investigate if social behaviors (i.e. aggression) can be acquired by imitation 36 boys and 36 girls were tested from the Stanford University Nursery School aged between 3 to 6 years old. The role models were one male adult and one female adult
Under controlled conditions, Bandura arranged for 24 boys and girls to watch a male or female model behaving aggressively towards a toy called a 'Bobo doll'. The adults attacked the Bobo doll in a distinctive manner - they used a hammer in some cases, and in others threw the doll in the air and shouted "Pow, Boom“. Another 24 children were exposed to a non-aggressive model and the final 24 child were used as a control group and not exposed to any model at all.
To test the inter-rater reliability of the observers, 51 of the children were rated by two observers independently and their ratings compared. These ratings showed a very high reliability correlation (r = 0.89), which suggested that the observers had good agreement about the behavior of the children

13. RELIABILITY OF MEASURES
Reliability and Accuracy of Measures
Reliability indexes do not indicate whether a particular measure is an accurate measure of the variable of interest
A measure can be highly reliable but not accurate
© 2012 The McGraw-Hill Companies, Inc.

14. CONSTRUCT VALIDITY OF MEASURES
Indicators of Construct Validity
Face validity
Content validity
Predictive validity
Concurrent validity
Convergent validity
Discriminant validity
© 2012 The McGraw-Hill Companies, Inc.

15. Construct Validity Face Validity
The content of the measure appears to reflect the construct being measured
Content validity
The content of the measure is linked to the universe of content that defines the construct
Predictive validity
Scores on the measure predict behavior on a criterion measured at a time in the future
Concurrent validity
Scores on the measure are related to a criterion measured at the same time (concurrently)
Convergent validity
Scores on the measure are related to other measures of the same construct
Discriminant validity
Scores on the measure are not related to other measures that are theoretically different
© 2012 The McGraw-Hill Companies, Inc.

16. RESEARCH ON PERSONALITY AND INDIVIDUAL DIFFERENCES
Systematic and detailed research on validity is most often carried out on measures of personality and individual differences
Should use measures of personality that have demonstrable validity and reliability
Example: Mental Measurement Yearbook
© 2012 The McGraw-Hill Companies, Inc.

17. REACTIVITY OF MEASURES
Measure is reactive if awareness of being measured changes an individual’s behavior
Measures of behavior vary in terms of their potential reactivity
© 2012 The McGraw-Hill Companies, Inc.

18. VARIABLES AND MEASUREMENT SCALES
Nominal Scales
Ordinal Scales
Interval and Ratio Scales
The Importance of the Measurement Scales
© 2012 The McGraw-Hill Companies, Inc.

19. Scales of measurement Scale Description Example Distinction Nominal
Categories with no numeric scales
Males / females
Introverts / extroverts
Impossible to define any quantitative values
Ordinal
Rank ordering
numeric values limited
2-, 3-, and 4-star restaurants
Ranking TV programs by popularity
Intervals between items is unknown
Interval
Numeric properties are literal
Assume equal intervals between values
Intelligence
Aptitude test scores
Temperature (Fahrenheit and Celsius)
No true zero point
Ratio
Zero indicates absence of variable measured
Reaction time
Weight
Age
Frequency of behavior
Can form ratios (someone weighs twice as much as another)
© 2012 The McGraw-Hill Companies, Inc.