1. Reliability & Validity

2.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If unreliable or invalid scale need to be very cautious

3. Item 1 Item 2 Item 3 CONSTRUCT Related measures & outcomes Unrelated measures & outcomes

4.  Captures how the value of one variable changes when the value of the other changes  Ranges from -1 to +1  A Pearson correlation is based on continuous variables  Important to remember this is a relationship for a group, not each person/item  Reflects the amount of variability shared by two variables

5. Correlations test 1test 2test3 test 1Pearson Correlation 1.000.555 **.364 ** Sig. (2-tailed).000 N 105 test 2Pearson Correlation.555 ** 1.000.613 ** Sig. (2-tailed).000 N 105 test3Pearson Correlation.364 **.613 ** 1.000 Sig. (2-tailed).000 N 105 **. Correlation is significant at the 0.01 level (2-tailed).

6. r xy = n ΣXY - ΣX ΣY [n ΣX 2 – (ΣX) 2 ][n ΣY 2 - (ΣY) 2 ]  r xy = correlation coefficient between x & y  n = size of sample  X = score on X variable  Y = score on Y variable

7. .80 to 1.0 very strong.60 to.80 strong.40 to.60 moderate.20 to.40 weak.00 to.20 weak/none Relationships of.70 or stronger are generally considered acceptable in reliability analyses

8.  The extent to which a scale measures construct consistently  Any measurement is an observed score  Reliability = true score/ (true score + error)  Less error = observed score is closer to true score (more reliable)  We never know the “true score”

9.  Extent to which a test is reliable over time  Calculate the correlation between two time points for each person ◦ Items should relate positively *Sometimes you expect the scores to be different

10.  Extent to which two forms of a test are equivalent  Calculate the correlation between the two forms of the test

11.  Extent to which items are consistent with one another and represent one dimension  Correlation between individual scores and the total score  Also estimate correlations among the items  Important that all items use the same scale and be in the same direction  Cronbach’s alpha (α)

12. α = k s 2 y – Σs 2 i k-1 s 2 y k = number of items S 2 y = variance associated with observed score Σ s 2 i = sum of all variances for each item

13. Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items.582.5763 Inter-Item Correlation Matrix -In uncertain times, I usually expect the best. -I’m always optimistic about my future. -If something can go wrong for me, it will. -In uncertain times, I usually expect the best. 1.000.474.262 -I’m always optimistic about my future..4741.000.200 -If something can go wrong for me, it will..262.2001.000

14.  Agreement between two raters ir = # of agreements # of possible agreements

15.  The extent to which the scale measures what it is intended to measure  Can be reliable without being valid

16.  Items sample the universe of items for a construct  Can ask an expert (or several) whether items seem representative

17.  Scale relates to other measures or behaviors in ways that would be expected  Concurrent ◦ At same time  or predictive ◦ Predicts later scores

18.  Scale measures the underlying construct as intended  Relation to the behaviors that the construct represents