Presentation is loading. Please wait.

Presentation is loading. Please wait.

Reliability & Validity.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If.

Similar presentations


Presentation on theme: "Reliability & Validity.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If."— Presentation transcript:

1 Reliability & Validity

2  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If unreliable or invalid scale need to be very cautious

3 Item 1 Item 2 Item 3 CONSTRUCT Related measures & outcomes Unrelated measures & outcomes

4  Captures how the value of one variable changes when the value of the other changes  Ranges from -1 to +1  A Pearson correlation is based on continuous variables  Important to remember this is a relationship for a group, not each person/item  Reflects the amount of variability shared by two variables

5 Correlations test 1test 2test3 test 1Pearson Correlation **.364 ** Sig. (2-tailed).000 N 105 test 2Pearson Correlation.555 ** ** Sig. (2-tailed).000 N 105 test3Pearson Correlation.364 **.613 ** Sig. (2-tailed).000 N 105 **. Correlation is significant at the 0.01 level (2-tailed).

6 r xy = n ΣXY - ΣX ΣY [n ΣX 2 – (ΣX) 2 ][n ΣY 2 - (ΣY) 2 ]  r xy = correlation coefficient between x & y  n = size of sample  X = score on X variable  Y = score on Y variable

7 .80 to 1.0 very strong.60 to.80 strong.40 to.60 moderate.20 to.40 weak.00 to.20 weak/none Relationships of.70 or stronger are generally considered acceptable in reliability analyses

8  The extent to which a scale measures construct consistently  Any measurement is an observed score  Reliability = true score/ (true score + error)  Less error = observed score is closer to true score (more reliable)  We never know the “true score”

9  Extent to which a test is reliable over time  Calculate the correlation between two time points for each person ◦ Items should relate positively *Sometimes you expect the scores to be different

10  Extent to which two forms of a test are equivalent  Calculate the correlation between the two forms of the test

11  Extent to which items are consistent with one another and represent one dimension  Correlation between individual scores and the total score  Also estimate correlations among the items  Important that all items use the same scale and be in the same direction  Cronbach’s alpha (α)

12 α = k s 2 y – Σs 2 i k-1 s 2 y k = number of items S 2 y = variance associated with observed score Σ s 2 i = sum of all variances for each item

13 Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items Inter-Item Correlation Matrix -In uncertain times, I usually expect the best. -I’m always optimistic about my future. -If something can go wrong for me, it will. -In uncertain times, I usually expect the best I’m always optimistic about my future If something can go wrong for me, it will

14  Agreement between two raters ir = # of agreements # of possible agreements

15  The extent to which the scale measures what it is intended to measure  Can be reliable without being valid

16  Items sample the universe of items for a construct  Can ask an expert (or several) whether items seem representative

17  Scale relates to other measures or behaviors in ways that would be expected  Concurrent ◦ At same time  or predictive ◦ Predicts later scores

18  Scale measures the underlying construct as intended  Relation to the behaviors that the construct represents


Download ppt "Reliability & Validity.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If."

Similar presentations


Ads by Google