1. 46-320-01 Tests and Measurements Intersession 2006

2. More Correlation Spearman’s rho: two sets of ranks Biserial correlation: continuous and artificial dichotmous variable Point biserial correlation: true dichotmous variable

3. Hypothesis Testing Review Independent and Dependent variables In Psychology we test hypotheses Null Hypothesis (H 0 ): a statement of relationship between the IV and DV, usually a statement of no difference or no relationship – we assume there is no relationship between IV and DV Alternative/Research Hypothesis (H a ): states a relationship, or effect, of the IV on the DV

4. Hypothesis Examples H 0 : Men and women do not differ in IQ (  men =  women ) H a : Men and women do differ in IQ (  men   women ) Any difference in value of the DV between the levels of the IV can be explained in 2 ways – the effect of the IV or sampling error

5. Hypothesis Testing with Correlations Null Hypothesis: there is no significant relationship between X and Y Alternative Hypothesis: there is a significant relationship between X and Y (r is significantly different from 0) We can use Appendix 3 (p. 641) df = N – 2 r obs =.832 r crit =.195 Reject H o

6. Regression We know the degree to which 2 variables are related - correlation How do we predict the score on Y if we know X? Regression line Principle of least squares

7. Equation Explained Y’: predicted value of Y b: regression coefficient = slope Describes how much change is expected in Y with one unit increase in X a: intercept = value of Y when X is 0

8. Line of Best Fit Actual (Y) and predicted (Y’) scores are almost never the same Residual Deviations from Y’ at a minimum Prediction Interpreting plot

9. More Correlation Standard error of estimate Coefficient of determination Coefficient of alienation Shrinkage Cross validation Correlation does not equal causation! Third variable

10. Multivariate Analysis 3 or more variables Many predictors, one outcome Linear Regression: linear combination of variables Weights Raw regression coefficients Standardized regression coefficients Predictive power

11. More Multivariate Discriminant Analysis Prediction of nominal category Multiple discriminant analysis Factor Analysis No criterion Interrelation Data reduction Principal components Factor loadings Rotation

12. Reliability Assess sources of error Complex traits Relatively free from error = reliable Spearman, Thorndike 1904 Coefficients Kuder and Richardson 1934 Cronbach 1972 on IRT True Score

13. Reliability Error and True Score X = T + E Random Error produces a distribution Mean is the estimated true score

14. Reliability True score should not change with repeated administrations Standard error of measurement Larger = less reliable Use to create confidence intervals

15. Reliability Domain Sampling Model Shorter test estimate, but sample = error Reliability: Usually expressed as a correlation Reliability: Sampling distribution, correlations b/w all scores, average correlation

16. Reliability Reliability: Percentage of observed variation attributable to variation in the true score r =.30: 70% of variance in scores due to random factors

17. Sources of Error Why are observed scores different from true scores? Situational factors Unrepresentative q’s What else?

18. Test-Retest Reliability Error of repeated administration Correlation b/w 2 times Consider: Carryover effects Time interval Changing characteristics

19. Parallel Forms Reliability 2 forms that measure the same thing Correlation between two forms Counterbalanced order Consider time interval Example: WRAT-3

20. Internal Consistency Split-Half reliability Divide and correlate (internal consistency) Check method of dividing Why use Spearman-Brown formula? Each test ½ length – decreases reliability Cronbach’s alpha – unequal variances

21. Internal Consistency Intercorrelations among items within same test Extent to which items measure same ability/trait Low? Several characteristics? Use KR 20, coefficient alpha Considers all ways of splitting data

22. Difference Scores Same trait: reliability = 0 Use z-score transformations Generally low

23. Observer Differences Estimate reliability of observers Interrater Reliability Percentage Agreement Kappa Corrects for chance agreement 1 (perfect agreement) to –1 (less than chance alone) Interpreting: >.75 = “excellent”.40 to.75 = “fair to good” <.40 = “poor”

24. Interpreting Reliability General rule of thumb: Above 0.70 to 0.80 – good Higher the stakes, higher the r Use confidence intervals (from standard error of estimate)

25. Low Reliability Increase items Spearman-Brown prophecy formula Factor item analysis Omit items that do not load onto one factor Drop items Correct for Attenuation (low correlations)

26. Validity Agreement b/w a test score and what it is intended to measure Face validity: Looks like it’s valid Content-validity Representative/fair sample of items Construct underrepresentation Construct-irrelevant variance

27. Criterion-Related Validity How well a test corresponds with a criterion Predictive validity Concurrent validity Validity Coefficient Coefficient of determination

28. Evaluating Validity Coefficients Changes in cause of relationship Meaning of criterion Validity population Sample size Criterion vs predictor Restricted range Validity generalization Differential prediction

29. Construct-Related Validity Define a construct and develop its measure Main type of validity needed Convergent evidence Correlates with other measures of construct Meaning from associated variables Discriminant evidence Low correlations with unrelated constructs Criterion-referenced tests