1. Predictors Predictor – variable used to forecast some outcome variable (or criterion)  Example: Supervisors rating of performance, actual measures of performance (sales, no. of complaints…). Effort is predictor of sale  Predictors must be: Reliable Valid  Predictor cannot be valid if it is not reliable, but may be reliable without being valid

2. Validity and Reliability

7. Predictors Reliability Reliability – consistency or stability of a measure  Does the instrument provide the same measurement every time or does it fluctuate due to instrument error?  Example: Reliability is the extent to which a score from a selection measure is stable or free from error

8. Measures of Reliability Test 1. Test-Retest 2. Alternate forms reliability 3. Internal reliability / Consistency 4. Scorer Reliability

9. 1. Test-Retest – use same measurement device twice. Same people take test twice if stable trait is being measured, should get similar results each time. If similar results are achieved then phenomenon is called temporal stability Coefficient of Stability – Correlation between first set of measurements and second (look for r >.70) Test-retest reliability (TRR) coefficient for tests used in industry is 0.86 (Hood, 2001) Longer interval b/w two tests lowers reliability coefficient ( Anastasi & Urbina, 1997)

10.  TRR is not appropriate for all kinds of tests.  E.g. Anxiety is of two types: Trait Anxiety and State Anxiety. TRR may not hold true in case of measurement of state anxiety

11. 2. Alternate forms reliability: Equivalent Forms – two forms of the same test given to the same people  Coefficient of Equivalence – Correlation between the 2 forms  Two forms of tests should have the same mean and SD (Clause, Mullins, Nee, Pulakos, & Schmitt, 1998)  Often difficult to develop 2 truly equivalent forms of a test Example: 1) Examination question papers are repeated in every test 2) Give 2 forms of IQ test to group of people to see whether the 2 forms assess the same construct.

12. 3. Internal reliability / Consistency – how well do items in test relate to other items in the test (homogeneity of test).  Extent to which similar items are answered in similar ways a. Split-Half – Correlate odd numbered items with the even numbered items.  Odds is one group and evens in other  Scores of 2 groups correlated  B/c number are reduced so to adjust correlation, Spearman-Brown prophecy is used

13. b. Cronbach’s Alpha – (Cronbach, 1951): Conceptually, the average correlation of each item on test with every other item  Example: A test covering just the concepts in psychology would have a higher internal consistency than one covering math, English, and history  Cronbach is most commonly used test for testing the internal reliability (Hogan, Benjamin & Brezinski, 2003)

14. c. K-R-20 (Kunder-Richardson formula 20) – (Kunder-Richardson, 1937): Similar to Cronbach Alpha. But K-R-20 can be used for only dichotomous items (yes/no, true/false…), while alpha can be used for all situations

15.  General rule: In general, longer the test, the higher its internal consistency  E.g. If a test consists of only 2 MCQs, if you wrongly attempt one your score will diminish by 50%. On the other hand, if test consists of 100 MCQs, then if you do one question wrong, it will not impact much

16.  4. Scorer Reliability– If the scorer makes mistake, results would become non- reliable.  Allard, Butler, Faust, and Shea (1995) found that 53% personality tests contained at least one scoring error, 19% contained enough errors to alter a clinical diagnosis

17. Inter-rater – Used when test scores are based on subjective ratings  Correlation among scores given by 2 or more different raters  Examples: Interview assessment Olympic skating judges

18. Summary of Reliability Estimates TypeCoefficientWhat is MeasuredMethods Test-retestStabilityReliability of a test or instrument inferred from examinee scores. Same test is administered twice to same subjects over an interval of less than six months. Correlation Parallel formsEquivalenceDegree to which alternative forms of the same measure produce same or similar results. Administered simultaneously or with a delay. Correlation Split-halfInternal Consistency Degree to which instrument items are homogeneous Specialized correlational formulas KR20 Conbach’s alpha

19. LOGO Validity of Measurement Tools Your site here Company Logo

20. Validity Validity – Does the test measure what it is supposed to (accuracy)?  Is the test appropriate for its intended use?  The validity coefficient is calculated as a correlation between the two items being compared, very typically success in the test as compared with success in the job.  Validity coefficient (r) greater than.4 considered good  A validity of 0.6 and above is considered high, which suggests that very few tests give strong indications of job performance. Example: Is typing speed a valid measure of secretary performance?

21. Types of validity  Content validity  Criterion validity  Construct validity  Face validity

22. 1. Content Validity  Extent to which test items sample the content that they are supposed to measure  Degree to which a predictor covers a representative sample of the behavior being assessed.  Does the content of the test represent aspects of the job?  Content validity reduced when important areas are missed.

23. 2. Criterion Validity  How well does a predictor relate to the criterion?  It reflects the success of measures used for prediction or estimation.  Criterion validity refers to the extent to which a test score is related to some measure of job performance  Commonly used criteria: supervisors rating of performance, actual measures of performance (sales, no. of complaints…)  Criterion Validity has two types: Concurrent validity Predictive Validity:

24. Concurrent – both predictor and criterion data collected at the same time Test is given to employees who are working on job. Then we correlate test scores with performance Example: give test to incumbent and correlate with existing performance records Predictive – predictor test given at one point, then correlated with criterion measure taken at some later time Test is administered to individuals who are going to be hired. Example: employees tested prior to training, then scores are correlated with measure of performance taken 6 months later

25. 3. Construct Validity  Most theoretical of the validity types  Extent to which a test actually measures the construct that it purports to measure (Kaplan & Saccuzzo, 2005)  Construct validity is more concerned with inferences about test scores while content validity is more concerned with inferences about the test construction  Construct validity is determined by correlating scores of the test with scores from other tests  Construct-irrelevant variation occurs when irrelevant factors contaminate the test  Example: Do items on a calculus test appear appropriate as a predictor for selecting bus drivers?

26. – The extent to which the test is an accurate representation of the theoretical construct  Correlation between scores on tests believed to assess similar constructs  Use to develop new, improved, or shorter tests 1.Convergent Validity – the correlations should be high for tests of similar constructs 2.Divergent Validity – Correlations should be low for test of unrelated concepts  Example: Test used for assessing pilots’ flying skill should be highly related to test of spatial ability (convergent), but unrelated to schizophrenia (divergent)

27. 4. Face Validity  Face Validity – Does the test appear to be appropriate by the average person taking the test?  Extent to which a test appears to be job related  If job applicants do not think a test is job related, their perceptions of its fairness decrease, as does their motivation to do well on the test (Hausknecht, Day & Thomas, 2004)  Chan, Schmitt, Deshon, Clause, and elbridge (1997): face validity resulted in high levels of test- taking motivation, which in turn resulted in higher levels of test performance  Just b/c a test has face validity does not mean it is valid (Jackson, O’Dell, & Olson, 1982) Your site hereCompany Logo

28. Summary of Validity Estimates TypeWhat is MeasuredMethods ContentDegree to which the content of the items adequately represents the universe of all relevant items under study Judgement or panel evaluation with content validity ratio Criterion- related Degree to which the predictor is adequate in capturing the relevant aspects of the criterion. Correlation a. Concurrent Description of the present; criterion data are available at same time as predictor scores. b. Predictive Prediction of the future; criterion data are measured after the passage of time. Construct Answers the question, “What accounts for the variance in the measure?” Attempts to identify the underlying construct(s) being measured and determine how well the test represents it (them). Judgmental Correlation of proposed test with established one Convergent-discriminant techniques Factor analysis Multitrait-multimethod analysis

29. LOGO Thank You! Let us work for the betterment of humanity