1. Foundations of Recruitment and Selection I: Reliability and Validity
Chapter 2
Foundations of Recruitment and Selection I: Reliability and Validity
© 2013 by Nelson Education

2. Chapter Learning Outcomes
After reading this chapter you should:
Understand the basic components that make up a traditional personnel selection model
Have a good understanding of the concepts of reliability and validity
Recognize the importance and necessity of establishing the reliability and validity of measures used in personnel selection
© 2013 by Nelson Education

3. Chapter Learning Outcomes (continued)
Identify common strategies that are used to provide evidence on the reliability and validity of measures used in personnel selection
Appreciate the requirement for measures used in personnel selection to evaluate applicants fairly and in an unbiased fashion
© 2013 by Nelson Education

4. The Recruitment and Selection Process
An employer’s goal is to hire an applicant who possesses the knowledge, skills, abilities, or other attributes (KSAOs) required to perform the job
© 2013 by Nelson Education

5. Recruitment and Selection Today 2.1
Selection Process for Winnipeg Police Constables: Minimum Qualifications
Age 18 years or older
Education (Manitoba Grade 12 or equivalency– e.g. GED)
Foreign credentials require assessment by Manitoba Labour and Immigration prior to application
Valid Full Class 5 driver’s licence with no more than four demerits on Driver’s abstract as assessed by the Winnipeg Police Service 
Selection Process (page 37) outlines the procedure used by the Winnipeg Police Department in selecting new recruits for the position of police constable:
must provide evidence that they meet the vision standards needed to do the job
only those candidates who pass the written examination are invited to a short, behavioural-based screening interview conducted by a panel
a three hour background interview is completed
candidates who are considered for employment undergo psychological and medical examinations, including testing for drug use
a selection panel compares all of the qualified candidates and decides on whom to offer employment
© 2013 by Nelson Education

6. Recruitment and Selection Today 2.1(continued)
Canadian citizen or landed immigrant
No involvement in any criminal activity within the last two years (including illegal drugs)
No criminal record for which a pardon has not been granted
© 2013 by Nelson Education

7. Recruitment and Selection Today 2.1(continued)
Must meet current Winnipeg Police Service vision standards
Successful completion of job-related physical test (Winnipeg Police Service—Physical Abilities Test: WPS-PAT) within 4 minutes and 15 seconds
© 2013 by Nelson Education

8. Table 2.1 (p. 38) compares some of the differences between empirical evidence-based and practice-based selection processes.
© 2013 by Nelson Education

9. Human Resources Management: Science versus Practice in Selection
© 2013 by Nelson Education

10. A Selection Model
Figure 2.1 (p. 34) presents an overview of the components and process of a traditional selection system.
© 2013 by Nelson Education

11. Construct
Construct: an idea or concept constructed or invoked to explain relationships between observations
For example, the construct “extroversion” has been invoked to explain the relationship between “social forthrightness” and sales; “learning” is a construct used to explain the change in behaviour that results from experience. Constructs are abstractions that we infer from observations and that we cannot directly observe. In the natural sciences, “gravity” is perhaps the most famous construct.
© 2013 by Nelson Education

12. Building a Foundation
Reliability: the degree to which observed scores are free from random measurement errors; an indication of the stability or dependability of a set of measurements over repeated applications of the measurement procedure
Reliability refers to the consistency of a set of measurements when a testing procedure is repeated on a population of individuals or groups.
In terms of testing, a test should provide approximately the same information each time it is given to that person.
Errors may also be systematic; that is, the errors may be made in a consistent, or predictable, fashion.
Systematic errors do not affect the accuracy of the measurements but rather the meaning, or interpretation, of those measurements.
© 2013 by Nelson Education

13. Interpreting Reliability Coefficients
True score: the average score that an individual would earn on an infinite number of administrations of the same test or parallel versions of the same test
Error score (or measurement error): the hypothetical difference between an observed score and a true score
© 2013 by Nelson Education

14. Factors Affecting Reliability
Temporary Individual Characteristics
Lack of Standardization
Chance
Temporary Individual Characteristics: factors such as health, motivation, fatigue, and emotional state introduce temporary, unsystematic errors into the measurement process.
Lack of Standardization: changing the conditions under which measurements are made introduces error into the measurement process.
Chance: factors unique to a specific procedure introduce error into the set of measurements.
© 2013 by Nelson Education

15. Methods of Estimating Reliability
Test and Retest
Alternate Forms
Internal Consistency
Inter-Rater Reliability
Choosing an Index of Reliability
Methods of estimating using two different, but parallel, measures of the characteristic or attribute:
Test and Retest: the identical measurement procedure is used to assess the same characteristic over the same group of people on two different occasions.
Alternate Forms: having a person take the same interview twice may lead to a false estimate of the reliability of the interview process. To prevent the intrusion of effects from the first interview, the manager asks the applicants alternate questions during the second interview; the correlation between both experience scores again estimates reliability, with high correlations once more indicating strong reliability.
Internal Consistency: consider any two questions in the interview to be an example of a test-retest situation, and determine the correlation between scores given to each item in that pair; logic behind establishing reliability through internal consistency. Rather than select any particular pair of items, the correlations between the scores of all possible pairs of items are calculated and then averaged; this average estimates the internal consistency, the degree to which all the questions in the set are measuring the same thing.
Inter-Rater Reliability: measurement in personnel selection is often based on the subjective assessment, or rating, of one individual by another.
Choosing an Index of Reliability: have to estimate the degree of variability in a set of scores that is caused by measurement error; obtain this estimate by Index of Reliability; the HR specialist chooses an appropriate index of reliability and determines the level of reliability that is acceptable for use of a specific measure.
The HR Specialist must consider the consequences of the decisions based on the measure; the need for accuracy increases with the seriousness of the consequences for the employee.
© 2013 by Nelson Education

16. Validity
Validity: the legitimacy or correctness of the inferences that are drawn from a set of measurements or other specified procedures; the degree to which accumulated evidence and theory support specific interpretations of test scores in the context of the test’s proposed use
© 2013 by Nelson Education

17. Construct, Content, Criterion-related Validity
Construct and content validity: validation strategies that provide evidence based on test content
Criterion validity: related validity provides evidence based on relationships to other variables
© 2013 by Nelson Education

18. Figure 2.2 (p. 47) illustrates different sources of evidence for validity using the cognitive ability data collected by the HR manager.
© 2013 by Nelson Education

19. Predictive and Concurrent Evidence for Test Criterion Relationships
Predictive evidence: obtained through research designs that establish a correlation between predictor scores obtained before an applicant is hired and criteria obtained at a later time, usually after an applicant is employed
© 2013 by Nelson Education

20. Predictive and Concurrent Evidence for Test Criterion Relationships (continued)
Concurrent evidence: obtained through research designs that establish a correlation between predictor and criteria scores from information that is collected at approximately the same time from a specific group of workers
© 2013 by Nelson Education

21. Validity Generalization
Validity generalization: the application of validity evidence, obtained through meta- analysis of data obtained from many situations, to other situations that are similar to hose on which the meta-analysis is based
© 2013 by Nelson Education

22. Factors Affecting Validity Coefficients
Range Restriction
Measurement Error
Sampling Error
Range Restriction: when measurements are made on a subgroup that is more homogeneous than the larger group from which it is selected, validity coefficients obtained on the subgroup are likely to be smaller than those obtained from the larger group; the reduction in the size of the validity coefficient due to the selection process.
Measurement Error: the size of a validity coefficient cannot exceed the reliability of the measures used to obtain the data.
© 2013 by Nelson Education

23. Class Activity
Discuss why it is better to base a selection system on science than a ‘gut feeling’?
Does an organization have an obligation to make the enterprise as profitable as possible on behalf of its owners, or does it have an obligation to meet the objectives of society by providing equal employment opportunities for members of different population groups?
© 2013 by Nelson Education

24. Bias and Fairness
Bias: systematic errors in measurement, or inferences made from those measurements, that are related to different identifiable group membership characteristic such as age, sex, or race
© 2013 by Nelson Education

25. The correlation between cognitive ability and job performance is used to derive an equation for a straight line that best fits the data points contained in the scatterplot shown in Figure 2.3 (p. 55). This is a regression line. The diagonal line in Figure 2.3 is the regression line that provides the best fit to the data contained in the scatterplot.
© 2013 by Nelson Education

26. Fairness
Fairness: the value judgments people make about the decisions or outcomes that are based on measurements
Principle that every test taker should be assessed in an equitable manner
© 2013 by Nelson Education

27. Recruitment and Selection Today 2.2
Different Views of Fairness:
Fairness as lack of bias
Fairness as equitable treatment in the testing process
Fairness as opportunity to learn
Fairness as equality in outcomes of testing
Fairness in selection and prediction
Recruitment and Selection Today 2.2 (p. 57) presents some differing views on fairness. Fairness issues also include the reaction of applicants to testing and personnel selection decisions.
© 2013 by Nelson Education

28. Summary Be familiar with measurement, reliability, and validity issues
The reliability and validity of the information used as part of personnel selection procedures must be established empirically
© 2013 by Nelson Education

29. Discussion Questions
We presented a summary of the Meiorin case at the outset of this chapter. Can you think of procedures or changes that the consultants could have done that would have allowed the test to meet the objections of the Supreme Court?
Can an invalid selection test be reliable? Can an unreliable selection best be valid?
© 2013 by Nelson Education