1. Chapter 8 Making Judgments and Decisions
Intuitive Predictions
Judgment Aids
Utility Analysis
chapter 8 Makeing Judgments and Decisions

2. chapter 8 Makeing Judgments and Decisions
JUDGMENTS OF VALIDITY
Evaluating a Measure
Without statistical validation – judge job relatedness based on sequential questions:
Is trait related to performance (link KSA -> task)?
is the construct a valid measure of the KSA?
Give an example
12 questions to ask
Be prepared to answer the questions for a measure you may use
chapter 8 Makeing Judgments and Decisions

3. 12 Questions to ask….to evaluate a measure
Clear idea of the attribute (construct) measured?
Was the measurement method consistent with the construct?
Was the stimulus content appropriate? Unambiguous? Relevant to the measure?
Was the development done skillfully with a pilot study?
Single attribute or heterogeneous domain?
Are scores stable over time? How do you know?
Do the scores show convergent validity?
Do the relationships show divergent validity?
Does the attribute relate to job performance?
Does the predictive hypothesis require other attributes equally important?
Could there be a non-monotonic relationship?
Are criteria measured validly? And predict with reasonable accuracy?
chapter 8 Makeing Judgments and Decisions

4. MANAGERIAL USE OF ASSESSMENTS
What do managers want?
Fill vacancy asap
Want the best people
Want it cheap
What do managers know about testing?
Little; no training in psychometrics or test theory
Nothing about construct validity
Misinformed on test validity
Often distrust testing
Some overestimate the validity of tests (especially when over-sold by test vendors (e.g. DISC, MBTI)
chapter 8 Makeing Judgments and Decisions

5. JUDGMENTS AS PREDICTIONS AND DECISIONS
What is “common sense”, “intuition?”
Give some examples.
A thought question -
Statistical v. judgmental Prediction
What are the pitfalls with each?
Which is generally superior? (Meehl, ‘54)
Prediction and Decision Without Statistics
Usually the most important jobs have few people
Why is this a problem for using statistics?
How do you deal with this?
Combining Information for Overall Assessment
Average assessments/use common scale
chapter 8 Makeing Judgments and Decisions

6. chapter 8 Makeing Judgments and Decisions
JUDGMENT AIDS
Clarifying Judgments to Be Made
JA is essential! –it makes management focus on what it will take to do the job
Procedural Planning
Make a detailed protocol and document it
Why is this a good thing to do?
Developing Assessment Scales
Use the same scale or different scales?
A structured interview?
Effect size alternatives
Common Language Effect Sizes (CLES)
Relative Risks (RR) –probability of positive/negative outcome)
Behavioral Effect Size Display (Rosenthal) Table 8.2
Expectancy Charts
What is a theoretical expectancy chart?
How is it used?
Give an example
Expectancy Graphs
Use selection ratio and possible cut scores –
to illustrate selection costs to show validity impact (for managers to understand)
Utility Analysis
Brogden (‘49), Cronbach & Gleser (‘57)
Show relative pay off (average dollar pay off)
Caveat (GIGO)
Concluding Comments
Judgments must be rational and include all infromation available
chapter 8 Makeing Judgments and Decisions