1. 1 Evaluating Psychological Tests

2. 2 Psychological testing Suffers a credibility problem within the eyes of general public Two main problems –Tests used inappropriately Goddard (1912) used a translation of Binet’s test to test ability of American immigrants - conclusion 79% of Italian immigrants = ‘feeble-minded’ - bias –Tests themselves can be flawed Often measures supposed constructs which are not supported by proper factor analysis - (Internal locus of control)

3. 3 External bias in tests Do group differences imply test bias (difficulty unrelated to characteristic being assessed)? –V1 - innate abilities can be different across groups (Reynolds, 1995; Kline, 1993) Japanese have higher than average spatial abilities African Americans have ‘lower IQ’ (Hernstein & Murray, 1996) –V2 – Ethnic and gender groups must have the same underlying abilities – evidence to the contrary must be a product of measuring something other than what is relevant Kline – ‘egalitarian fallacy’

4. 4 Dealing with differences Detected through different regression equation – not through different means What purpose does research in this area serve? –Within group differences far outweigh between group differences

5. 5 Detecting internal bias If only gross scores are considered, hard and easy items for each group might balance themselves out giving a false impression of the test’s ‘health’ Alternative – Run a mixed factorial ANOVA –Each test item (question) is entered as a level of repeated measures factor –Group = between subjects variable Main effect of item – expected Main effect of group shows external bias Interaction show internal bias in that the pattern of responding is different across the groups Such a method is susceptible to power manipulation

6. 6 Bias - performance characteristics Response bias –individuals are more likely to agree than disagree (Cronbach, 1946) – response set of acquiescence Does not cause a problem if everyone behaves in same manner – standard score will be unaffected But there are considerable individuals differences in acquiescence therefore it can cause a major problem –Changing polarity removes this difficulty Social desirability –Counter acted by lie scales and consistency measures

7. 7 Obvious influences Motivation Expectation Anxiety Test specific practise

8. 8 Revisiting Validity

9. 9 Validity – different definitions Correctness or truth of an inference Validity with respect to IV –Are we truly manipulating that which we think we are Often relies on the construct of interest being adequately described How do you manipulate something like the unconscious? Validity with respect to the DV –Extent to which you are measuring what you claim to measure

10. 10 Different types of validity Content validity –Whether the target construct is adequately addressed –When measuring depression should assess aspects such as fatigue, anxiety, appetite, motivation, libido Is assessed through expert opinion –Has a certain amount of subjectivity

11. 11 Criterion-Related validity –How measure compares to some already validated measure Two types –Predictive –Concurrent Different types of validity

12. 12 Different types of validity Construct validity –Most important – Are the experimental manipulations that we make really manipulating the construct of interest –Evaluation requires Clear definition of the construct –Can be difficult e.g., IQ – has many different facets Assess match between construct and operations used to represent it (exp manipulations) –Can involve criterion and content validity –Viewed as an evolving never ending process

13. 13 Different types of validity Internal validity – degree to which the independent and dependent variables are causally linked External validity – degree to which causal relationship holds across different settings

14. 14 How relevant is validity to you Reviewing articles is essentially addressing validity and reliability issues –In examination situation would be useful although not essential to talk about the different forms of validity In discussion sections of reports again you are essentially evaluating the results with respect to validity and reliability –Would not really use the formal language used here – is a style issue