1. Chapter 4A Validity and Test Development

2. Basic Concepts of Validity Validity must be built into the test from the outset rather than being limited to the final stages of test development. Validity defines the meaning of test scores. Test validation is a developmental process that begins with test construction and continues indefinitely.

3. Validity: A Definition Standards for Educational and Psychological Testing: A test is valid to extent that inferences made from it are appropriate, meaningful, and useful. Traditionally, the different ways of accumulating validity evidence have been grouped into three categories: Content validity, Criterion-related validity, Construct validity.

4. Content Validity In theory, content validity is really nothing more than a sampling issue. If the sample (specific items on the test) is representative of the population (all possible items), then the test possesses content validity. Content validity is a useful concept when a great deal is known about the variable that the researcher wisher to measure.

5. Content Validity Content validity is more difficult to assure when the test measures an ill-defined trait. Quantification of Content Validity: determining the overall content validity of a test from the judgments of experts. However, it cannot identify nonexistent items that should be added to a test to help make the pool of questions more representative of the intended domain.

6. Content Validity Face validity is not really a form of validity at all A test has face validity if it looks validity to test users, examiners, and especially the examinees. Face validity is really a matter of social acceptability and not a technical form of validity in the same category as content, criterion-related, or construct validity.

7. Criterion-Related Validity Criterion-related validity is demonstrated when a test is shown to be effective in estimating an examinee’s performance on some outcome measure (criterion). In concurrent validity, the criterion measures are obtained at approximately the same time as the test scores. In predictive validity, the criterion measures are obtained in the future, usually months or years after the test scores are obtained.

8. Characteristics of a Good criterion A criterion is any outcome measure against which a test is validated. A good criterion must also be reliable, approximate, and free of contamination from the test itself. The theoretical upper limit of the validity coefficient is constrained by the reliability of both the test and the criterion.

9. Characteristics of a Good criterion Criterion contamination A criterion must also be free of contamination from the test itself. Criterion contamination is also possible when the criterion consists of ratings from experts. When validating a test against a criterion of expert ratings, the test scores must be held in strict confidence until the ratings have been collected.

10. Concurrent Validity Correlations between a new test and existing tests are often cited as evidence of concurrent validity. But two conditions must be satisfied: 1. the criterion (existing) tests must have been validated through correlations with appropriate nontest behavioral data. 2.the instrument being validated must measure the same construct as the criterion tests.

11. Predictive Validity Validity coefficient and the standard error of the estimate:

12. Construct Validity A construct is a theoretical, intangible quality or trait in which individuals differ. All psychological constructs possess two characteristics in common: 1.There is no single external referent sufficient to validate the existence of the construct; that is, the construct cannot be operationally defined. 2.Nontheless, a network of interlocking suppositions can be derived from existing theory about the construct.

13. Approaches to Construct Validity Test Homogeneity Appropriate Developmental Changes Theory-Consistent Group Differences Theory-Consistent Intervention Effects Convergent and Discriminant Validation Factor Analysis Classification Accuracy

14. Extravalidity Concerns and the Widening Scope of Test Validity Psychologists confirm that the decision to use a test involves social, legal, and political considerations that extend far beyond the traditional questions of technical validity. Unintended side effects of Testing

15. The Widening Scope of Test Validity A general definition: A test is valid if it measures “what it purports to measure.” A recent claim: A test is valid if it serves the purpose for which it is used. Test validity, then, is an overall evaluative judgment of the adequacy and appropriateness of inferences and actions that flow from test scores.

16. Structural Equation Model Website: http://www.ssicentral.com/lisrel/downloads.html