Validity Is the Test Appropriate, Useful, and Meaningful?

Properties of Validity Does the test measure what it purports to measure? Validity is a property of inferences that can be made. Validity should be established over multiple inferences.

Evidence of Validity Meaning of test scores Reliability Adequate standardization and norming Content validity Criterion-related validity Construct validity

Content Validity How well does the test represent the domain? Appropriateness of items. Completeness of items. How the items assess the content.

Do the Items Match the Content? Face validity Is the item considered part of the domain Curriculum match Do the items reflect what has been taught Homogeneity with the test Is the item positively correlated with the test score? (Minimum.25) Point bi-serial r xx

Are the Total Items Representative? Are items included representing various parts of the domain? Are different parts differentially represented? How well are the parts represented?

How Is the Content Measured? Different types of measures Multiple choice, short answer Performance-based vs. Factual recall Measures should match the type of content taught Measures should match HOW the content was taught

Criterion-related (Cr)validity How well does performance on the test reflect performance on what the test purports to measure? Expressed as r xx Based on concurrent CR validity And predictive CR validity

Concurrent Criterion-related Validity How well does performance on the test estimate knowledge and/or skill on the criterion measure? Does the reading test estimate the students current reading performance? Compares performance on one test with performance with other similar tests. KTEA w/ Woodcock-Johnson

Predictive Criterion-related Validity Will performance on a test now be predictive of performance at a later time? Will a score derived now from one test be as accurate as a score derived by another and later test ? Will a student’s current reading scores accurately reflect reading measured at a later time by another test?

Criterion-related Validity Should Describe criterion measures accurately Provide rationales for choices. Provide sufficient information to judge adequacy of the criterion Describe adequately the sample of students used. Include analytical data used to determine predictiveness

Criterion-related Validity Should Basic statistics should include Number of cases Reasons for eliminating cases Central tendency estimates Provide analysis of the limits toward generalizability of the test What kind of inferences about the content can be made

Construct Validity How validly does the test measure the underlying constructs it purports to measure? Do IQ tests measure intelligence? Do self-concept scales measure self concept?

Definition of Construct A psychological or personality trait E.G. Intelligence, learning style Or A psychological concept, attribute, or theoretical characteristic E.G. Problem solving, locus of control, or learning disability

Ways to Measure Construct Validity Developmental change: determining expected differences among identified groups Assuming content validity Assuming reliability Convergent /divergent validity: high correlation with similar tests and low correlation with tests measuring different constructs

Ways to measure Construct Validity Predictive Validity: High scores on one test should predict high scores on similar tests. Accumulation of evidence FAILING TO DISPROVE: If the concept or trait tested can be influenced by intervention, intervention effects should be reflected by pre and posttest scores. If the test score should not be influenced by intervention, changes should not be reflected in pre and posttest scores.

Factors Affecting Validity Unsystematic error Lack of reliability Systematic error Bias

Reliability Effects on Validity The validity of a measure can NEVER exceed the measure’s reliability Reliability measures error Validity measures expected traits (content, constructs, criteria) r xy = r x(t)y(t) r xx r yy

Systematic Bias Method of Measurement Behaviors in testing Item selection Administration errors Norms

Who is Responsible for Valid Assessment? The Test Author Authors are reponsible for ensuring and publishing validation. The Test Giver Test administrators are reponsible for following procedures outlined in administration guidelines.

Guidelines for Giving Tests Exact Administration Read the administration instructions. Note procedures for establishing baselines. Note procedures for individual items. Practice giving the test. Appropriate Pacing Develop fluency with the test.