Validity & Reliability

OBJECTIVES Define validity and reliability Understand the purpose for needing valid and reliable measures Know the most utilized and important types of validity seen in special education assessment Know the most utilized and important types of reliability seen in special education assessment

VALIDITY Denotes the extent to which an instrument is measuring what it is supposed to measure.

© A. Taylor Do not duplicate without author’s permission 4 What is Validity? Validity asks – if an instrument measures what it is supposed to – how “true” or accurate the measurement is

© A. Taylor Do not duplicate without author’s permission 5 Thinking More about Validity Validity is whether or not the instrument measures what it is designed to measure. Below are three constructs that you can use to evaluate the validity of a measure: Face Validity Predictive Validity Concurrent Validity

© A. Taylor Do not duplicate without author’s permission 6 Three Validities Face Validity -- Do the questions look like measure what they are supposed to? Predictive Validity -- Do you imagine that this measure would predict something that it logically should? Concurrent Validity – Do you think this measure correlates strongly with something that it logically should?

Criterion-Related Validity A method for assessing the validity of an instrument by comparing its scores with another criterion known already to be a measure of the same trait or skill.

PREDICTIVE VALIDITY The extent to which a procedure allows accurate predictions about a subject’s future behavior.

CONTENT VALIDITY Whether the individual items of a test represent what you actually want to assess

FACTORS AFFECTING VALIDITY 1.Test-related factors 2.The criterion to which you compare your instrument may not be well enough established 3.Intervening events 4.Reliability

RELIABILITY The consistency of measurements A RELIABLE TEST Produces similar scores across various conditions and situations, including different evaluators and testing environments.

12 What is Reliability? Reliability is: – the consistency of your measurement instrument – the degree to which an instrument measures the same way each time it is used under the same condition with the same subjects

© A. Taylor Do not duplicate without author’s permission 13 Reliability Imagine that you are using a ruler to measure a book And it measures about 7 inches across What do you think would happen if you waited 10 minutes and measured the book again, how long would it be then? …Probably still 7 inches What if you spun the ruler around! And shook it up really good?! Now what would it say? …Probably still 7 inches

© A. Taylor Do not duplicate without author’s permission 14 Reliability Your ruler… – was consistent – measured the same way each time it was used under the same condition with the same object The book did not change and therefore the ruler reported back the same measurement Your ruler is RELIABLE

© A. Taylor Do not duplicate without author’s permission 15 Reliable but not Valid Remember our reliable ruler? Can it measure how loud the radio is? how full the glass is? how smart the girl is? The ruler may be reliable (and perhaps even valid) but not in these situations! It is only valid for measuring length.

RELIABILITY COEFFICIENTS The statistic for expressing reliability. Expresses the degree of consistency in the measurement of test scores. Donoted by the letter r with two identical subscripts (r xx )

Types of reliability Inter-rater: Different people, same test. Inter-rater Two people may be asked to categorize pictures of animals as being dogs or cats. A perfectly reliable result would be that they both classify the same pictures in the same way. Test-retest: Same people, different times. Test-retest Parallel-forms: Different people, same time, different test. Parallel-forms An experimenter develops a large set of questions. They split these into two and administer them each to a randomly- selected half of a target sample. Internal consistency: Different questions, same construct. Internal consistency When asking questions in research, the purpose is to assess the response against a given construct or idea. Different questions that test the same construct should give consistent results.

FACTORS AFFECTING RELIABILITY 1.Test length 2.Test-retest interval 3.Variability of scores 4.Guessing 5.Variation within the test situation