Meaningful Inference b Says something about the meaning of the information obtained through an instrument
Useful Inference b Helps researchers make decisions related to what they studied
Do we get useful information about the topic/variables?
Validity depends on: b The amount and type of evidence supporting the interpretations
Types of Evidence b Content-related b Criterion-related b Construct-related
Content-related b Appropriate? b Comprehensive? b Logical? b Adequate?
b Adequacy of the sampling representative of content?representative of content? b Format of the instrument language, directions, etc.language, directions, etc. b Who judges the validity of the content? InterpretationInterpretation judgementjudgement
Criterion-related b Relationship between scores from different instruments b Strong relationship? b Accurate?
b Instrument results compared with independent criterion a second test or measurementa second test or measurement b Predictive validity time between measurementstime between measurements b Concurrent validity instrument and criterion measurements at same timeinstrument and criterion measurements at same time
b Validity Coefficient r = correlation coefficientr = correlation coefficient degree of relationship between scores from instrument and criteriondegree of relationship between scores from instrument and criterion Between -1.00 and +1.00Between -1.00 and +1.00.00 = no relationship.00 = no relationship
Construct-related b How well does the theoretical construct account for or explain differences? b Is the construct sufficient?
b Three steps: variable measured clearly definedvariable measured clearly defined hypotheses (based on theory)hypotheses (based on theory) hypotheses tested logically and empiricallyhypotheses tested logically and empirically b Collect variety of different types of evidence
Reliability b Consistency of the scores among individualsamong individuals among itemsamong items
b Reliable -- valid or not valid b Not reliable -- not valid b Validity and reliability dependent on the context in which the instrument is used.
Measurement Error b Same test, different scores b Caused by variety of factors: motivationmotivation anxietyanxiety environmentenvironment etc.etc.
Reliability Coefficient b Reliability estimates b Relationship between scores at two different times, or b Relationship between two parts of the same instrument
Test-Retest b Same test, same group, two times b Reliability coefficient indicates relationship between the two scores
Equivalent-Forms b Two different, equivalent tests, same group b Reliability coefficient calculated
Internal-Consistency b Single administration of the instrument b Split-half procedure b Kuder-Richardson approach b Alpha coefficient
Split-Half Procedure b Score two halves of test separately b Degree to which the two halves provide same results b Reliability increases with length of the test/instrument
Standard Error of Measurement b Index showing how much a measurement would vary under changed circumstances