Classroom Assessment Chapters 4 and 5 ELED 4050 Summer 2007

Nature of Validity 1. Validity refers to the appropriateness of the interpretation and use made of the results. (in layman’s terms validity refers to whether a test measures what it says it measures) 2. Validity is a matter of degree. 3. Validity is always specific to some particular use or interpretation. 4. Validity is a unitary concept. 5. Validity involves an overall evaluative judgment.

Chapter Four Learning Exercises # 2

Major Consideration in Assessment Validation The strongest case of validity can be made when the following considerations are made: 1. Content 2. Construct 3. Assessment- Criterion Relationship 4. Consequences

Content Considerations 1. The goal in the consideration of content validation is to determine the extent to which a set of assessment results are made.

Methods Used in Construct Validation Defining the domain or tasks to be measured. 2. Analyzing the response process required by the assessment tasks. 3. Comparing the scores of unknown groups 4. Comparing scores before and after a particular learning experience or experimental treatment. 5. Correlation the scores with other measures.

Correlation

Consequences Pg.92 3rd paragraph through page 94 Pg.92 3rd paragraph through page 94

Factors influencing validity 1. Unclear directions 2. Reading vocabulary and sentence structure too difficult (construct-irrelevant variance). 3. Ambiguity 4. Inadequate time limits (construct-irrelevant variance). 5. Overemphasis of easy-to-assess aspects of domain at the expense of important but difficult-to-assess aspects (construct under-representation). 6. Test items inappropriate for the outcomes being measured.

Factors influencing validity (Continued) 7. Poorly constructed test items 8. Test too short 9. Improper arrangement of items 10. Identifiable pattern of answers

Reliability 1. Reliability refers to the results obtained with an assessment instrument and not the instrument itself (in layman’s terms, a test that consistently yields the same or similar scores. 2. An estimate or reliability always refers to a particular type of consistency. 3. Reliability is a necessary but not sufficient condition for validity. 4. Reliability is assessed primarily with statistical indices.

Methods of Determining Reliability Test-Retest Method Test-Retest Method Equivalent forms Equivalent forms Test-retest with equivalent forms Test-retest with equivalent forms Split-half Split-half Kuder-Richardson and coefficient alpha Kuder-Richardson and coefficient alpha Interrater Interrater

Factors influencing reliability measures

Usability Ease of Administration Ease of Administration Time required for Administration Time required for Administration Ease of Interpretations and application Ease of Interpretations and application Availability of Equivalent or Comparable Forms Availability of Equivalent or Comparable Forms Cost of Testing Cost of Testing

Learning Exercises Chapter Four Number 5 Number 5 Chapter Five Number 3 Number 3 Number 4 Number 4 Number 5 Number 5 Number 10 Number 10