1. McGraw-Hill/Irwin © 2012 The McGraw-Hill Companies, Inc. All rights reserved. Obtaining Valid and Reliable Classroom Evidence Chapter 4:

2. 4-2  Assessment can be a recognizable event or operation performed in the classroom.  It is also an instructional action and process used to directly impact students’ learning.  All assessments need to provide accurate and consistent information about a learner’s progress and/or performance. Assessment is a “thing” and a “do”

3. 4-3  Evidence should be collected with multiple measures and methods.  Methods must accurately measure what they were designed to measure.  Methods must also be consistent in the data they provide.  Validity and reliability are essential qualities for any measurement process and/or procedure.

4. 4-4  Validity is the degree to which a test measures what it is designed to measure so results can be correctly interpreted and utilized.  The most important element of any measurement ◦ It must be the essential core attribute of any assessment process.  Validity has been viewed as a unitary entity (Reynolds, Livingston, & Willson 2006). ◦ Evidence is collected to support the accuracy and authenticity of test scores and their interpretation.

5. 4-5  Content-Related: the review and confirmation of material that is supposed to be measured by a test of some other measure.  Criterion-Related: the relationship between scores on a particular test with some external criterion (e.g., ACT or SAT). ◦ Validity coefficient is generated to measure the strength of the relationship between the measures  Construct-Related: the collection of confirmatory evidence demonstrating the existence of recognized concepts or constructs a test contends to measure.

6. 4-6  Reliability: the consistency of measurement  High reliability means that similar results will be obtained each time a measurement is taken.  Reliability is a good and desirable quality  How do you get it? ◦ Minimize procedural error  One of the primary reasons standardization procedures are followed with formal measures. Reliability: The Same Every Time

7. 4-7  Stability/Test-Retest  Alternate Form  Internal Consistency In Education, Three Major Types of Reliability to Focus On

8. 4-8  Stability deals with consistency of results from a measure: ◦ Two administrations close in time ◦ Examined through a correlation coefficient (+1.0 to -1.0)  Commonly associated with standardized tests, but it can be connected with classroom measures.

9. 4-9  Two parallel or equivalent forms of a test are developed, both forms are administered, and a correlation between the two performances is computed.  Both forms must be comparable to each other in several key areas: ◦ Material coverage and content ◦ Item difficulty ◦ Length ◦ Administration procedures If a high correlation exists than either form can be given with confidence. Alternate Form Reliability

10. 4-10  This is connected with examining how consistently items within a test correspond to each other and uniformly measure a concept.  By design, questions and items that make up a test should be correlated if they are attempting to measure the same thing.  Two basic approaches: ◦ 1) Split halves ◦ 2) Odd-even Internal Consistency Reliability

11. 4-11  Split halves involves separating a test into two equal halves. ◦ Doesn’t always work if the difficulty level of the items is not evenly distributed.  Odd-Even: All even numbered test items make up one half and the odd numbered items make up the other half of a test. Split Halves/Odd-Even

12. 4-12  Test scores do not reflect perfect measurement of human performance ◦ If a test or procedure claims it does, throw it out.  If carefully constructed, administration factors are held constant, and special events do not take place: ◦ Scores obtained at one time should closely reflect scores that are obtained at another time. Keep In Mind…

13. 4-13  Teachers must be aware of bias and how it can adversely impact and influence student learning evidence.  As Popham (2008) has reported, “Assessment bias refers to qualities of an assessment instrument that offend or unfairly penalize a group of students because of students’ gender, race, ethnicity, socioeconomic status, religion, or other group defining characteristics.” Assessment Bias

14. 4-14  Assessment bias is directly connected to the materials and measures that teachers may use in their classroom. ◦ Biased instruments and/or procedures create a disadvantaged learning situation for certain students. ◦ It can consequently lessen and limit their academic progress as well as create a learning advantage for others.  Teachers don’t go out and try to develop biased assessments in their classrooms.  Usually teachers are unaware of how utilized procedures may cause certain students distress and disadvantage them. Assessment Bias

15. 4-15  Bias is often found and associated with content and material used for tests or other evaluative measures. ◦ It typically involves how individual tasks and items are constructed.  For instance, if sport examples and references are used in a test, then these items may be biased in favor of students who are athletes and biased against those who are not. ◦ The student athletes may not have any greater skill base, but because they have a greater understanding of the context and details of the sporting situation(s) their chances of being successful with these items are enhanced. Assessment Bias

16. 4-16  So what is a teacher suppose to do about assessment bias? ◦ Do your best to be aware of assessment bias and make sure that it does not become part of your teaching and assessment. ◦ Biased results invalidate any findings that you have and must be avoided at all costs. ◦ In developing sensitivity to this topic, you may become aware of things that you do or ideas that you hold that may need to be examined and possibly changed. Be Aware

17. 4-17  Don’t assume common prior knowledge and/or experiences exist for your students.  Don’t assume directions are clear and understood by all your students.  Don’t surprise your students with test items or tasks they haven’t seen before.  Make sure the grading process and evaluation of student work is clearly explained and understood by your students. Some Useful Guidelines to Follow

18. 4-18  As a teacher, you need to be knowledgeable about these concepts.  Educators must put into practice the information about reliability and validity.  Both reliability and validity must exist for all classroom assessment activities.  Bias must not exist within any assessment activities. So What Does This Mean to You