McGraw-Hill/Irwin © 2012 The McGraw-Hill Companies, Inc. All rights reserved. Obtaining Valid and Reliable Classroom Evidence Chapter 4:

4-2  Assessment can be a recognizable event or operation performed in the classroom.  It is also an instructional action and process used to directly impact students’ learning.  All assessments need to provide accurate and consistent information about a learner’s progress and/or performance. Assessment is a “thing” and a “do”

4-3  Evidence should be collected with multiple measures and methods.  Methods must accurately measure what they were designed to measure.  Methods must also be consistent in the data they provide.  Validity and reliability are essential qualities for any measurement process and/or procedure.

4-4  Validity is the degree to which a test measures what it is designed to measure so results can be correctly interpreted and utilized.  The most important element of any measurement ◦ It must be the essential core attribute of any assessment process.  Validity has been viewed as a unitary entity (Reynolds, Livingston, & Willson 2006). ◦ Evidence is collected to support the accuracy and authenticity of test scores and their interpretation.

4-5  Content-Related: the review and confirmation of material that is supposed to be measured by a test of some other measure.  Criterion-Related: the relationship between scores on a particular test with some external criterion (e.g., ACT or SAT). ◦ Validity coefficient is generated to measure the strength of the relationship between the measures  Construct-Related: the collection of confirmatory evidence demonstrating the existence of recognized concepts or constructs a test contends to measure.

4-6  Reliability: the consistency of measurement  High reliability means that similar results will be obtained each time a measurement is taken.  Reliability is a good and desirable quality  How do you get it? ◦ Minimize procedural error  One of the primary reasons standardization procedures are followed with formal measures. Reliability: The Same Every Time

4-7  Stability/Test-Retest  Alternate Form  Internal Consistency In Education, Three Major Types of Reliability to Focus On

4-8  Stability deals with consistency of results from a measure: ◦ Two administrations close in time ◦ Examined through a correlation coefficient (+1.0 to -1.0)  Commonly associated with standardized tests, but it can be connected with classroom measures.

4-9  Two parallel or equivalent forms of a test are developed, both forms are administered, and a correlation between the two performances is computed.  Both forms must be comparable to each other in several key areas: ◦ Material coverage and content ◦ Item difficulty ◦ Length ◦ Administration procedures If a high correlation exists than either form can be given with confidence. Alternate Form Reliability

4-10  This is connected with examining how consistently items within a test correspond to each other and uniformly measure a concept.  By design, questions and items that make up a test should be correlated if they are attempting to measure the same thing.  Two basic approaches: ◦ 1) Split halves ◦ 2) Odd-even Internal Consistency Reliability

4-11  Split halves involves separating a test into two equal halves. ◦ Doesn’t always work if the difficulty level of the items is not evenly distributed.  Odd-Even: All even numbered test items make up one half and the odd numbered items make up the other half of a test. Split Halves/Odd-Even

4-12  Test scores do not reflect perfect measurement of human performance ◦ If a test or procedure claims it does, throw it out.  If carefully constructed, administration factors are held constant, and special events do not take place: ◦ Scores obtained at one time should closely reflect scores that are obtained at another time. Keep In Mind…

4-13  Teachers must be aware of bias and how it can adversely impact and influence student learning evidence.  As Popham (2008) has reported, “Assessment bias refers to qualities of an assessment instrument that offend or unfairly penalize a group of students because of students’ gender, race, ethnicity, socioeconomic status, religion, or other group defining characteristics.” Assessment Bias

4-14  Assessment bias is directly connected to the materials and measures that teachers may use in their classroom. ◦ Biased instruments and/or procedures create a disadvantaged learning situation for certain students. ◦ It can consequently lessen and limit their academic progress as well as create a learning advantage for others.  Teachers don’t go out and try to develop biased assessments in their classrooms.  Usually teachers are unaware of how utilized procedures may cause certain students distress and disadvantage them. Assessment Bias

4-15  Bias is often found and associated with content and material used for tests or other evaluative measures. ◦ It typically involves how individual tasks and items are constructed.  For instance, if sport examples and references are used in a test, then these items may be biased in favor of students who are athletes and biased against those who are not. ◦ The student athletes may not have any greater skill base, but because they have a greater understanding of the context and details of the sporting situation(s) their chances of being successful with these items are enhanced. Assessment Bias

4-16  So what is a teacher suppose to do about assessment bias? ◦ Do your best to be aware of assessment bias and make sure that it does not become part of your teaching and assessment. ◦ Biased results invalidate any findings that you have and must be avoided at all costs. ◦ In developing sensitivity to this topic, you may become aware of things that you do or ideas that you hold that may need to be examined and possibly changed. Be Aware

4-17  Don’t assume common prior knowledge and/or experiences exist for your students.  Don’t assume directions are clear and understood by all your students.  Don’t surprise your students with test items or tasks they haven’t seen before.  Make sure the grading process and evaluation of student work is clearly explained and understood by your students. Some Useful Guidelines to Follow

4-18  As a teacher, you need to be knowledgeable about these concepts.  Educators must put into practice the information about reliability and validity.  Both reliability and validity must exist for all classroom assessment activities.  Bias must not exist within any assessment activities. So What Does This Mean to You