Presentation on theme: "Classroom Assessment A Practical Guide for Educators by Craig A"— Presentation transcript:
1 Classroom Assessment A Practical Guide for Educators by Craig A Classroom Assessment A Practical Guide for Educators by Craig A. MertlerChapter 3Characteristics ofAssessments
2 IntroductionThe quality of educational decisions is only as good as the information that leads to them.If inappropriate information is collected, or if it is collected without precision, the decisions that follow will logically be inaccurate.Two key characteristics of all assessments are validity and reliability.
3 What Is Validity?Validity: the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests.does not emphasize the results themselves, but rather how the results are used (validity deals with the decisions that follow the interpretation of test results—i.e., appropriate and inappropriate uses of assessment results)the most fundamental consideration when developing and evaluating tests and other assessmentsexample—the SMART
4 What Is Validity? Validity (continued) Three important points about validity:concept of validity applies to the ways teachers interpret and use assessment resultsassessment results have different degrees of validity, depending on purposesjudgments about validity should be made only after examination of several types of validity evidence
5 What Is Validity? Sources of Validity Evidence validity is an abstract concept; it cannot be directly observed; must gather evidence in support of itcontent evidence of validityfocuses on extent to which content addressed by assessment items adequately samples the larger domain of performancemost important type of evidence for teachersrelevance: do items emphasize what has been taught?representativeness: how well do the items represent the total content area?two-way tables (“content” x “taxonomic level”) can assist teachers in gathering this evidence
6 What Is Validity? Sources of Validity Evidence (continued) criterion evidence of validityfocuses on extent to which scores resulting from an assessment are related to another similar, well-established assessment (the criterion)predominantly a concern for standardized testsPredictive evidence of validity: form of criterion-related evidence where criterion is measured sometime in the future.Concurrent evidence of validity: form of criterion-related evidence where criterion is measured at the same time or consists of some measure available at the same time.
7 What Is Validity? Sources of Validity Evidence (continued) construct evidence of validityfocuses on degree to which there is a fit between hypothetical construct (unobservable human trait) being measured and the responses actually supplied by studentstypically a concern for standardized testssometimes viewed as an “umbrella” for all sources of validity evidence
8 What Is Validity? Sources of Validity Evidence (continued) face evidence of validitynot considered a formal source of evidenceinformal measure of extent to which the users or takers of tests believe that the test results are validoften plays an important role in terms of student (and teacher) motivation
9 What Is Validity? Establishing Validity of Quantitative Assessments for classroom assessments—five guiding questionsDoes my assessment procedure emphasize what I have taught?Do my assessment tasks accurately represent outcomes specified in my school’s, district’s, or state’s curriculum guide?Is the content in my assessment procedure important and worth learning?Do students perceive that the problems or tasks on my assessment emphasize the concepts and other material that I have taught?Do students generally believe that the assessment measures the appropriate behaviors, skills, or characteristics as they were taught?content evidenceface evidence
10 What Is Validity?Establishing Validity of Quantitative Assessments (continued)for standardized assessments—evidence of validity is based on statistical analysis (especially for criterion-related evidence)Correlation coefficient (r): statistical measure that indicates the extent to which scores on one assessment agree with scores on the other; ranges from to +1.00; known as a validity coefficient.
11 What Is Validity? Establishing Validity of Qualitative Assessments for informal classroom assessments—five questionsHave I limited my observations to concrete behaviors, as opposed to more global impressions of students?Have I observed/noted the specific behavior a sufficient number of times in order to draw definitive conclusions?Have I observed/noted the behavior in different settings or situations?Have I based my conclusions only on the information that I have gathered?Are there plausible, alternative explanations for the given behavior?representa-tiveness of observationsnature of inferences
12 What Is Reliability?Reliability: the consistency of measures when the testing procedure is repeated on a population of individuals or groups.validity = accuracy; reliability = consistencyalso speaks to scores and their interpretation and use, not to the assessment itselfscores—and their consistency—are affected by errorerror can result from student illness, content assessed but not taught, etc.random errors affect consistency; systematic errors affect validity
13 What Is Reliability?Establishing Reliability of Quantitative Assessmentsestablished by correlating test results with themselves or with other forms of the test (anticipate that high scores on one form of the assessment are associated with high scores on the other)Reliability coefficient (r): a correlation coefficient representing measures of reliability.
14 What Is Reliability?Establishing Reliability of Quantitative Assessments (continued)Test-retest method: estimates reliability over time; results in a coefficient of stability.procedure is not realistic for classroom useAlternate-forms and equivalent-forms methods: administration of tests with different items, or same items, that have been rearranged; results in an alternate-forms coefficient.again, procedure not realistic for classroom use
15 What Is Reliability?Establishing Reliability of Quantitative Assessments (continued)Internal consistency methods: estimate of reliability with only one administration; determines how well items correlate with one another.Split-half method: divides test into two comparable halves.KR-21 method: all possible split-half combinations.Cronbach’s a method: similar to KR-21, but for items with different point values.
16 What Is Reliability?Establishing Reliability of Qualitative AssessmentsInterrater consistency: calculation of percent agreement between two or more raters of student performance.
17 The Relationship Between Validity and Reliability Validity is the more important feature.Reliability is a prerequisite to validity (in other words, if items accurately assess a domain of content, the scores will also be consistent).Assessment results may be reliable (i.e., consistent) but not valid (i.e., accurate).
18 The Relationship Between Validity and Reliability Valid test results are also reliable, but reliable test results are not necessarily valid.
19 Teacher Responsibilities Related to Validity and Reliability Ensuring the validity and reliability of classroom assessments is a primary responsibility of teachers.Refer to both The Standards for Teacher Competence in the Educational Assessment of Students and The Code of Fair Testing Practices in Education.