1. University of Colorado at Boulder/CRESST Promoting Effective Classroom Assessment in the Context of NCLB Lorrie A. Shepard University of Colorado at Boulder Washington Educational Research Association March 4, 2005

2. University of Colorado at Boulder/CRESST Assessment Chapter in Preparing Teachers  Formative assessment – carried out during the instructional process to improve teaching and learning  Summative assessment and grading  Appropriate preparation for and use of high-stakes accountability tests

3. University of Colorado at Boulder/CRESST The knowledge base for classroom assessment is closely tied to contemporary theories of learning -- about how knowledge is organized in the mind and about how participation in communities of practice shapes understanding. How People Learn Bransford, Brown, & Cocking, 1999, NRC Knowing what Students Know Pellegrino, Chudowsky, & Glaser, 2001, NRC

4. University of Colorado at Boulder/CRESST Reformed Vision of Curriculum All students can learn. Challenging subject matter aimed at higher order thinking & problem solving Equal opportunity for diverse learners Socialization into the discourse & practices of academic disciplines Authenticity in the relationship between learning in & out of school Fostering of important dispositions and habits of mind Enactment of democratic practices in a caring community Cognitive & Constructivist Learning Theories Intellectual abilities are socially and culturally developed. Learners construct knowledge & understandings within a social context. New learning is shaped by prior knowledge and cultural perspectives. Intelligent thought involves “meta-cognition” or self monitoring of learning and thinking. Deep understanding is principled and supports transfer. Cognitive performance depends on dispositions and personal identity. Classroom Assessment Challenging tasks to elicit higher order thinking Addresses learning processes as well as learning outcomes An on-going process, integrated with instruction Used formatively in support of student learning Expectations visible to students Students active in evaluating their own work Used to evaluate teaching as well as student learning Shared principles of curriculum theory, learning theory, and assessment theory. From Shepard, Educational Researcher, 2000.

5. University of Colorado at Boulder/CRESST Social-Constructivist and Sociocultural Learning Theories  Intellectual abilities and an identity of mastery are socially and culturally developed.  Scaffolding refers to the support – guidance, coaching, hints, and encouragement – that adults provide in the zone of proximal development (ZPD) to enable and challenge the learner to perform at a level that she would not otherwise be able to reach.

6. University of Colorado at Boulder/CRESST Formative Assessment // Instructional Scaffolding Sadler’s formative assessment model  Where are you trying to go?  Where are you now?  How can you get there? A shared understanding and ownership of the learning goal, support in reaching the goal, internalization, and taking over of responsibility by the learner are closely parallel features of formative assessment and scaffolding.

7. University of Colorado at Boulder/CRESST To enable more insightful formative assessment strategies, classroom assessment must change in two fundamentally important ways. First, its form and content must be changed to better represent important thinking and problem solving skills in each of the disciplines. Second, the way that assessment is used in classrooms and how it is regarded by students must change.

8. University of Colorado at Boulder/CRESST The Importance of Content: Instructional and Assessment Tasks that Embody Learning Goals Rather than alignment, embodiment might be a better term to characterize the more complete and substantive alignment that occurs when the tasks, problems, and projects in which students are engaged represent the range and depth of what we want students to understand and be able to do. Wiggins and McTighe use “backward design” to get from intended goals to compelling evidence or demonstrations of learning.

9. University of Colorado at Boulder/CRESST A Sampler of Science Assessment © California Department of Education

10. University of Colorado at Boulder/CRESST A Sampler of Science Assessment © California Department of Education

11. University of Colorado at Boulder/CRESST © Wyoming Body Evidence Activities Consortium

12. University of Colorado at Boulder/CRESST © Wyoming Body Evidence Activities Consortium

13. University of Colorado at Boulder/CRESST Note that good assessment tasks are interchangeable with good instructional tasks. (The exact same task should not be used for both purposes, however.) We also have evidence that “teaching to” problem types like these improves learning.

14. University of Colorado at Boulder/CRESST Recommendations for Teachers in a Context of High-Stakes Testing  Construct domain maps (using curriculum frameworks) to highlight explicitly what the high-stakes test covers and what it leaves out.  Commit to teaching to the full domain with limited attention to test preparation.  Help students see the connection between what they will be asked to do to show what they know for the test and the larger range of ways they will be asked to use their knowledge in school and in the world.  Educate parents and school board members about what tests leave out by systematically providing examples of student work.

15. University of Colorado at Boulder/CRESST Classroom Instruction & Assessment Accountability Testing

16. University of Colorado at Boulder/CRESST Classroom Instruction & Assessment Accountability Testing

17. University of Colorado at Boulder/CRESST

21. Formative Assessment Shifts the Purpose of Assessment from Grading to Learning Assessment of Learning vs. Assessment for Learning Assessment Reform Group England, Northern Ireland, Scotland, Wales

22. University of Colorado at Boulder/CRESST

24. Prior knowledge Prior knowledge techniques – Instructional conversations (Tharp & Gallimore, 1988) K-W-L (Ogle, 1986) are not seen as assessments but yield valuable data for revising instruction. Prior knowledge is more than facts amassed at home. It includes language patterns and ways of thinking developed through social roles and cultural experiences. Moll, Amanti, Neff, & Gonzalez (1992) use "funds of knowledge" to describe household knowledge of children from poor families based on farming,carpentry, childcare, cooking, medicine, religion, and budget management that can be used to support school knowledge.

25. University of Colorado at Boulder/CRESST Feedback Feedback is most effective when it focuses on correction of errors in relation to original learning goals. “A key role of the scaffolder is to summarize the progress that has been made and point out behaviors that led to the successes … One type of feedback is pointing out the distinction between performance and the ideal … another type of feedback is attributing success to effort … and explicitly restating the concept that has been learned” (Hogan & Pressley, 1997)

26. University of Colorado at Boulder/CRESST An Intervention Study Elawar & Corno (1985). A factorial experiment in teachers’ written feedback on student homework: changing teacher behavior a little rather than a lot, Journal of Educational Psychology. Study design: Teachers were trained to give written feedback focused on specific errors and poor strategy with suggestions about how to improve. The control group received grades on homework but no comments. Findings: The effect of focused feedback on final achievement was as great as the effect of prior achievement. There were also large positive effects on attitudes toward mathematics and the initial superiority of boys over girls was reduced.

27. University of Colorado at Boulder/CRESST

28. Teaching and Assessing for Transfer “Teaching for Robust Understandings” Multiple Ways to Ask about the Concept One-Half 1. Circle each shape that has one-half shaded Source: Assessing Mathematical Understanding, 1989

29. University of Colorado at Boulder/CRESST Measuring Up, Mathematical Sciences Education Board, National Research Council, 1993.

30. University of Colorado at Boulder/CRESST Measuring Up, Mathematical Sciences Education Board, National Research Council, 1993.

31. University of Colorado at Boulder/CRESST Explicit criteria The transition from feedback to self- monitoring can occur only when the student comes to know what constitutes quality. (Sadler, 1989) “The assessment system (should) provide a basis for developing a metacognitive awareness of what are important characteristics of good problem solving, good writing, good experimentation, good historical analysis, and so on.” (Frederiksen & Collins, 1989)

32. University of Colorado at Boulder/CRESST Student self assessment promises to increase students’ responsibility for their own learning. In case studies, students became more interested in the criteria and substantive feedback than grades… more honest about their own work, fair with other students, and able to defend their opinions in terms of the evidence. (Klenowski, 1995) Self assessment

33. University of Colorado at Boulder/CRESST An Intervention Study White & Frederiksen (1996). The Thinker Tools Inquiry Project: Making Scientific Inquiry Accessible to Students Center for Performance Assessment, Educational Testing Service. Assessment criteria were developed for attributes desired while conducting investigations in science. Students engaged in a set of activities to foster “reflective assessment.” è At several stages in the Inquiry Cycle curriculum, students evaluated their own work in terms of the criteria. è Each time they applied the criteria AND wrote a brief rationale pointing to the features of their work that supported their rating. è Students in the reflective assessment classrooms also used the criteria to give feedback to classmates after oral presentations. Compared to controls, students in reflective classrooms produced more highly rated projects (with the greatest gains for low-achieving students). Low-achieving students also showed dramatic gains on a measure of conceptual understanding.

34. University of Colorado at Boulder/CRESST Evaluation of teaching Assessment should be used to evaluate teaching as well as to improve student learning. If we want the cultural practices in the classroom to support development of students’ identities as learners – where students naturally seek feedback and critique their own work – then it is reasonable that teachers would model this same commitment to using data systematically as it applies to their own role in the teaching and learning process.

35. University of Colorado at Boulder/CRESST Summative Assessment and Grading  Summative assessment and grading pose a serious threat to the learning purposes avowed for formative assessment.  If tests diverge from valued learning goals, students focus only on the graded portion of the curriculum.  The use of grades as rewards and punishments undermines intrinsic motivation to learn.  Using effort grades to manage student behavior is not the same thing as creating a learning environment that is motivating to students.

36. University of Colorado at Boulder/CRESST Students with a Performance Orientation (Extrinsically motivated)  Believe in fixed ability  Work toward “performance goals,” i.e., for grades, to please the teacher, and to appear competent  Focus on the “exchange value” of learning  Performance-oriented students pick easy task and are less likely to persist once they encounter difficulty.  Girls are over represented in this category. Stipek, 1996

37. University of Colorado at Boulder/CRESST Students with a Learning Orientation (Intrinsically motivated)  Attribute success to their own efforts  Work toward “learning goals,” i.e., to increase a sense of mastery and to become competent  Focus on the “use value” of learning  Learning-oriented students are more engaged in schoolwork, use more self-regulation, and develop deeper understanding of subject matter. Stipek, 1996

38. University of Colorado at Boulder/CRESST Mutually Supportive Formative and Summative Assessments To be mutually supportive, formative and summative assessments must be conceptually aligned. They should represent important learning goals using the same broad range of tasks and problem types to tap students’ understandings. Summative assessments should not be repeats of earlier formative tasks but should require students to use their knowledge in ways that generalize and extend what came before. Summative assessments are milestones on the same learning continua that under gird formative assessment.

39. University of Colorado at Boulder/CRESST To Foster a Learning Environment, Findings from Cognitive and Motivation Research Should Be Brought Together.  Focus efforts on developing competence.  Provide diverse opportunities to demonstrate mastery.  Adapt instruction to students’ knowledge and understanding.  Provide opportunities for students to take responsibility for learning.  Emphasize working hard and learning rather than right answers.  Make thinking visible in classroom discourse.  Treat errors and mistakes as a normal part of learning.  Evaluate progress as well as outcomes.  Help students learn the meaning of criteria for good work.  Provide opportunities for students to improve competence based on feedback.  Base grades on mastery standards rather than social norms.

40. University of Colorado at Boulder/CRESST Grading Practices that Emphasize Learning Goals  Grades should represent achievement, not normative comparisons.  Achievement-based grades will be more transparently aligned with feedback on the same standards used for formative assessment and will communicate better to external audiences.  A commitment to achievement-based grading would mean doing away with compliance grading practices such as extra credit points, points for turning in homework that is never graded, and so forth.  Other ways to soften student worries about grades can be allowed so long as they provide opportunities for students to demonstrate mastery – e.g., replacement assignments and replacement tests or throwing out test scores when learning is verified by later assessments.

41. University of Colorado at Boulder/CRESST

42. Coda: Cautions Regarding Data-Driven Systems In the current political and commercial context, it is important to emphasize that readily available data- systems and benchmark tests do not necessarily reflect desired content and may not be supportive of effective formative assessment. Data-systems that routinize the use of standardized assessments throughout the year, using formats that imitate external accountability tests, narrow the curriculum and limit the generalizability of achievement gains. They should not be thought of as first-steps in implementing the assessment for learning vision. Teachers need support to learn how to gain greater insight about student thinking – not more tests scores.