TRANSFORMATIVE ASSESSMENT IN ACTION TRANSFORMATIVE ASSESSMENT IN ACTION W. James Popham University of California, Los Angeles 2011 Teacher Leader Institute Minnesota ASCD Brainerd, Minnesota October 5-7, 2011

CUDDLING UP TO THE CURRENT CONTEXT IN EDUCATION AA Personal Perspective A National PerspectiveA National Perspective

THE GHOSTS OF EDUCATION PAST, EDUCATION PRESENT, AND EDUCATION FUTURE (With Apologies to Charles Dickens)

THE PAST: A SLOW BUT RELENTLESS MOVE TOWARD FEDERAL INFLUENCE ON LOCAL CLASSROOM EVENTS THE PRESENT EDUCATIONAL SCENE:THE PRESENT EDUCATIONAL SCENE: 3 DELPHI-DEFYING DEVELOPMENTS! 3 DELPHI-DEFYING DEVELOPMENTS! THE FUTURE OF U.S. PUBLIC SCHOOLS: BOTH SHORT-TERM AND LONG-TERM PERILS TO BE DODGEDTHE FUTURE OF U.S. PUBLIC SCHOOLS: BOTH SHORT-TERM AND LONG-TERM PERILS TO BE DODGED

THE COMMON CORE STATE STANDARDS (CCSS) Developed by CCSSO and the NGA Center (Council of Chief State School Officers and the National Governors Association Center for Best Practices)Developed by CCSSO and the NGA Center (Council of Chief State School Officers and the National Governors Association Center for Best Practices) Fewer, Clearer, Higher?Fewer, Clearer, Higher? Although created rapidly and with scant external scrutiny, getting the CCSS so widely adopted is an amazing accomplishment by CCSSO and NGA.Although created rapidly and with scant external scrutiny, getting the CCSS so widely adopted is an amazing accomplishment by CCSSO and NGA.

THE TWO ASSESSMENT CONSORTIA The Partnership for Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced Assessment Consortium (SBAC)The Partnership for Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced Assessment Consortium (SBAC) Although many similarities exist between the two consortia, key differences are also present.Although many similarities exist between the two consortia, key differences are also present.

Workshop Foci: Four “What-It-Is” Understandings and a Perplexing Puzzle What It Is and What It Isn’t What It Can Do and What It Can’t Two Cuts of Formative Assessment’s Cake Why Learning Progressions Must Lurk ************* Why Aren’t More Minnesota Teachers Using Formative Assessment?

Formative Assessment: What It Is Formative assessment is a planned process in which assessment-elicited evidence of students’ status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning-tactics.

Formative Assessment: What It Isn’t It is not a test. It is not an interim test (also referred to as a benchmark or periodic test) administered every few months by schools or districts. It is not the unplanned, serendipitous use of student cues to adjust teaching.

Formative Assessment: What It Can Do In a research review based on 250 empirical studies of classroom assessment that had been drawn from more than 680 published investigations, Paul Black and Dylan Wiliam concluded: “The research reported here shows conclusively that formative assessment does improve learning.” (Assessment in Education, 1998)

Formative Assessment: What It Can Do Two Other Quotes from the Research Review: The student gains in learning triggered by formative assessment were “amongst the largest ever reported for educational interventions.” “Significant gains can be achieved by many different routes, and initiatives here are not likely to fail through neglect of delicate and subtle features.”

AND, MORE RECENTLY: “Five reviews of the research in this area synthesized a total of more than 4,000 research studies undertaken during the last 40 years. The conclusion was clear: When implemented well, formative assessment can effectively double the speed of student learning.” (Wiliam, Educational Leadership, )

AND, EVEN MORE RECENTLY: There is now a strong body of theoretical and empirical work that suggests that integrating assessment with instruction may well have unprecedented power to increase student engagement and to improve learning outcomes.” (Wiliam, D., 2011, Studies in Educational Evaluation) “There is now a strong body of theoretical and empirical work that suggests that integrating assessment with instruction may well have unprecedented power to increase student engagement and to improve learning outcomes.” (Wiliam, D., 2011, Studies in Educational Evaluation)

On October 13, 2009, I received an e- mail from a colleague in Ohio, Dr. Saundra Brennan, who relayed to me an invitation she had received from the Ohio Association of Elementary School Administrators. It was to take part in: A FORMATIVE ASSESSMENT WORSHIP BESIDES AN ANALYTICAL AND EMPIRICAL RATIONALE, HERE’S A SPOT OF SPIRITUAL APPROBATION:

Formative Assessment: What It Can’t Do It cannot raise scores sufficiently on instructionally insensitive accountability tests such as those so widely used these days to satisfy the requirements of the No Child Left Behind Act.

A DEFINITION OF INSTRUCTIONAL SENSITIVITY A DEFINITION OF INSTRUCTIONAL SENSITIVITY The degree to which students’ performances on a test accurately reflect the quality of instruction specifically provided to promote students’ mastery of what is being assessed.

WHY MIGHT A TEST ITEM BE INSTRUCTIONALLY INSENSITIVE? Alignment LeniencyAlignment Leniency Excessive EasinessExcessive Easiness Excessive DifficultyExcessive Difficulty Confusion-Engendering Item FlawsConfusion-Engendering Item Flaws Socioeconomic Status (SES) LinksSocioeconomic Status (SES) Links Academic Aptitude LinksAcademic Aptitude Links

SOCIOECONOMIC STATUS (SES) LINKS If an item gives a meaningful advantage to students from higher SES families, then the item will tend to measure what students bring to school rather than how well they are taught once they get there.

A 6th-Grade Science Item: A plant’s fruit always contains seeds. Which of the items below is not a fruit?A plant’s fruit always contains seeds. Which of the items below is not a fruit? A.orange B.pumpkin C.apple D.celery

In which of the sentences below does the word field mean the same thing as in the sentence above? A.The shortstop knew how to field his position. B.We prepared the field by plowing it. C.What field do you plan to enter when you graduate? D.The nurse examined my field of vision. A 4th-Grade Reading Item: My father’s field is computer graphics.

ACADEMIC APTITUDE LINKS If an item gives a meaningful advantage to students who possess greater inherited quantitative, verbal, or spatial aptitudes, then the item will tend to measure what students bring to school rather than how well they are taught once they get there.

A 3rd-Grade Mathematics Item: The secret number is inside the circle. It is also inside the square. It is NOT inside the triangle. Which of these is the secret number?The secret number is inside the circle. It is also inside the square. It is NOT inside the triangle. Which of these is the secret number? A. 2B. 3 C. 5 D

A 4 th -Grade Mathematics Item A 4 th -Grade Mathematics Item: Which of the letters below, when folded in half, will have two parts that match exactly?Which of the letters below, when folded in half, will have two parts that match exactly? F(A)Z(B) S(C)B(D)

TWO WAYS OF SLICING UP THE FORMATIVE-ASSESSMENT PIE 1. Levels of Implementation, that is, Four Distinct Ways of Conceptualizing the Formative-Assessment Process 2. Applications of the Process, that is, Five Different Applications of the Formative-Assessment Process

Formative Assessment: Why Levels Can Lead to Lucidity Because there are two potential sets of players (teachers adjusting instruction and students adjusting learning tactics), it is often confusing to lump the two together. Because there are substantially different varieties and levels of implementation, clarity can be compromised by regarding one variant of formative assessment as coterminous with another variant.

Four Levels of Formative Assessment Level 4: Schoolwide Implementation Level 3: Classroom-Climate Shift Level 2: Students’ Learning-Tactic Adjustments Level 1: Teachers’ Instructional Adjustments Students’ Level 2 Steps 1. Consider adjustment occasions. 2.Consider assessments. 3.Consider adjustment triggers. 4.Adjust learning-tactics? Teachers’ Level 1 Steps 1.Identify adjustment occasions. 2.Select assessments. 3.Establish adjustment triggers. 4.Make instructional adjustments? Level 4 Strategies 1.Professional Development 2. Teacher Learning Communities Level 3 Shifts 1.Learning Expectations 2.Responsibility for learning 3.Role of classroom assessment Learning Progressions

The Formative-Assessment Process Immediate Instructional Adjustments Near-Future Instructional Adjustments Last-Chance Instructional Adjustments Students’ Learning Tactic Adjustments Classroom- Climate Shifts

FIVE APPLICATIONS OF THE PROCESS Immediate Instructional Adjustments (based on assessments or self-reports)Immediate Instructional Adjustments (based on assessments or self-reports) Near-Future Instructional AdjustmentsNear-Future Instructional Adjustments Last-Chance Instructional AdjustmentsLast-Chance Instructional Adjustments Students’ Learning-Tactic AdjustmentsStudents’ Learning-Tactic Adjustments Classroom Climate ShiftsClassroom Climate Shifts

Formative Assessment: Why Learning Progressions Must Lurk (What a learning progression is:) A learning progression is a sequenced set of building blocks (that is, subskills and/or bodies of enabling knowledge) it is thought students must master en route to mastering a more remote, target curricular aim.

TWO LEARNING PROGRESSIONS: Both Useful to Educators Upper-case LEARNING PROGRESSIONS Lower-case Learning Progressions

Target Curriculum Aim X Enabling Knowledge B Enabling Knowledge A Subskill A Subskill B An Illustrative Learning Progression

A Horizontally Represented Learning Progression Enabling Knowledge X Enabling Knowledge Y Subskill Z Target Curriculum Aim Q

Formative Assessment: What a Learning Progression is not A learning progression isn’t unerringly accurate. A particular learning progression isn’t going to work for all students. A learning progression isn’t necessarily better because it’s more complicated.

THE AFFECTIVE CONSEQUENCES OF FORMATIVE ASSESSMENT The significance of affect in general, and students’ affect specifically related to formative assessmentThe significance of affect in general, and students’ affect specifically related to formative assessment How the assessment of students’ affect will incline teachers to attend to students’ affective dispositionsHow the assessment of students’ affect will incline teachers to attend to students’ affective dispositions How students’ affect related to formative assessment can be measuredHow students’ affect related to formative assessment can be measured

Formative Assessment: And Now the Puzzle Given what we know about the success of formative assessment in boosting students’ achievement, why aren’t more Minnesota teachers using the formative-assessment process?

For a no-cost copy of a study guide for using Transformative Assessment with an extended-duration professional learning community, see the following: Scroll down to the book’s title and you’ll also find info about how to acquire this enchanting, yet insightful volume. The sequel, equally enchanting, contains Reflection Questions for each chapter rather than a study guide. My