ASSESSMENT LITERACY: A BONA FIDE “MAGIC BULLET” FOR EDUCATION W. James Popham University of California, Los Angeles California Educational Research Association 88 th Annual Conference San Francisco November 18-19, 2009

THE DEFINITION OF A PROFESSION “The body of persons engaged in an occupation, e.g., the medical profession or the education profession”

AN UNPLEASANT TRUTH Any profession that allows the competence of its members to be determined by procedures over which it has little or no influence is a profession fully deserving of the low esteem with which, over time, its members are sure to be held. Any profession that allows the competence of its members to be determined by procedures over which it has little or no influence is a profession fully deserving of the low esteem with which, over time, its members are sure to be held.

UNPLEASANT TRUTH NO. 2 Any profession that allows the competence of its members to be determined by procedures over which it has little or no influence is a profession destined to underachieve its inherent potential. Any profession that allows the competence of its members to be determined by procedures over which it has little or no influence is a profession destined to underachieve its inherent potential.

UNPLEASANT TRUTH NO. 3 The education profession allows the competence of our members to be determined by procedures over which we have little or no influence, namely, procedures rooted in students’ performances on educational accountability tests. The education profession allows the competence of our members to be determined by procedures over which we have little or no influence, namely, procedures rooted in students’ performances on educational accountability tests.

WHY HAVE WE BEEN SO SILLY? The chief reason that the education profession has allowed the competence of its members to be determined by others is because of flat-out, pervasive, profound, Sunday-go-to-meeting: The chief reason that the education profession has allowed the competence of its members to be determined by others is because of flat-out, pervasive, profound, Sunday-go-to-meeting: ASSESSMENT ILLITERACY

WHAT IS ASSESSMENT LITERACY? Assessment literacy represents the degree to which someone understands the most fundamental measurement concepts that are focally relevant to the way we operate and evaluate our schools. Assessment literacy does not imply that someone must be able to procedurally implement all of these concepts—only to understand them. Assessment literacy represents the degree to which someone understands the most fundamental measurement concepts that are focally relevant to the way we operate and evaluate our schools. Assessment literacy does not imply that someone must be able to procedurally implement all of these concepts—only to understand them.

A NEW BOOK BY JIM WRITTEN FOR CORWIN PRESS When released? Spring/Summer, 2010 Possible titles? A. What School Leaders Must Know about Educational Assessment B. Twenty Crucial Understandings about Educational Measurement C. Seduction in the Sand D. None of the above?

MY TWO “MOST CRUCIAL” ASSESSMENT UNDERSTANDINGS  Why it is that we are evaluating our schools with the wrong accountability tests  A recognition that teachers’ use of the formative-assessment process can have a profoundly beneficial impact on students’ learning

TWO DOMINANT KINDS OF ACCOUNTABILITY TESTS IN THE U.S. “Augmented” versions of traditional, nationally standardized achievement tests. “Augmented” versions of traditional, nationally standardized achievement tests. Standards-based tests ostensibly assessing students’ mastery of a state’s officially approved content standards. Standards-based tests ostensibly assessing students’ mastery of a state’s officially approved content standards.

BOTH OF THESE TYPES OF ACCOUNTABILITY TESTS Tend to measure what students bring to school rather than how effectively those students have been taught once they arrive. Tend to measure what students bring to school rather than how effectively those students have been taught once they arrive.

WHERE IS THE VALIDITY EVIDENCE? WHERE ARE THE VALIDITY ARGUMENTS? If the inference at issue is whether students’ scores on accountability tests accurately reflect the success with which those students have been taught, then where is the evidence that today’s accountability tests can accomplish this measurement mission satisfactorily?

FORMATIVE ASSESSMENT Formative assessment is a planned process in which assessment-elicited evidence of students’ status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning-tactics. Formative assessment is a planned process in which assessment-elicited evidence of students’ status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning-tactics. (Me, 2008)

THE EVIDENTIARY BASIS In a research review based on 250 empirical studies of classroom assessment that had been drawn from more than 680 published investigations, Paul Black and Dylan Wiliam concluded: In a research review based on 250 empirical studies of classroom assessment that had been drawn from more than 680 published investigations, Paul Black and Dylan Wiliam concluded: “The research reported here shows conclusively that formative assessment does improve learning.” (Assessment in Education, 1998) “The research reported here shows conclusively that formative assessment does improve learning.” (Assessment in Education, 1998)

CLINCHERS—FOR ME Two Other Quotes from the Research Review:  The student gains in learning triggered by formative assessment were “amongst the largest ever reported for educational interventions.”  “Significant gains can be achieved by many different routes, and initiatives here are not likely to fail through neglect of delicate and subtle features.”

LET’S LOOP BACK 40+ YEARS TO THE 1960s, B.D.

OPTIONAL MEANINGS OF B.D. A. An era of benighted distress B. Before Dylan C. Both of the above

A RAPID REVIEW 1. For both professional survival and to achieve greater good, all educators must become assessment literate. 2. One focus for assessment-literate educators should be to improve the accountability tests used to judge us. 3. A second task for assessment-literate educators is to foster wider use of the formative-assessment process.

WHAT CAN YOU DO? 1. You can personally enhance your own assessment literacy. 2. You can work solo, with colleagues, or with associations to demand more appropriate accountability tests. 3. You can, with word and deed, encourage more teachers, schools, and districts to use the formative- assessment process.

YOU REALLY SHOULD!