Assessment and accountability: From teachable moments to tests Peter Afflerbach University of Maryland afflo@umd.edu Reading Research Conference International Reading Association Atlanta, GA May 3, 2008

Some past history…and assessment Public School 33 Queens

Junior High School 109 Queens

Jamaica High School, Jamaica, Queens

“Leader of Tomorrow” Award 1974 Betty Crocker “Leader of Tomorrow” Award

The state of research on assessment Assessment as the dependent variable in much reading research Assessment itself is much less likely to be the focus of research Psychometric inquiry related to validity and reliability Questions about most useful, most effective, most efficient assessment need our attention (Afflerbach, 2007; Black & Wiliam, 1998; Calfee & Hiebert, 1991; Crooks, 1988; Davis, 1998; Pellegrino, Chudowsky & Glaser, 2001)

Assessment in the nexus Politics Economics Agendas READING ASSESSMENT Student growth and achievement Teacher professional development High quality teaching Zone of proximal development Curriculum materials

Framing questions and an observation Where is our assessment focus? Do we attend to one type of assessment at the expense of others? Is our assessment focus one that may actually impede student achievement and limit teacher effectiveness? Is our assessment one that supports students and teachers in their critical, daily work? How might a balance of assessment, focused on both teachable moments and tests, help us reach our goals?

is measured by a single test on a single day, but Accountability is measured by a single test on a single day, but accountability is accomplished with daily, useful assessments that inform our best instruction.

Accountability for teaching and learning will be realized when there is balance in literacy assessment, including balance in: • meeting the needs of different audiences and purposes of assessment • formative assessment and summative assessment • the assessment that is done to or for students and assessment that is done with and by students

…the need for balance in literacy assessment, including balance in: • the assessment of cognitive and affective factors related to literacy • the assessment of what students learn and how students use this knowledge • the demands for teacher and school accountability and professional development opportunities that help teachers develop expertise in assessment

The contexts in which we conduct reading assessment are influenced by (and reflect) political, economic and social factors In 2008… • The prevalence of high stakes testing, NCLB or no NCLB • Traditions and habits of assessment • The tests that our children take today look much like those that we took when we were in school • How does our current understanding of literacy influence our assessments? • How does our current understanding of assessment influence our assessments?

‘Saved by a Fly’ What is the main idea of the story? 1. A moose drinks all the water from a river 2. A group of animals is afraid of a big moose 3. A fly gets rid of a troublesome moose 4. A group of animals is always quarreling (Maryland State Assessment, 2006) ‘The Report Card’ What do you think Johnny did next? 1. hid his report card 2. sneaked away 3. scolded his mother 4. showed his report card to his parents (Nelson Denny Reading Test, 1966)

Necessary alignments for useful reading assessments: The construct of reading Reading standards and benchmarks Reading curriculum and instruction Reading assessment

1. Balance in meeting the needs of different audiences and purposes of assessment Audience Purpose Students To report on learning and communicate progress To motivate and encourage To teach children about assessment and how to assess their own work and progress To build student independence Teachers To determine nature of student learning To inform instruction To evaluate students and construct grades To diagnose student strengths and weaknesses Afflerbach, 2007

1. Balance in meeting the needs of different audiences and purposes of assessment (cont.) Audience Purpose Parents To inform of their children’s achievement To help connect home and school efforts to support student School To determine instructional program administrators effectiveness To prove school and teacher accountability Politicians To establish accountability of schools To inform the public of school progress Taxpayers To demonstrate that tax dollars are well spent Afflerbach, 2007

2. Balance in formative assessment and summative assessment Formative assessment provides information for: • Understanding individual students and their immediate needs • The teachable moment • Creating scaffolded approaches to instruction (Black & Wiliam, 1998; Crooks, 1988)

Summative assessment provides information for: • Comparing students’ literacy products • Judging students’ achievement in relation to benchmarks, standards and each other (National Reading Panel, 2000; Pellegrino et al, 2001) In a thoughtful assessment system, formative and summative assessment work together…

Summative -score at end of game -team standing at end of season -personal improvement from season to season Formative -ball control -kicking -passing -vision -resilience -position -speed -team play -creativity

Zone of proximal development, assessment and teachable moments Student's next level of competency and achievement Assessment and teaching Zone of proximal development Teaching and assessment Student's current level of competency and achievement (Vygotsky, 1978; Palincsar & Brown, 1984; Rosenshine & Meister, 1994; Palincsar & Herrenkohl, 2002)

A bit more on this… Student's next level of competency and achievement Where our best teaching happens Assessment and teaching Zone of proximal development Teaching and assessment Where accountability is created Student's current level of competency and achievement

Formative assessment is used for: Understanding individual students and their immediate needs Informing the teachable moment Creating scaffolded approaches to reading instruction Formative assessment done well leads to good news on summative assessments Is the daily assessment that identifies individual student needs and allows us to address them high stakes assessment ?

Important balance questions: Who is best served by the current array of assessments? What percentage of assessment across the school year is formative? Summative? Is this a good thing? A test score tells us little or nothing of the means by which it was achieved.

What does assessment mean to our students? 3. Balance the assessment that is done to or for students and assessment that is done with and by students What does assessment mean to our students? How do students conceptualize assessment? If a universal goal of reading instruction is to help students become independent and successful, are we doing enough to foster independence? How can assessment help foster this independence?

The need to make transparent and tangible what assessment “is”… “The black box” My work in my work out = The grade Independent and successful readers continually monitor and evaluate their learning and performance Metacognition and comprehension monitoring are learned… Who teaches them? How are they taught? (Black & Wiliam, 1998; Flavell, 1978; Markman, 1979; Pressley & Afflerbach, 1995)

Do classroom assessments help students become independent in assessment? Are students “insiders” or “outsiders” to the culture of assessment? Is assessment done to or for students? Is assessment done with and by students? Without the ability to self-assess reading (or any other learning), how will students be independent?

It is legitimate to further characterize the broadpoint appearance as a major archeological horizon marker for the eastern seaboard. In the terms of Willey and Phillips, a horizon is “a primarily spatial continuity represented by cultural traits and assemblages whose nature and mode of occurrence permit the assumption of a broad and rapid spread.” That a quick expansion of the broadpoint-using peoples took place is indicated by the narrow range of available radiocarbon dates, along with a correspondingly wide areal distribution of components. Once established, the broadpoint horizon developed as a “whole cultural pattern or tradition” in its own right by persisting and evolving over an expansive region for 500 to 1000 years.

Broadpoint is a type of arrowhead

How are we metacognitive?

With reading… Self-assessment happens when students have metacognitive strategies to use, the mindset that they should use them, and motivation to do so. Self-assessment is strategic, so our successes with strategy instruction can inform our teaching of self-assessment strategies. Explain, model, think-aloud and practice. (Pressley, 2005; Duffy, 1993)

Representative State Level Intended Learning Outcomes 4. Balance the assessment of cognitive and affective factors related to learning Representative State Level Intended Learning Outcomes Beginning in kindergarten and by the end of second grade students will be able to: 1. Demonstrate a positive learning attitude. 2. Develop social skills and ethical responsibility. 3. Demonstrate responsible emotional and cognitive behaviors. 4. Understand and use basic concepts and skills.

and learning benchmark descriptions The assessment of cognitive and affective factors that contribute to and influence achievement Most state and local standards address affective goals and outcomes in their curriculum goals and standards and learning benchmark descriptions Motivation to read Self-esteem as a reader Self-concept as a reader Are these assessed? To what end? (Chapman & Tunmer, 1995; Guthrie & Wigfield, 1997; CORI Projects; McKenna & Kear, 1990)

When we think of our teaching successes, do we think of students who scored well on tests? Or do we think of students who went from reluctant readers to enthusiastic readers? Do we think of students who evolved from easily discouraged readers to readers whose motivation helped them persevere through challenges? Do we remember students who avoided reading at all costs evolving into students who learned to love reading?

5. Balance the assessment of what students learn from reading and how students use this knowledge The 2009 Framework of the National Assessment of Educational Progress derives from expert consensus and conceptualizes reading as: Reading is an active and complex process that involves • Understanding written text • Developing and interpreting meaning • Using meaning as appropriate to type of text, purpose and situation (National Center for Education Statistics, 2007)

“Using meaning as appropriate to type of text, purpose and situation…” In the 21st century, can our students Identify claims and evidence in texts they read? Conduct critical evaluations of texts? Make judgments of texts and authors? Apply what they learn from reading to identify and solve problems? Synthesize information from text and non-text sources? (Coiro & Dobler, 2007; Gee, 2000)

The application of what is learned from reading Increasing complex performance and assessment Meaning construction and critical evaluation is assessed with … - Multiple choice items - Brief constructed responses - Extended constructed responses - Performances and judgments (Pellegrino, Chudowsky & Glaser, 2001)

Dolgeville, NY

This Factory's Bats Are Going, Going, Gone; As Home of McGwire's 'Big Stick,' Struggling Upstate Town Gets a Lift And that is the best economic news in a long, long time for Dolgeville, tucked in the hills north of the Mohawk River and 23 miles east of Utica. The century-old clapboard houses and the flower beds here are well tended, but behind the trim exterior, the village is fading, its population and job base declining steadily, like so many places upstate that have lost the manufacturing that once fed them. Jamestown lost its furniture makers and the Crescent wrench works, Troy lost the Arrow shirt collar factory, and Schenectady lost the American Locomotive Company and most of General Electric. Dolgeville's soon-to-be relic is the Daniel Green shoe factory, a great stone castle along East Canada Creek that once employed nearly 1,000 people, but has seen a series of layoffs. Most of the jobs have moved overseas, and the factory will cease production in June. RICHARD PEREZ-PENA NY Times April 25, 1999

6. Balance the demands for accountability with professional development opportunities that help teachers develop expertise in assessment How do teachers become expert at assessment? How can teachers attain reliability in their classroom-based assessments? Opportunity costs of different approaches to reading assessment (Darling-Hammond, 1997; Stiggins & Conklin, 1992)

How are teachers supported in the extremely challenging work of becoming expert at classroom assessment?

Professional development in classroom assessment is central to school success. For example, a teacher’s checklist for asking appropriate questions can be used to sharpen daily assessment routines: _____ I ask questions that are appropriately phrased and understood by students. _____ I ask questions that are at an appropriate level for the materials being covered. _____ I ask questions that require students to think at various intellectual levels. _____ My questions follow a logical sequence. _____ Student responses are used to guide my next questions. _____ My questions are consistent with the intended goals or objectives of the lesson. _____ I ask questions that assess student understanding. _____ I ask processing questions if a student’s answer is incomplete or superficial. _____ I encourage students to answer difficult questions by providing cues or rephrasing. _____ I avoid closed-ended questions that restrict students’ demonstration of learning. Slack, J. Questioning strategies to improve student thinking and comprehension. Southeast Educational Development Laboratory. Reprinted with permission.

Towards expertise in classroom assessment Vertical and horizontal reading assessment In for the long term Avoiding faddism Opportunity costs Connect teachable moments with tests (Afflerbach, 2007; Johnston, 1991)

Effective reading assessment is conceptualized and practiced Conclusions Effective reading assessment is conceptualized and practiced with clear understanding of how teachable moments connect with tests The extreme focus on high stakes tests, AYP and means to meet it skews school assessment towards summary statements, and away from valuable formative assessment

Such formative assessment is that which informs our teachable moments— Conclusions Such formative assessment is that which informs our teachable moments— Providing information that describes students’ zones of proximal development-- and where and what and how our instruction should address students’ needs Reading assessment is influenced by powers great and small--and this often creates imbalances for teachers and students

Our addressing imbalances should be informed by our goals Conclusions Our addressing imbalances should be informed by our goals for (and with) reading assessment To help us understand student learning and to help us guide student learning

This duality of reading assessment allows us to promote: Conclusions This duality of reading assessment allows us to promote: -Strategic, skillful reading -Lifelong, enthusiastic readers -The Matthew effect -Cognitive and affective and social development -Independence and self-assessment

Achieving balance in our Lastly… Achieving balance in our reading assessment programs, materials and procedures promotes accountability from teachable moment to test