Academic Rigor: Where Are We Now Looking at Student Reading Achievement and Increasing Rigor Using Grade 8 NAEP Item Maps and Percentile Graphs Prepared by Jeanne Foy Alaska State NAEP Coordinator

NAEP: the Common Measurement of Student Achievement among States The National Assessment of Educational Progress is called the “gold standard of assessment” Created in 1969; the No Child Left Behind Act (NCLB) mandated state participation in NAEP reading and math every other year Alaska has NAEP data for 2003, 2005, 2007, 2009, and 2011

Purpose of This Presentation NAEP reading item maps and percentile graphs can help to frame the question of where Alaska is now in terms of student achievement and what would need to be done to raise student achievement. Alaska is revising reading, writing, and math standards against the background of a national push to raise the rigor of standards.

Alaska’s Revised Standards Many states have adopted the Common Core standards for English language arts and mathematics. Alaska has developed proposed standards to equal the Common Core standards in rigor: they will be posted at The next slide shows the rigor intended to be reached by the new standards

CC Standards Designed to Be Internationally Competitive The standards have been informed by the best available evidence and the highest state standards across the country and globe... These standards are designed to ensure that students graduating from high school are prepared to go to college or enter the workforce and that parents, teachers, and students have a clear understanding of what is expected of them. The standards are benchmarked to international standards to guarantee that our students are competitive in the emerging global marketplace.

NAEP Resources Set Stage for Discussions about Higher Rigor NAEP percentile box graphs show current academic achievement gaps by ethnic groups NAEP item maps show what students are achieving now

A Disclaimer! The issues of increasing academic performance and closing gaps are complicated This presentation is designed to show current student achievement in Alaska and serve as a springboard for discussion in considering the challenges of increasing academic rigor and closing achievement gaps among ethnic groups

How Percentiles Represent Student Achievmeent A percentile is a value on a scale of one hundred that indicates the percent of a distribution that is below it. For example, if a student scores at the 84 th percentile on a test, he scored higher than 84% of the students who took the test

Understanding Box Plots Box plots allow data to be explored by showing the median value of a set of data and the 25 th and 75 th percentiles The middle line in the box shows the median value, the reading score at which half the students performed below and half the students performed above Students who scored at the 75 th percentile performed better than 75% of students who took the NAEP; students who scored at the 25 th percentile performed better than 25% of students who took the NAEP The range of the distribution is shown on the box plot by showing a minimum and maximum value (the 10 th and 90 th percentiles for the NAEP box plots)

Alaska Gr. 8 Reading Percentile Box Plots for 2011 NAEP Note where the median (the dot in the middle of the box) is for each group in relation to the other groups Note where the other data points are in relation to the other groups What are your observations? What questions or issues do your observations raise?

NAEP Item Maps Illustrate What a Reading Score Represents The item maps have descriptions of questions students who achieved certain scores on the NAEP are likely to have correctly answered Item maps combined with percentile box charts help to illustrate the knowledge and skills demonstrated by students performing at different scale scores

For instance, the item or question description at 268 on the scale represents the types of items grade 8 students who scored a 268 on NAEP answered correctly; item descriptions in blue represent items that are available to the public

Looking at the actual question for score 268 helps show what students can do in a concrete way. The score of 268 is just below the median score of 276 for White students. White students scoring at the 50 th percentile would probably have answered this question correctly. The question is shown on the next slide.

268 Recognize specific information in expository event—Locate/recall Question: According to “Home on the Range*,” some scientists think that moving African animals to the Great Plains would help improve the area’s A. resistance to alien species B. economy C. biodiversity D. research facilities Nationally, 68% of students chose C (correct answer). In Alaska, 57% chose C, 26% chose B, 13% chose A. *The two reading passages referred to in this presentation are nonfiction. The proposed English/Language Arts standards emphasize the importance of reading nonfiction.

Each NAEP Question Has a Cognitive Target There are three targets--locate/recall, integrate/interpret, and critique/valuate. Locate/recall description: Identify textually explicit information and make simple inferences within and across texts, such as: Definitions Facts Supporting details

What Skills Are Required to Answer the Question What is the text complexity of the passage for eighth grade students? What skills do students demonstrate when they answer the question correctly? Why would students choose the wrong answer? Consider the percentage who chose the wrong answers. What might lead students to choose those answers?

Consider Another Locate/Recall Question The question at 239 is close to the median score for 237 for American Indian/Alaska Native students. The question is on the next slide.

239 Recognize causal relationship between facts in article—locate/recall Question According to “Alien Invasion,” Australian farmers imported cane toads into Australia in order to A help save their native beetle populations B eat the sugarcane that was taking over their other crops Cstop an alien species of beetle imported from Hawaii Deat beetles that were destroying sugarcane crops Nationally, 84% chose D (correct answer). In Alaska 87% of students chose D and 9% chose C.

How Do the Two Locate/Recall Questions Differ What makes one question easier than the other? What skills are needed for students to answer the more difficult question?

What Does Item at Upper End of Item Map Show The question at 340 is located above the 90 th percentile for any ethnic group. The question is shown on the next slide.

340 Compare two texts of different genres to provide similarity and difference— integrate/interpret Question Describe a similarity and a difference between the way the two articles* approach the subject of invasive species. Support your answer with references to both of the articles. Note: the score of 340 is for students who scored at the extensive level according to the scoring guide, which is presented in the following slides *”Alien Invasion” and “Home on the Range”

Integrate/Interpret Make complex inferences within and across texts to describe problem and solution, cause and effect: Compare or connect ideas, problems, or situations. Determine unstated assumptions in an argument. Describe how an author uses literary devices and text features.

Item: Compare two texts of different genres to provide similarity and difference Consider text complexity of reading passages. How difficult do you think this item is? How do you think grade 8 students should perform on this item?

Scoring Guide for Extensive Responses at this level describe a similarity and a difference between the way the two articles approach the subject of invasive species and support the answer using a reference to a harm or a benefit discussed in the articles. Both articles talk about invasive species being a problem with the environment. The Home on the Range article differs in the fact that it supports putting species in a different environment to make a more diverse animal population and prevent extinction. Or, responses describe a similarity and a difference by referring to author's craft or the structure of both articles. The similarity is they both give some examples and give you at least one side of the story. The difference is that Alien Invasion only gives one side of the story but leaves out the other side. Home on the Range gives you both sides of the story and supports both sides. So Home on the Range is neutral, but Alien Invasion is not neutral.

Scoring Guide for Essential a) Responses at this level describe a similarity or a difference between the way the two articles approach the subject of invasive species and support the answer using reference to a harm or a benefit discussed in the articles. The difference is in Alien Invasion it talks about getting rid of species and Home on the Range talks about saving African animals. The articles are similar because they both say why alien species are bad. The first article tells about the cane toad in Australia and the damage it caused. OR b) Responses describe a similarity or a difference by referring to author's craft or the structure of both articles. Alien Invasion uses multiple examples of invasive species to make a point. In Home on the Range the author presents both sides of the issue.

Scoring Guide for Partial a) Responses at this level describe a similarity and/or a difference between the way the two articles approach the subject of invasive species but do not refer to a harm or a benefit. The articles are the same because they both say alien species can be bad. Both stories talk about animals moving from one place to another. They both talk about cane toads. In Alien Invasion they talk about toads and zebra mussels. In Home on the Range they talk about camels and lions. OR b) Responses describe a similarity or a difference by referring to author's craft or the structure of one article. The second article presents both sides of the alien species debate. It describes pros and cons of moving animals from Africa.

Scoring Guide for Unsatisfactory Responses at this level provide irrelevant details or personal opinions or draw incorrect conclusions or may simply repeat the question. Snakehead fish kill native fish and many species of frogs. I think the articles are trying to get people's attention. Both of the articles are against invasive species. They are both about invasive species.

What Skills Do Students Need to Demonstrative Extensive Comprehension? Example of Extensive Comprehension

Another Example of Extensive Comprehension

An Example of Essential Comprehension

What Does Partial Comprehension Reveal? Example of Partial Comprehension

What Is Going On with Unsatisfactory Comprehension? Example of Unsatisfactory Comprehension

What Does Data Suggest about Student Skills? In Alaska, 18% got extensive, 19% got essential, 34% got partial, and 22% got unsatisfactory. National performance data

Questions to Consider What does data show about the current achievement gaps? What are the challenges for increasing academic rigor and decreasing student achievement gaps?

Additional Resources The Alaska-created “NAEP Assessment Toolbox for Teachers” show how NAEP questions can be used instructionally NAEP questions for a variety of subjects can be found at