1. NEA and AFT Item Review Boot Camp Working Session: ELA/Literacy

2. Overview of Training
Purpose and Features of PARCC Summative Assessments
PARCC Summative Assessments (PBA, EOY)
Your Charge as a Committee Member
Item Review Process by Core Leadership Group
Item Review Criteria by Core Leadership Group
Item Review Process and Criteria by State Educators

3. Purpose of PARCC Summative Assessments
Determine whether students are college- and career-ready (CCR) or on track to become CCR
Assess the full range of the Common Core State Standards (CCSS) for reading, writing, and language
Measure the full range of student performance, including the performance of high- and low-performing students
Provide data for accountability, including measures of growth
Incorporate innovative approaches throughout the system

4. Features of the PARCC Summative Assessment System
The PARCC summative assessment system is composed of two assessments:
Performance-Based Assessment
End-of-Year Assessment

5. Performance-Based Assessment
Performance-Based Assessment (PBA) consists of three tasks:
Literary Analysis Task (LAT)
Research Simulation Task (RST)
Narrative Task (NT)
Talking Points:
Literary Analysis Task (LAT)
The Literary Analysis Task plays an important role in honing students’ ability to read complex text closely, a skill that research reveals as the most significant factor differentiating college-ready from non-college-ready readers. This task will ask students to carefully consider literature worthy of close study and compose an analytic essay.
Research Simulation Task (RST)
The Research Simulation Task asks students to exercise the career- and college- readiness skills of observation, deduction, and proper use and evaluation of evidence across text types. In this task, students will analyze an informational topic presented through several articles or multimedia stimuli, the first text being an anchor text that introduces the topic.  Students will engage with the texts by answering a series of questions and synthesizing information from multiple sources in order to write an analytic essay.
Narrative Task (NT)
The Narrative Task broadens the way in which students may use this type of writing. Narrative writing can be used to convey experiences or events, real or imaginary. In this task, students may be asked to write a story,  detail a scientific process, write a historical account of important figures, or to describe an account of events, scenes or objects, for example.

6. Performance-Based Assessment
Eligible Item Types for Performance-Based Assessment (PBA):
Evidence-Based Selected Response (EBSR)
Technology-Enhanced Constructed Response (TECR)
Prose-Constructed Response (PCR)
Talking Points:
pp Item Guidelines

7. Sample Evidence-Based Selected Response (EBSR) Item
Part A: What does the word regal mean as it is used in the passage?
generous
threatening
kingly*
uninterested
Part B: Which of the phrases from the passage best helps the reader understand the meaning of regal?
“wagging their tails as they awoke”
“the wolves, who were shy”
‘their sounds and movements expressed goodwill”
“with his head high and his chest out”*

8. Sample Technology-Enhanced Constructed Response (TECR) Item
Part A : Select the claim that is supported by the most relevant and sufficient evidence within “Earhart’s Final Resting Place Believed Found.”
Part B : Select two facts within the article that best provide evidence to support the claim selected in Part A.
Claims
Earhart and Noonan lived as castaways on Nikumaroro Island.
Earhart and Noonan’s plane crashed into the Pacific Ocean.
People don’t really know where Earhart and Noonan died.

9. Sample Prose-Constructed Response (PCR) Item
Use what you have learned from reading “Daedalus and Icarus” by Ovid and “To a Friend Whose Work Has Come to Triumph” by Anne Sexton to write an essay that analyzes how Icarus’s experience of flying is portrayed differently in the two texts.
As a starting point, you may want to consider what is emphasized, absent, or different in the two texts, but feel free to develop your own focus for analysis.
Develop your essay by providing textual evidence from both texts.

10. End-of-Year Assessment
The End-of-Year Assessment will be:
Focused on supporting Reading Comprehension Claims
Machine scored
Eligible Item Types for End-of-Year Assessment (EOY):
Evidence-Based Selected Response (EBSR)
Technology-Enhanced Constructed Response (TECR)

11. Item Review Committee Charge
Your role is to provide expert CONTENT review of items and tasks.
You will use information provided in this item review training to review the items.
You should focus exclusively on PARCC’s item review process.

12. Item Review Committee Please also note the following:
Passage review committees have already approved the passages according to PARCC content guidelines.
Bias/sensitivity item review committees will apply bias/sensitivity guidelines to all items.
Concerns beyond the scope of your charge as an item reviewer will be placed in the “parking lot” for consideration by PARCC leadership.

13. Item Review Criteria: Core Leadership Group
Does the item allow for the student to demonstrate the intended evidence statement(s) and to demonstrate the standard(s) to be measured?
Is the wording of the item clear, concise, and appropriate for the intended grade level?
Does the item provide sufficient information and direction for the student to respond completely?
Is the item free from internal clueing and miscues?
Do the graphics and stimuli included as part of the item accurately and appropriately represent the applicable content knowledge?
(continued)

14. Item Review Criteria: Core Leadership Group
Are any graphics included as part of the item clear and appropriate for the intended grade level?
If the item has a technology-based stimulus or requires a technology-based response, is the technology design effective and grade appropriate?
Are the scoring notes clear, correct, and aligned with the expectations for performance that are expressed in the item or task?
If the item is part of a PBA task, does it contribute to the focus and coherence of the task model?
“If you see something troubling in an item, parking lot.”

15. Previous Steps in the Item Development Process
A group of experts in the area of English Language Arts have previously viewed these items, including:
English Language Arts teachers
ELA assessment experts
English higher education faculty
State Department of Education personnel
School and district administrators
The items have been previously reviewed and edited for content area accuracy, alignment to the standards/evidence statements, and grade level appropriateness. This ensures that the items you will be viewing have been vetted.

16. Item Review Process and Criteria for State Educators
Step 1: Each reviewer on your team will read the passage/text and the items independently.
Step 2: For each item, check to make sure the item meets the evidence statements/standards. If the item can be edited to align with the evidence statements/standards, note in comments how to align the item. If not, reject the item and move on.
Step 3: Determine if there are any fatal flaws in the item (i.e. anything that makes the item unusable as written).
Step 4: Advise to accept, accept with edits, or reject the item.

17. Fatal Flaws
Use of language in the item in a way that could have unintended consequences
Language that is convoluted and obscure
Note that the flaw should be such that when one raises the issue, the whole review group immediately says, “Oh yes—this can’t go forward as is.” 
If some see “the flaw” and others don’t see it as a flaw, then it shouldn’t be seen as a fatal flaw.
Will a student “giggle” over this innocent word?
Are there any glaring issues with this item that would cause you concern should it land on the front page of the New York Times?

18. Questions?