Presentation is loading. Please wait.

Presentation is loading. Please wait.

PARCC Core Leadership Group Item Review Meeting January 2013 1 Note: all items included in this presentation are for illustrative training purposes only.

Similar presentations


Presentation on theme: "PARCC Core Leadership Group Item Review Meeting January 2013 1 Note: all items included in this presentation are for illustrative training purposes only."— Presentation transcript:

1 PARCC Core Leadership Group Item Review Meeting January Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

2 1.Charge as Committee Member 2.Purpose of PARCC Summative Assessments 3.PARCC Summative Assessments (PBA, EOY) 4.Item Review Process 5.Item Review Criteria 6.Apply Criteria 1-9 to Item Review Overview of Training 2

3 Your role is to provide expert CONTENT review of items and tasks. You will use information provided in this item review training and will apply item review criteria to review the items. You should focus exclusively on PARCC’s item review criteria during your review. Item Review Committee Charge 3

4 Please also note the following: Passage review committees have already approved the passages according to PARCC content and bias/sensitivity guidelines. Bias/sensitivity item review committees will apply bias/sensitivity guidelines to all items at a different time. Other concerns identified will be placed in “the parking lot” for consideration by PARCC leadership. Item Review Committee 4

5 Determine whether students are college- and career-ready or on track Assess the full range of the Common Core State Standards (CCSS) for reading, writing, and language Measure the full range of student performance, including the performance of high- and low- performing students Provide data for accountability, including measures of growth Incorporate innovative approaches throughout the system Purpose of PARCC Summative Assessments 5

6 Composed of Three Tasks Literary Analysis Task (LAT) Research Simulation Task (RST) Narrative Task (NT) Eligible Item Types for Performance-Based Assessment (PBA) Evidence-Based Selected Response (EBSR) Technology-Enhanced Constructed Response (TECR) Prose-Constructed Response (PCR) Performance-Based Assessment 6

7 7 A. Literary Analysis Tasks—The Literature Task plays an important role in honing students’ ability to read complex text closely, a skill that research reveals as the most significant factor differentiating college-ready from non-college-ready readers. This task will ask students to carefully consider literature worthy of close study and compose an analytic essay. B. Research Simulation Task—The Research Simulation Task is an assessment component worthy of student preparation because it asks students to exercise the career- and college- readiness skills of observation, deduction, and proper use and evaluation of evidence across text types. In this task, students will analyze an informational topic presented through several articles or multimedia stimuli, the first text being an anchor text that introduces the topic. Students will engage with the texts by answering a series of questions and synthesizing information from multiple sources in order to write two analytic essays. C. Narrative Writing Task—The Narrative Task broadens the way in which students may use this type of writing. Narrative writing can be used to convey experiences or events, real or imaginary. In this task, students may be asked to write a story, detail a scientific process, write a historical account of important figures, or to describe an account of events, scenes or objects, for example. Pg 13 Item Guidelines

8 Focused on supporting Reading Comprehension Claims Item development must be informed by CCSS and Evidence Statements Eligible Item Types for End-of-Year (EOY) Evidence-Based Selected Response (EBSR) Technology-Enhanced Constructed Response (TECR) End-of-Year Assessment 8

9 EBSR items:  Are designed to measure Reading Standard 1 and at least one other Reading Standard.  May have one part. In grade 3, a one part EBSR is allowable because Reading Standard 1 evidence 1 is distinctly different from Reading Standard 1 in grades In grades 4-11, a one part EBSR is allowable when there are multiple correct responses that elicit multiple evidences to support a generalization, conclusion or inference. Evidence-Based Selected Response (EBSR) Items: 9 Pgs Item Guidelines

10  May have multiple parts.  The Item Guidelines document describes how PART A and PART B may function in a multiple part EBSR.  Among things to consider: Each part must consist of a selected-response question with a minimum of four choices. In the first part, students select a correct answer among a minimum of four choices. (continued on next slide) Evidence-Based Selected Response (EBSR) Items: 10 Pgs Item Guidelines

11  In additional parts, one format requires students to select among a minimum of four choices to demonstrate the ability to locate and/or connect details/evidence from the text that explains, justifies, or applies the answer chosen for the first part of the item.  There is no requirement for one-to-one alignment between Part A options and Part B options.  EBSR items can meet evidence statements that specify “explain,” “provide an analysis,” etc., by selecting options that comprise explanation or analysis. Evidence-Based Selected Response (EBSR) Items: 11

12 EBSR items may possess the following characteristics:  For items with one correct response, four answer choices are requisite. For those items with two correct responses, six answer choices are requisite. For those items with three correct response (allowed only in grades 6-11), seven answer choices are requisite. Evidence-Based Selected Response (EBSR) Items: Variable Elements 12 Pgs Item Guidelines

13  For items with one correct response in Part A and one correct response in Part B, there is no partial credit. One part items do not offer partial credit.  For those items with one or more correct responses in Part A and more than one correct response in Part B, partial credit should be available.  To receive partial credit, students must answer Part A correctly AND select at least one correct response in Part B. This will earn the student 1 point. For these items, to receive full credit, students must answer both Part A and Part B correctly.  When an item allows for more than one correct choice, each correct choice must be equally defensible. Evidence-Based Selected Response (EBSR) Items: 13 Pgs Item Guidelines

14 EBSR items can be written with an inference to be drawn in Part A and a requirement for students to find another example of how that inference applies in a different part of the text for Part B. Example 1: Part A – What character trait does Snow White reveal when Snow White does X? Part B—Which paragraph from the passage best shows additional evidence of this character trait? Example 2: Part A—What theme is revealed in the passage? Part B—Which paragraph from the passage best shows this same theme? Example 3: Part A—What is the point of view/perspective in this passage? Part B—Which paragraph from the passage best shows this same point of view/perspective (or the opposite point of view/perspective)? See the Item Guidelines document pages for other types of EBSR items. Evidence-Based Selected Response (EBSR) Items: 14 Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

15  The primary purpose of a distracter is to provide evidence that a student is not able to meet the standard(s) assessed due to student misconceptions.  Distracters must be plausible responses to item stems.  The written style of all distracters in an item should be similar to that of the correct response(s), but need not be “perfectly parallel” in length, grammatical function, or in use of punctuation.  The content of the distracters, rather than the parallelism of style, is the primary focus for distracter choices.  Answer responses are not ordered alphabetically by first word or from short to long, etc. They may be ordered in any sequence as is appropriate to the content measured by the specific item. Additional Considerations for Distracters 15

16  If answer responses are quotations or paraphrased textual evidence, place the answers in the order they appear in the passage.  Particular care must be taken for Part B in EBSR items (where students are asked to select evidence from the text) such that achieving parallelism in distracters does not overly influence distracter wording.  In Part B, when writing the distracters for evidences, all of the answer choices must be the same type of citation of evidence (e.g. all quotes or all paraphrases).  All answer choices for Part B (distracters) must be accurate/relevant/from the passage (whether exact citations or paraphrases). All distracters must originate accurately from the text. In Part A, distracters may be written as plausible misreadings of the text. Additional Considerations for Distracters 16

17 Part A: What does the word “regal” mean as it is used in the passage? A.generous B.threatening C.kingly* D.uninterested Part B: Which of the phrases from the passage best helps the reader understand the meaning of “regal?” A.“wagging their tails as they awoke” B.“the wolves, who were shy” C.“their sounds and movements expressed goodwill” D.“with his head high and his chest out”* 17 Sample Evidence-Based Selected Response (EBSR) Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

18 TECR items must:  Allow for machine scoring  Award the student two (2) points for full credit  Be delivered and responded to using technology, allowing for a variety of technology-enhanced student responses, including but not limited to the following: drag and drop highlighting the text annotating text other negotiated methods Technology-Enhanced Constructed Response (TECR) Items: 18 Pgs Item Guidelines

19 TECR items may possess the following characteristics:  When a TECR uses an EBSR structure (e.g. with Part A [measuring one or more of standards 2-9] and Part B [measuring standard 1]), use the same rules as applied for EBSR.  Use the same guidelines for distractors for EBSR items for TECR items where applicable.  For TECR items, partial credit may be offered when an item allows for partial comprehension of the texts to be demonstrated. Technology-Enhanced Constructed Response (TECR): 19 Pgs Item Guidelines

20 Claims Earhart and Noonan lived as castaways on Nikumaroro Island. Earhart and Noonan’s plane crashed into the Pacific Ocean. People don’t really know where Earhart and Noonan died. Sample Technology-Enhanced Constructed Response (TECR) Item 20 Part A : Below are three claims that one could make based on the article “Earhart’s Final Resting Place Believed Found.” Highlight the claim that is supported by the most relevant and sufficient evidence within “Earhart’s Final Resting Place Believed Found.” Part B : Click on two facts within the article that best provide evidence to support the claim selected in Part A. Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

21 PCR items must: Visibly align questions/tasks with specific Standards; that is, the actual language of the Standards should be used in the prompts/questions Elicit evidence(s) supporting the Sub Claim for Written Expression and the Sub Claim for Conventions and Knowledge of Language Prose Constructed-Response (PCR): Required Elements 21 Pgs Item Guidelines

22 PCR items must: Establish a clear purpose for writing, modeling the language found in the Writing Standards Specify the audience to be addressed State clearly the topic, issue, or idea to be addressed Reference the source text (or texts) serving as the stimulus (or stimuli) for a student response Specify the desired form or genre of the student response Note: Standardized wording for PCRs is under discussion. Prose Constructed-Response (PCR): Required Elements 22 Pgs Item Guidelines

23 Elicit evidence(s) aligned with at least one Reading Standard (even when not scored for a sub claim associated with the Major Claim for Reading Complex Text) Allow students to earn partial credit Prose Constructed-Response (PCR): Required Elements 23 Pgs Item Guidelines

24  In addition, prose constructed-response items must provide all students the opportunity to demonstrate a full range of sophistication and nuance in their responses. Prose constructed-response items must be designed to elicit meaningful responses on aspects of a text that may be discussed tangentially or in great detail and elaboration, thereby enabling measurement of the full range of student performance. Prose Constructed-Response (PCR): Required Elements 24 Pgs Item Guidelines

25 Prose Constructed-Response (PCR): Narrative Description 25 Narrative writing takes two distinct forms in the PARCC assessment system: 1)Narrative Story 2)Narrative Description*

26 Prose Constructed-Response (PCR): Narrative Story 26 Narrative Story – is about imagined situations and characters. It uses time as its deep structure.

27 Prose Constructed-Response (PCR): Narrative Story- Scenarios 27 Scenarios should be associated with the text and should culminate by stating the objective of the final prose constructed response for the task. Scenarios for Narrative Writing –Narrative Story Task Model : Today you will read [fill in the text type/title]. As you read, pay close attention to [fill in general focus of PCRs] as you answer the questions to prepare to write a narrative story. Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

28 Use what you have learned from reading “Daedulus and Icarus” by Ovid and “To a Friend Whose Work Has Come to Triumph” by Anne Sexton to write an essay that analyzes how Icarus’s experience of flying is portrayed differently in the two texts. As a starting point, you may want to consider what is emphasized, absent, or different in the two texts, but feel free to develop your own focus for analysis. Develop your essay by providing textual evidence from both texts. Be sure to follow the conventions of standard English. Sample Prose-Constructed Response (PCR) Item 28 Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

29  Several styles for presenting vocabulary words/phrases are viable. In considering which means is best for presenting the vocabulary words/phrases, item writers should use the means that most efficiently directs students to the word/phrase in the text, while allowing students to see the full relevant context for determining the meaning of the word/phrase.  The part of a vocabulary item that asks for word or phrase meaning should not use qualifiers (e.g., best, most likely: Which of the phrases from the excerpt best helps the reader understand the meaning of “XXX”?  The part of a vocabulary item that asks for support may use qualifiers if needed: Which sentence from the excerpt best supports the response in Part A?  Distractors should always be syntactically plausible. This is especially important in vocabulary items. Additional Considerations for Vocabulary Items 29

30  When writing vocabulary items use the following formats: Part A - “What is the meaning of the word XXX as it is used in [paragraph 13, or line, or text]?—Part A wording Part B - “Which of the phrases from the excerpt helps the reader understand the meaning of XXX? Unless referencing in this way creates a problem for the item. In this case, the item may require the use of a text box approach where the options for Part B come only from the excerpted text.  Use of Technology Enhancement:  To measure vocabulary using a drag/drop technology use the following format:  “Drag the sentences from the passage into your notes that help create the meaning of the word “XXX” as it is used in the passage.”  “Drag the words/phrases from the passage into your notes that help create the tone of the passage.” Note: Give a selection of sentences or words/phrases from which to choose where not all are correct. Additional Considerations for Vocabulary Items 30

31 Item Review Process 31 Step 1 - Each reviewer on your team will read the passage/text and the items independently. Step 2 - For each item, check to make sure the item meets the evidence statements/standards noted in the metadata for the item. If the item is aligned, continue to review the item. If the item is not aligned, but can be edited to align with the evidence statements/standards, note in comments how to align the item. If the item cannot be aligned, reject the item and move on to review the next item. Step 3 - Review the item in terms of other criteria on the criteria evaluation sheet. Make comments to reflect needs for revision/strengths noted.

32 Item Review Process 32 (continued) Step 4 - The facilitator will go item by item to determine which items are to be accepted, accepted with edits, or rejected. If all members of a group have marked an item “accepted” or “rejected,” no discussion occurs. Participants discuss those items where there is not 100% agreement. Step 5- Allow a maximum of 10 minutes per item for consensus building to determine whether to accept, accept with edits, or to reject an item. If consensus cannot be achieved, move on. The team will return to these items on the last day of the review.

33 1.Does the item allow for the student to demonstrate the intended evidence statement(s) and to demonstrate the standard(s) to be measured? 2.Is the wording of the item clear, concise, and appropriate for the intended grade level? 3.Does the item provide sufficient information and direction for the student to respond completely? 4.Is the item free from internal clueing and miscues? 5.Do the graphics and stimuli included as part of the item accurately and appropriately represent the applicable content knowledge? Item Review Criteria 33

34 Item Review Criteria 34 6.Are any graphics included as part of the item clear and appropriate for the intended grade level? 7.If the item has a technology-based stimulus or requires a technology-based response, is the technology design effective and grade appropriate? 8.Is the scoring guide/rubric clear, correct, and aligned with the expectations for performance that are expressed in the item or task? 9.If the item is part of a PBA task, does it contribute to the focus and coherence of the task model? (continued)

35 Criterion 1. Alignment to the CCSS and Evidence Statements 35 Does the item allow for the student to demonstrate the intended evidence statement(s) and to demonstrate the CCSS to be measured?

36 An item should:  be aligned to more than one CCSS and corresponding evidence statements. Focus is no longer on a one-to-one relationship between an item and tested skill All reading comprehension items will align to a focus standard as well as RL 1 or RI 1  elicit the intended evidence(s) to demonstrate ability with the skills expressed in the standard(s). Items should focus on important aspects of the text and be worth answering Items are intended to provide evidence to support the claims Criterion 1. Alignment to the CCSS and Evidence Statements 36

37 Vocabulary items should:  target Tier 2 academic vocabulary Words with wide use across academic subjects Words that are important for students to know  have distracters in the same part of speech as the assessed word  elicit evidence from at least one language standard and from reading standard RI1 or RL1 Criterion 1. Alignment to the CCSS and Evidence Statements 37 Pgs Item Guidelines

38 Criterion 1. Alignment to the CCSS and Evidence Statements 38 Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

39 Criterion 2. Language Clarity and Appropriateness 39 Is the wording of the item clear, concise, and appropriate for the intended grade level?

40 Items should:  convey a clearly defined task or problem in concise and direct language  use the language of the evidence statements and standards when appropriate  focus on what is important to learn rather than on trivial content  use a variety of approaches rather than a canned approach Criterion 2: Language Clarity and Appropriateness 40

41 Criterion 2: Language Clarity and Appropriateness Example from Grade 10: Poor Clarity What was the problem that Galileo’s experiment did not solve? A. the influence of air pressure on the tubes’ water level Example from Grade 10: Improved Clarity According to the excerpt, what problem did Galileo’s experiment fail to solve? A. the influence of air pressure on the glass tube’s water level 41 Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

42 Criterion 3. Clarity of the Student Directions 42 Does the item provide sufficient information and direction for the student to respond completely?

43 Items should:  pose the central idea in the stem and not in the answer choices  provide directions that are specific and direct  require no background knowledge Criterion 3: Clarity of the Student Directions 43

44 Criterion 3: Clarity of the Student Directions 44 Example of Poor Clarity in Student Directions Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

45 Criterion 3: Clarity of the Student Directions 45 Example of Improved Clarity in Student Directions Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

46 Criterion 4: Avoiding Clueing and Miscues 46 Is the item free from internal clueing and miscues?

47  Options should NOT echo a stem word.  Distractors should be plausible so that students will consider them carefully.  Items should not be answerable by reading other items in the set. Criterion 4: Avoiding Clueing and Miscues 47

48 Example of Miscue: How is the life cycle of a frog similar to the life cycle of the butterfly? A.Both turn into a pupa. B.Both breathe with gills. C.Both begin life as an egg.* D.Both break out as a chrysalis. Criterion 4: Avoiding Clueing and Miscues 48 Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

49 Criterion 5: Accuracy of Content Represented by Graphics 49 Do the graphics and stimuli included as part of the item accurately and appropriately represent the applicable content knowledge?

50 Items should: not include graphics simply for the sake of including graphics. use graphics that allow students to understand the problem or task or demonstrate ability to solve a problem or task. elicit the intended evidence(s) to demonstrate ability with the skills expressed in the standard(s). Criterion 5: Accuracy of Content Represented by Graphics 50

51 Criterion 6: Clarity and Appropriateness of Graphics 51 Are any graphics included as part of the item clear and appropriate for the intended grade level?

52 Items should: include graphics that are not too complex in structure or language for the intended grade level utilize graphics that allow students to understand the problem or task or demonstrate ability to solve a problem or task have figures, graphs, charts, and diagrams precisely labeled Criterion 6: Clarity and Appropriateness of Graphics 52

53 Criterion 7: Effectiveness and Grade Appropriateness of Technology 53 If the item has a technology-based stimulus or requires a technology-based response, is the technology design effective and grade appropriate?

54 Items should: include the number of options and correct answers that are appropriate for the grade level. use a format that is sufficiently simple and interesting for students. include directions that are clear and detailed. Criterion 7: Effectiveness and Grade Appropriateness of Technology 54

55 Grade 3: Example of Effective and Grade-Appropriate Use of Technology Drag the words from the word box into the correct locations on the graphic to show the life cycle of a butterfly as described in “How Animals Live.” Words: Criterion 7: Effectiveness and Grade Appropriateness of Technology 55 Pupa Adult Egg Larva Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

56 Criterion 8: Clarity and Alignment of Performance Expectations to Scoring Guide 56 Is the scoring guide/rubric clear, correct, and aligned with the expectations for performance that are expressed in the item or task?

57 Scoring Guidelines for EBSRs and TECRs:  Partial credit is allowable and desirable. See the specifics for scoring in the Item Guidelines document. Criterion 8: Clarity and Alignment of Performance Expectations to Scoring Guide 57 Pgs Item Guidelines

58 Criterion 9: Alignment of PBA Task to Model 58 If the item is part of a PBA task, does it contribute to the focus and coherence of the task model?

59 Task TypeLiterary Analysis Grade5 Number and type of Texts 1 Extended Literature Text 1 Additional Literature Text Both texts must be stories from the same genre (e.g., mysteries and adventure stories) with similar themes and topics. Number and type of Prose Constructed Response Items 1 Analytic PCR  Measures reading literature sub-claim using standards 1 and 9  Measures all writing claims Number and type of EBSR and/or TECR reading items 6 total items = 12 points 2 of 6 items(4 points) to measure the reading sub-claim for vocabulary (one per text) 4 of 6 items (8 points) measuring standards RL 2,3 and 5  Items that do not measure reading sub-claim for vocabulary are designed to measure reading literature sub-claim Task Complexity (including text, item, and task complexity) To be determined Total # of Items for the Task Model:7 Order of Student Actions:  Students read extended literature text  Students respond to 1 item to measure the reading sub-claim for vocabulary  Students respond to 2 EBSR or TECR items  Students read 1 additional literature texts  Students respond to 1 item to measure the reading sub-claim for vocabulary  Students respond to 2 EBSR or TECR items  Students respond to 1 PCR Sample Task Generation Model ELA Task Generation Model 5A.4 Task Focus: Comparing themes and topics 59 Criterion 9: Alignment of PBA Task to Model

60 Task Model Review for PBA Tasks Task Alignment with the TGM 1.Does the provided task match the task generation model designated in the metadata provided? Task Directions 2.Are the directions for the task clear, coherent, and appropriate for the intended grade level? Criterion 9: Alignment of PBA Task to Model 60

61 Questions? Comments? 61


Download ppt "PARCC Core Leadership Group Item Review Meeting January 2013 1 Note: all items included in this presentation are for illustrative training purposes only."

Similar presentations


Ads by Google