1. Washington Educational Research Association Conference Washington Educational Research Association Conference Kelly Neely Reading Assessment Specialist kelly.neely@k12.wa.us Kelly Neely Reading Assessment Specialist kelly.neely@k12.wa.us

2. WERA Agenda WASL Development Process WASL Test Map Secondary Reading Targets Overview of 2006 Reading WASL Secondary Performance 2006 Item Analysis of Operational Assessment Achieving Maximum Score Points on Constructed Response Items-Instructional Tips 2006 Released Item Documents Q and A

3. Reading Passage Selection and Copyrights Research Item Pilot Testing Item Reviews for Content by OSPI Item Writing Training and Item Writing Pilot Rangefinding Pilot Scoring Training and Pilot Scoring Operational Rangefinding Operational Testing Operational Scoring Teacher Scoring Data Review Standard Setting (First Year with Periodic Reviews) Develop Item and Test Specifications Passage Bias/Fairness Reviews Item Reviews for Content & Bias/Fairness by Committee

4. Test Map TEST CHARACTERISTICS Test CharacteristicsGrade 3Grade 4Grade 5Grade 6Grade 7Grade 8High School # of total items (MC, SA, ER) on test 28293132343537 # of total points possible on test 40424446485052 Number of operational passages on each test 6666666 Number of strands assessed per test 4446666 Minimum # of points per strand per test 6666666 # of multiple choice (MC) items on test 20 22 24 26 # of short answers (SA) on test 6778899 #of extended responses (ER) on test 2222222 Percent of multiple choice in total score 50%48%50%48%50%48%50% Percent of constructed response in total score (short answer and extended response)* 50%52%50%52%50%52%50% Total Number of Embedded Items 7777777

5. Secondary Reading Targets and Strands GRADES 6-HIGH SCHOOL READING TARGETS ASSESSED ON THE READING WASL and Their Corresponding Grade Level Expectations Literary TargetsInformational Targets LC01 Demonstrate understanding of theme or message and supporting details (Reading GLE 2.1.3) IC11 Demonstrate understanding of major ideas and supporting details (Reading GLE 2.1.3) LC02 Summarize with evidence from the reading (Reading GLE 2.1.7) IC12 Summarize with evidence from the reading (Reading GLE 2.1.7) LC03 Make inferences or predictions based on the reading (Reading GLE 2.1.5) IC13 Make inferences or predictions based on the reading (Reading GLE 2.1.5) LC04 Interpret vocabulary critical to the meaning of the text (Reading GLE 1.3.2) IC14 Interpret vocabulary critical to the meaning of the text (Reading GLE 1.3.2) LA05 Demonstrate understanding of literary elements (genres; story elements such as plot, character, setting; stylistic devices) and graphic elements/illustrations (Reading GLEs 2.2.2, 2.2.3, 2.3.3) IA15 Demonstrate understanding of text features (titles, headings and other information divisions, table of contents, indexes, glossaries, prefaces, appendices, captions) and graphic features (Reading GLE 2.2.2) LA06 Compare and contrast elements between and within texts (Reading GLEs 2.3.1, 2.4.6) IA16 Compare and contrast information between and within texts (Reading GLEs 2.3.1, 2.4.6) LA07 Make connections (cause and effect) within a text (Reading GLE 2.3.1) IA17 Make connections (cause and effect) within a text (Reading GLE 2.3.1) LT08 Analyze author’s purpose and evaluate effectiveness for different audiences (includes fact/opinion, author’s point of view, tone, and use of persuasive devices) (Reading GLEs 2.4.2, 2.4.3, 2.4.4) IT18 Analyze author’s purpose (including distinguishing between fact and opinion) and evaluate effectiveness for different audiences (Reading GLEs 2.4.2, 2.4.3, 2.4.4) LT09 Evaluate reasoning and ideas/themes related to the text (Reading GLEs 2.4.1, 2.4.5) IT19 Evaluate reasoning and ideas/themes related to the text (Reading GLEs 2.4.1, 2.4.5) LT10 Extend information beyond text (make generalizations beyond the text to a broader idea or concept, draw conclusions, or apply information to other texts or situations) (Reading GLEs 2.4.1, 2.4.5) IT20 Extend information beyond text (make generalizations beyond the text to a broader idea or concept, draw conclusions, or apply information to other texts or situations) (Reading GLEs 2.4.1, 2.4.5)

6. Types of Passages Literary PassagesInformational Passages Stories Poems Novel Excerpts Biographies/Autobio graphies Literary essays Magazine articles Newspaper articles Excerpts/articles as in a social studies text Excerpts/articles as in a science text Historical documents Nonfiction narratives (In all selections we attempt to incorporate text features such as timelines, text boxes, charts, graphs, maps, diagrams, pictures, and captions.)

7. 13.3 30.0 28.619.3 Did not meet standardExceeded standardMet standard 1998 1999 2000 2001 2002 2003 2004 2005 N=5802 N=17,410 N=23,931 N=30,886 Percentages do not total 100 as chart does not include untested students Grade 7 Reading

9. Data Analysis for 2006 Operational Assessment Secondary Students Blank responses Summaries for sections and processes Author’s Purpose Compare and contrast with similarities and differences Use of figurative language Text Features (purpose of identified text feature) Conclusion with supporting details Evaluating Reasoning

10. Constructed response items will follow these general guidelines: Short-answer (2 pts. with 9 lines) and extended-response (4pts. with 18 lines) Item-specific scoring guide is developed for each short-answer and extended-response item Extended-response items generally require longer, more detailed responses providing more evidence, information, or examples When an item has two parts, the parts sometimes appear separately (scaffolded), with lines following each part. This encourages students to notice and answer both parts of the item. Note: With paired passages, items follow each passage. The items requiring comparison between the passages appear last, after both passages. There should always be more than one item that compares the two passages. Constructed-response items should be asked about individual passages in addition to the passages as a pair.

11. What Do Students Who Meet Standard on the Secondary WASL Do When Responding to Constructed Response Items? They carefully read the item and understand exactly what it is asking of them. They understand assessment language such as summary, compare/contrast, explain, reasonable conclusion, inference, assumption, support, details, examples, etc. They carefully read to see how much text evidence is required. For example, on a 4 pt extended response, if they are asked to “Provide four examples from the selection in your answer,” they make sure they provide at least four examples to support their answer. In fact, students who achieve all possible points on constructed responses often go beyond the minimum number of text-based details asked for.

12. They don’t use words such as “things” “it” “stuff.” They use characters’ names to differentiate between characters in their responses. They provide text-based evidence to back up their inferences and assumptions. For example, they answer “ I infer (or I think/I assume) that ________ because in the selection _________.” They understand that personal opinion does not receive score points. However, they often write their own personal views and opinions while still answering what the item asks them to do. Their personal views and opinions are given as side commentary, and supports their text-based evidence. What Do Students Who Meet Standard on the Secondary WASL Do When Responding to Constructed Response Items?

13. Multiple choice items will follow these general guidelines: Each multiple-choice item contains four answer options, only one of which is correct. The choices are approximately the same length, have the same format, and are syntactically and semantically parallel; students should not be able to rule out a wrong answer or identify a correct response simply by virtue of its looking or sounding different. Distracters adopt the language and sense of the material in the selection so that students must think their way to the correct answer rather than simply identify incorrect responses by virtue of a distracter's obviously inappropriate nature. Distracters should always be plausible (but of course incorrect) in the context of the selection. To the greatest extent possible, no item or response choice "clues" the answer to any other item.

14. Scoring! Anchor, practice, and qualifying sets are used to train for scoring MC=1 point, SA=2 points, ER=4 points Cut Score is determined at Standard Setting Rubrics –Bullets vs open rubrics –Conclusions –Summarization

15. Reading WASL Released Item Documents 2006 Released Item Documents for grades 3, 5, 6, and 8 Items and passages were previously used on the 2006 operational assessments Items were scored by teachers in the summer of 2006 Contains state level data for each item Current booklets contain only reading selections and constructed response items at this time. Multiple choice items will be posted in the released documents on our website in January

16. Involving Students in Assessment  Students complete assessment  Train students on sample responses and have students score responses using scoring guides/rubrics  Classroom discussion on student responses and results/data analysis  Mini-lessons to follow up on areas of concern  Train students to use question stems

17. What can I do for YOU? Website: www.k12.wa.uswww.k12.wa.us Assessment Reading Committee Application Visit reading assessment page after December 6, 2006 Scoring Online TODAY! Teacher scoring deadline is December 22, 2006 Support Kelly.neely@k12.wa.usKelly.neely@k12.wa.us (360)725-6441

18. Kelly Neely Secondary Reading Assessment Specialist kelly.neely@k12.wa.us (360)725-6441