1. Washington Educational Research Association Conference Washington Educational Research Association Conference Beth Dorr Reading Assessment Specialist beth.dorr@k12.wa.us Beth Dorr Reading Assessment Specialist beth.dorr@k12.wa.us

2. WERA Agenda WASL Development, Test Matrix, and Targets Overview of 2006 Reading Performance Grades 3, 4, and 5 General Guidelines for responses and scoring Introduction to the 2006 Reading Released Items Q & A

3. Reading Passage Selection and Copyrights Research Item Pilot Testing Item Reviews for Content by OSPI Item Writing Training and Item Writing Pilot Rangefinding Pilot Scoring Training and Pilot Scoring Operational Rangefinding Operational Testing Operational Scoring Teacher Scoring Data Review Standard Setting (First Year with Periodic Reviews) Develop Item and Test Specifications Passage Bias/Sensitivity Reviews Item Reviews for Content & Bias/Sensitivity by Committee

4. We Want You to Participate! To apply for Reading WASL Committees visit: www.k12.wa.us/assessment/WASL/ReadingAssessment.aspx

5. Test Characteristics Matrix What makes up each Operational Reading WASL? (Refer to the Test Characteristics Matrix Handout)

6. Test Map TEST CHARACTERISTICS Test CharacteristicsGrade 3Grade 4Grade 5Grade 6Grade 7Grade 8High School # of total items (MC, SA, ER) on test 28293132343537 # of total points possible on test 40424446485052 Number of operational passages on each test 6666666 Number of strands assessed per test 4446666 Minimum # of points per strand per test 6666666 # of multiple choice (MC) items on test 20 22 24 26 # of short answers (SA) on test 6778899 #of extended responses (ER) on test 2222222 Percent of multiple choice in total score 50%48%50%48%50%48%50% Percent of constructed response in total score (short answer and extended response)* 50%52%50%52%50%52%50% Total Number of Embedded Items 7777777

7. Strands and Targets The blue prints of the Reading WASL (Refer to the Strands and Targets Handout)

8. Grades 3, 4, 5 Strands and Targets GRADES 3, 4, AND 5 READING TARGETS ASSESSED ON THE READING WASL and Their Corresponding Grade Level Expectations Literary TargetsInformational Targets LC01 Demonstrate understanding of theme or message and supporting details (Reading GLE 2.1.3) IC11 Demonstrate understanding of major ideas and supporting details (Reading GLE 2.1.3) LC02 Summarize with evidence from the reading (Reading GLE 2.1.7) IC12 Summarize with evidence from the reading (Reading GLE 2.1.7) LC03 Make inferences or predictions based on the reading (Reading GLE 2.1.5) IC13 Make inferences or predictions based on the reading (Reading GLE 2.1.5) LC04 Interpret vocabulary critical to the meaning of the text (Reading GLE 1.3.2) IC14 Interpret vocabulary critical to the meaning of the text (Reading GLE 1.3.2) LC05 Order steps, sequence, and/or events from the reading (Reading GLE 2.2.1) IC15 Order steps, sequence, and/or process from the reading (Reading GLE 2.2.1) LA06 Demonstrate understanding of literary elements (genres; story elements such as plot, character, setting; stylistic devices; problem/resolution) and graphic elements/illustrations (Reading GLEs 2.2.2, 2.2.3, 2.3.3) IA16 Demonstrate understanding of text features (titles, headings, and other information divisions, table of contents, captions) and graphic features (Reading GLE 2.2.2) LA07 Compare and contrast elements of text (Reading GLEs 2.3.1, 2.4.5, 2.4.6) IA17 Compare and contrast information presented in text (Reading GLEs 2.3.1, 2.4.5, 2.4.6) LA08 Make connections (cause and effect) within a text (Reading GLE 2.3.1) IA18 Make connections (cause and effect) within a text (Reading GLE 2.3.1) LA09 Analyze author's purpose and evaluate effectiveness for different audiences (Reading GLEs 2.4.2, 2.4.3, 2.4.4) IA19 Analyze author's purpose (including distinguishing between fact and opinion) and evaluate effectiveness for different audiences (Reading GLEs 2.4.2, 2.4.3, 2.4.4) LA10 Extend information beyond text—apply information, give a response to reading, express insight gained from reading (Reading GLEs 2.4.1, 2.4.5) IA20 Extend information beyond text—apply information, give a response to reading, express insight gained from reading (Reading GLEs 2.4.1, 2.4.5)

9. N=3,899 N=10,448 N=32,239 N=26,748 Percent of Students at Each Level 46.5 38.6 44.5 41.4 40.0 31.1 35.9 27.9 27.0 21.5 22.4 17.7 15.6 16.836.0 28.3 25.3 19.4 14.0 27.4 27.0 31.2 34.6 11.4 6.4 4.7 4.9 5.2 4.9 5.1 7.3 7.5 42.7 43.3 43.4 24.0 1997 1998 1999 2000 2001 2002 2003 2004 2005 Did not meet standardExceeded standardMet standard Percentages do not total 100 as chart does not include untested students Grade 4 Reading

10. Grades 3, 4, & 5 2006

11. 2006 58,44413,550 14.6%3.4%46.4%34.4% 1997 36.0%11.4%31.1%16.8% Did not meet standardExceeded standard Met standard 29,62629,134 Grade 4 Reading Movement Across Achievement Levels Percent of Students at Each Level * Percentages do not total 100 as chart does not include untested students

12. Grade 3 High percent of blanks for level 1 students. When presented with a choice of words to describe a character, students performed low. (LA06) ** Explanation: Any of these words could describe the character in the story.** Choose the word that best describes the character in the story. Adj.Adj.Adj. Use two details from the story to support your choice. Author’s Purpose is difficult. (LA09/IA19) Grade 4 Successful at word choice for (LA06). Successful at Author’s Purpose. Low areas Cause/Effect (LA08) Compare/Contrast (LA07) especially in informational passages. Grade 5 Struggle with: Figurative Language Summaries and author’s purpose for level 1 student

13. Summarizing Students Who Met Standard on the Grade 4 Reading WASL Can respond to multiple choice and constructed response items for a variety of genres Use clear details from the text. Students do not use “vague” answers or simply repeat the question Can correctly select most of the multiple choice responses and cite appropriate text-based evidence for constructed response items Are familiar with reading assessment vocabulary (summary, prediction, compare and contrast, author’s purpose)

14. Summarizing Students Who Met Standard on the Grade 4 Reading WASL Refer to the text when answering both multiple choice and constructed response items. They know that evidence from the text is something they can “put their finger on” in the selection. Cite requested number of pieces of text- evidence in their responses (i.e., if asked to provide two differences, they provide two or more differences).

15. Constructed response items will follow these general guidelines: Short-answer (2 pts. with 9 lines) and extended-response (4pts. with 18 lines) Item-specific scoring guide is developed for each short-answer and extended-response item Extended-response items generally require longer, more detailed responses providing more evidence, information, or examples When an item has two parts, the parts sometimes appear separately (scaffolded), with lines following each part. This encourages students to notice and answer both parts of the item. Note: With paired passages, items follow each passage. The items requiring comparison between the passages appear last, after both passages. There should always be more than one item that compares the two passages. Constructed-response items should be asked about individual passages in addition to the passages as a pair.

16. Multiple choice items will follow these general guidelines: Each multiple-choice item contains three answer options, only one of which is correct. The choices are approximately the same length, have the same format, and are syntactically and semantically parallel; students should not be able to rule out a wrong answer or identify a correct response simply by virtue of its looking or sounding different. Distracters adopt the language and sense of the material in the selection so that students must think their way to the correct answer rather than simply identify incorrect responses by virtue of a distracter's obviously inappropriate nature. Distracters should always be plausible (but of course incorrect) in the context of the selection. To the greatest extent possible, no item or response choice "clues" the answer to any other item.

17. Scoring Anchor, Practice, and Qualifying sets are used to train in scoring. MC=1, SA=2, ER=4 Cut Score Rubrics –Summarizing –Bullet points vs. Letters

18. Released Item Documents Items and Passages that were previously used on the 2006 Operational test, released to the field for teacher use. Released from 3, 5, 6 & 8 Item-specific scoring guides, student responses, and annotations for these responses. The items were scored by teachers in the summer of 2006 Contains school/district/state data table with state-level data already in place

19. Involving Students in Assessment  Have students complete the items  Train students on the sample student responses, then have students score their own response using the scoring guide (rubric)  Open discussion “What do the results tell us?”  Mini-lessons to follow up in area of concern  Train student to use WASL stems (questions)

20. How Can We Help You? OSPI @ www.k12.wa.uswww.k12.wa.us –Reading –Assessment Committee Application Support Beth.Dorr@k12.wa.usBeth.Dorr@k12.wa.us or 360-725-6442

21. What can I do for YOU? Website: www.k12.wa.us Assessment Reading Assessment

22. Click on Application Click drop-down menu for all of the dates

23. Beth Dorr Reading Assessment Specialist beth.dorr@k12.wa.us Beth Dorr Reading Assessment Specialist beth.dorr@k12.wa.us