1. Science Test Design & Item Specifications Release
Hello, and welcome to this webinar presentation about the “Science Test Design & Item Specifications Document Release.” My name is Dawn Cope, and I am the Science Assessment Lead at OSPI.
I need to cover a few logistical points before we get too far. The first is that we are recording this webinar. That recording will be available through your District Test Coordinator. We will give more details about accessing the recording at the end of the webinar.
Second, the final PowerPoint slides, with our script in the notes section, will be posted to the Science Assessment webpage by the end of this week. Tony has sent a pdf for note-taking to everyone via . If you didn’t notice it yet, I’ll give you a moment to check for it.
[PAUSE]
And third, we have muted all our participants, but we will be taking questions through the Q&A box function in the Zoom Meeting. Please do not use the Hand Raise button as we will not be using it today. We may answer a few of the questions during our time together today, but we will most likely not have time for all of them. We will compile the questions into an FAQ document that we will post along with the PowerPoint slides on our webpage. It will probably take us until February 9th to have the FAQ document finalized and posted.
Assessment Development Team
January 24, 2018
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

2. Contact Information
General account
Dawn Cope—Science Assessment Lead
Amy Viveiros—Science Assessment Specialist
Kara Todd—Content Coordinator for Assessment Development
Anton Jackson—Director of Assessment Development
Now I’d like to let you know who else is in the room with me today. Again, my name is Dawn Cope, and I’ll be presenting information today. Amy Viveiros, Science Assessment Specialist and Kara Todd, our Content Coordinator, will be monitoring the Q&A box for us.

3. Agenda
This presentation contains a script in the notes section. If you are accessing this information with a screen reader, be sure you are reading the notes section as well as the text contained in the slides.
Goal: To help educators understand the information contained in the Washington Comprehensive Assessment of Science (WCAS) Test Design & Item Specifications documents.
The big goal for today is to help educators understand the information contained in the Test Design & Item Specifications documents for the Washington Comprehensive Assessment of Science.
The first working drafts were released last month.
The purpose of the Test Design and Item Specifications Documents is to describe how the WCAS are designed to assess the state standards.
The Test Design and Items Specifications documents are not intended to guide classroom instruction. Classroom instruction should align to the Washington State K–12 Science Learning Standards (NGSS), not the WCAS.

4. Document Location
Science Assessment webpage:
You can find the Test Design and Item Specifications documents by going to the Science Assessment webpage. The documents are the first set of resources listed, as you can see at the bottom of this screen shot.
There are separate documents for 5th grade, 8th grade, and high school. Many parts of the documents are similar. Key differences occur in the Test Blueprint and the Item Specifications sections of the document. You can refer to any of the three grade level documents as we proceed through the webinar today.

5. Document Structure Front Matter and Global Information (pages 1-17)
Purpose and Development Cycle
Structure of the Test
Item Types
Test Design
Standards overview
Structure of an item specification
Resources and References
Item Specifications (pages 18-end)
The front half of the current drafts contains front matter and global information (pages 1-17). We will be touching on many components of this section during the this webinar. One of the purposes of the documents is to serve as a reference for answering important, and commonly asked questions about the WCAS, so we encourage you to become familiar with the draft for your grade level or grade levels. When future drafts of the document are released, we encourage you to revisit this front matter, as we may revise it for clarity if needed.
The second half of each document contains a small sample of the item specifications (pages 18 to the end). This section will grow in future drafts until specifications for all performance expectations in the state standards are included.

6. Item Specifications Section
The current draft represents a small sample of PE specifications for each grade. The sample will continue to expand through the 2017–18 and 2018–19 school years until full Performance Expectation coverage is achieved.
The small sample of PE Specifications included in the December 2017 draft were chosen to represent a variety of Science and Engineering Practices, Crosscutting Concepts, and Disciplinary Core Ideas.
The small sample of PE Specifications included in the December draft may or may not be assessed on the spring 2018 WCAS.
Before we walk through the Test Design and Item Specifications documents, we want to clear up a misconception about the Item Specifications section (pages 18 to the end).
Within the current drafts, and in every communication we have sent to the field about the Test Design and Item Specifications documents, we have included a statement that seems to be getting overlooked.
That statement is, “The Item Specifications section of the current draft represents a small sample of PE specifications for each grade. The sample will continue to expand through the 2017–18 and 2018–19 school years until full Performance Expectation coverage is achieved.”
We have heard that some have interpreted the small samples to mean that these are the only 6 Performance Expectations that will be on the WCAS at each grade this spring, or that these PEs are guaranteed to be assessed on the WCAS this spring so you MUST teach them. This is not correct information. When we discuss the test structure, we will show mathematically how this is not the case.
The small sample of PE Specifications included in the December 2017 drafts may or may not be assessed on the spring 2018 WCAS. The expectation is that students have been learning the science standards since they were adopted in 2013 and that any of the Performance Expectations within a grade band are eligible to be tested on the WCAS.
The small sample of PE Specifications (6 per grade level) included in the December 2017 drafts were carefully chosen to represent a variety of Science and Engineering Practices, Crosscutting Concepts, and Disciplinary Core Ideas.

7. PE Dimensions represented in the current Grade 5 Draft
Item Specification
Pages
Science and Engineering Practice
Disciplinary Core Idea
Crosscutting Concept
4-PS3-2
20-21
Planning and Carrying Out Investigations
PS3.A
PS3.B
Energy and Matter
5-PS1-1
22-23
Developing and using Models
PS1.A
Scale, Proportion, and Quantity
3-LS4-3
26-27
Engaging in Argument from Evidence
LS4.C
Cause and Effect
4-LS1-1
28-29
LS1.A
Systems and System Models
4-ESS1-1
32-33
Constructing Explanations and Designing Solutions
ESS1.C
Patterns
5-ESS2-2
34-35
Using Mathematical and Computational Thinking
ESS2.C
Scale, Proportion and Quantity
For example, in the 5th grade document, the 6 different items specifications chosen include two PEs from the Physical Sciences Domain, two PEs from the life sciences domain, and two PEs from the Earth and Space Sciences Domain. Among the 6 PEs, 5 different Science and Engineering Practices, 7 different Disciplinary Core Ideas, and 6 different Crosscutting Concepts are represented.

8. PE Dimensions represented in the current Grade 8 Draft
Item Specification
Pages
Science and Engineering Practice
Disciplinary Core Idea
Crosscutting Concept
MS-PS2-5
20-21
Planning and Carrying Out Investigations
PS2.B
Cause and Effect
MS-PS2-4
22-23
Developing and Using Models
PS4.A
PS4.B
Structure and Function
MS-LS2-4
26-27
Engaging in Argument from Evidence
LS2.C
Stability and Change
MS-LS4-4
28-29
Constructing Explanations and Designing Solutions
LS4.B
MS-ESS1-3
32-33
Analyzing and Interpreting Data
ESS1.B
Scale, Proportion, and Quantity
MS-ESS2-4
34-35
ESS2.C
Energy and Matter
For the 8th grade document the 6 different items specifications chosen include two PEs from the Physical Sciences Domain, two PEs from the Life Sciences domain, and two PEs from the Earth and Space Sciences Domain. Among the 6 PEs, 5 different Science and Engineering Practices, 7 different Disciplinary Core Ideas, and 5 different Crosscutting Concepts are represented.

9. PE Dimensions represented in the current High School Draft
Item Specification
Pages
Science and Engineering Practice
Disciplinary Core Idea
Crosscutting Concept
HS-PS1-5
20-21
Constructing Explanations and Designing Solutions
PS1.B
Patterns
HS-PS2-6
22-23
Obtaining, Evaluating, and Communicating Information
PS2.B
Structure and Function
HS-LS2-6
26-27
Engaging in Argument from Evidence
LS2.C
Stability and Change
HS-LS3-1
28-29
Asking Questions and Defining Problems
LS1.A
LS3.A
Cause and Effect
HS-ESS2-2
32-33
Analyzing and Interpreting Data
ESS2.A
ESS2.D
HS-ESS3-2
34-35
ESS3.A
ETS1.B
For the High School document the 6 different items specifications chosen include two PEs from the Physical Sciences Domain, two PEs from the Life Sciences domain, and two PEs from the Earth and Space Sciences Domain. Among the 6 PEs, 5 different Science and Engineering Practices, 9 different Disciplinary Core Ideas, and 4 different Crosscutting Concepts are represented.

10. Purpose of state assessments
So, let’s start at the beginning…
Again, you can refer to any of the three Test Design and Item Specifications documents—Grade 5, grade 8 or Grade 11 during the webinar.
The first page of the Test Design & Item Specification document details the purpose of the document and of the WCAS, so we will briefly touch on the purpose of state assessments. Washington has K–12 learning standards that define the knowledge and skills every student needs to be ready for career, college, and life when they complete high school. State-wide tests are designed to measure the level of student proficiency on those standards.

11. Accountability
Federal law, through the Every Student Succeeds Act (ESSA), requires that we give a state science assessment once each in elementary, middle, and high school for the purpose of school and district accountability.
Washington’s ESSA consolidated plan includes a new accountability structure. “Phase Two” of the indicators includes WCAS results from , 2019, and 2020 to identify schools in 2021.
Federal law, through the Every Student Succeeds Act (ESSA), requires that we give a state science assessment once each in elementary, middle, and high school for the purpose of school and district accountability. In Washington, we assess in grades 5, 8, and 11 to fulfill this purpose.
Our recently approved ESSA consolidated plan has a new accountability structure wherein 3 years worth of assessment results are combined together to identify schools and or districts for receiving extra support. The identification that is happening this spring does not include science test results, but the current planning for “Phase Two” does include adding the WCAS as one of the accountability indicators. That would mean that the combined, three-year summative WCAS results from spring 2018, 2019, and 2020 will be used to identify schools in 2021.

12. Every Student Succeeds Act
We are not the experts on the accountability structure, nor are we the experts on ESSA or the Consolidated plan. OSPI does have a webpage dedicated to the topic, with many resources, and we encourage you to go there if you want to learn more.
Link to Every Student Succeeds Act page

13. Assessment graduation requirements
House Bill 2224 (2017) delayed the science requirement until the class of 2021 (current 9th graders).
2018 Legislation to “delink” assessment from graduation
Senate Bill 6144
House Bill 2621
We are not going to spend a lot of time on this, but we know there will be questions about the assessment graduation requirements. In the 2017 Legislative session, house bill 2224 delayed the science assessment graduation requirement until the class of These students are the current 9th graders, who will take the WCAS in spring of 2020.
We are aware that there are at least two bills active in the current legislative season which propose to “delink” the assessments from the state graduation requirements. The bill numbers are here if you wish to track them. We are often asked about bills that are in the midst of the legislative process. We do follow these bills, and there are internal conversations about their possible impacts. It’s difficult to predict legislative outcomes, though, so we continue to focus on administering current rules. One fact that we do know is that we still have a federal requirement to test high school students in science, so we focus on making sure that test happens.

14. WCAS Development Process
Pages 1 and 2 of the Test Design and Item Specifications document give an overview of the WCAS Development Process.
Washington educators are heavily involved in this process, which takes place over the course of at least 2 years from when an item is first drafted to when it could possibly be seen by a student and count in their test score. The next two slides will detail those steps.
Educator involvement

15. Science Assessment Development Cycle (a)
(1) OSPI and Teachers develop
Test Design & Item Specifications
(2) OSPI leads Teachers in Item Cluster Writing Workshops
(3) Clusters are reviewed for content by OSPI/
contractor
(5) Field Testing
Items embedded in the operational test; not counted in student scores.
(4) Content Review by OSPI and Teachers & Bias/Sensitivity Review by community
The first step shown is the development of the Test Design and Item Specifications themselves. Step 1 does not happen in isolation, but actually takes place throughout each of the other steps of this process. We began thinking about the Test Design and Item Specifications when the standards were adopted in 2013 and through each of these work groups as we developed items based on the standards over the past few years. We will continue to refine them as we develop in the future.
The second step shown in the diagram is the Item Cluster Writing Workshops. This is where Washington educators work in groups write stimuli, items, and rubrics for item clusters. The clusters are then reviewed by OSPI and our contractors for content, including improvements to the artwork. Then educators in a Content Review workshop (step 4) review the products of the Item Cluster Writing Workgroup to ensure that every stimulus, item, and rubric is scientifically accurate and gathers appropriate evidence about student skills. At the same time, a separate committee of community members reviews the items and stimuli for any bias and sensitivity issues. Recommendations from the Bias/Sensitivity Review Workgroup are considered by the Content Review Workgroup.
After that we work with our test delivery vendor to get the items put into their system so the items can be tested by students. This “testing of the item” is called a Field Test—represented in step 5. Students usually see the Field Test items as part of their operational test, but the items are not counted in student test scores. I say “usually” because last spring we did a Field Test of WCAS items with high school students that was a separate test. Moving forward, we expect to have only embedded Field Tests. 15

16. Science Assessment Development Cycle (b)
(6) Field Test Range Finding and Rubric Validation of constructed response & TEI items by Teachers, OSPI, and contractor
(7) Field Test Scoring by contractor under OSPI supervision
(8) Content Review with Data of all items by Teachers, OSPI and contractor
(10) Operational Range Finding
confirm scoring sets by OSPI and contractors
(9) OSPI and Contractor use Test Map to build Operational Test; Test Given to Students
(11) Operational Scoring
by contractor under OSPI supervision
After student testing is completed, educators meet in Field Test Range Finding and Rubric Validation Workgroups (step 6) to look at a range of student responses to each item and decide how to score each response. This educator workgroup refines scoring rubrics and produces the materials that will be used to score the field test items. This includes the materials for hand scoring items, as well as checking the logic of computer scored items. Then the items are scored by our contractor.
The 8th step in the process is, the Content Review with Data Workgroup, which usually meets in late September. In this workgroup, educators use item performance data (which is a variety of statistics about how students responded to the items), as well as members’ science content knowledge, to decide whether each item should advance into the item bank.
Step 9 actually takes about 6 months, from the point when we put a set of items together into a test form in the fall to when students see that test the following spring. Student responses to constructed response items from the first operational test are incorporated into the scoring materials during a meeting between OSPI staff and our scoring contractor after spring testing has completed. Then our contractor conducts operational scoring.
This year we will have a special step #12, which is Achievement Level Setting which is where the cut scores are determined by a group of Washington educators.
With our previous science assessments, you likely got used to scores being available in the Online Reporting System sometime in July, and then paper score reports arriving in district just before the start of the school year. For 2018, the WCAS results will be slightly delayed from this timetable because of the Achievement Level Setting activities. We will not know the timetable with any firmness until mid-August, and those dates will be communicated to District Test Coordinators at that time.
After 2018, WCAS score results will likely be available in the July timeframe. The WCAS is not adaptive like the Smarter Balanced tests, and while the cut scores will remain constant, we have to go through a process called Equating each summer before student scores can be released. To do that, we have to conduct the Operational Range Finding and Operational Scoring steps each year after the testing window closes. These processes are not conducive to a 3-week score turn around like the Smarter Balanced tests have.
[PAUSE]
To circle back to the Test Design and Item Specifications… I said on the previous slide that we refine them throughout each step of this development process. After we have released a versions that contains an Item Specification for each PE, we will typically release an updated Test Design and Item Specifications draft once per year. This release will likely take place after the Content Review with Data meeting in the fall, so usually posted in November.
If you are interested in participating in one of these workgroups in the future, we’ll give you some information at the end about how to become involved. 16

17. Features and Item Types
Pages 4 through 7 of the Test Design and Item Specifications document give details about the online features and item types for the WCAS.

18. Features & Item Types Features Item Types Collapsible Stimuli
Locking items
Multipart items
Animation
Item Types
Edit Task Inline Choice
Grid Interaction
Hot Text
Multiple Choice
Multiple Select
Short Answer
Simulation
Table Input
We have listed these features and item types for you here. We suggest that you read the information presented in Test Design and Item Specifications document, then go try the features and item types out for yourself using the Online Training Tests. We presented a webinar 2 weeks ago about how to use the Training Tests with students. The presentation slides and a document with answers to questions from that webinar are posted on the Science Assessment webpage in the section just below the Test Design and Item Specifications documents we showed on slide 4 today.
Unfortunately, we ran into a problem with the audio recording of the Online Training Test webinar, so we are having a delay in posting it. We are working to get the recording available as quickly as we can, and estimate that District Test Coordinators will have access to it early next week.

19. NGSS 101 Washington state 2013 k-12 science learning standards
Pages 10 through 12 of the Test Design and Item Specifications document describe how the standards are the basic foundation of the design of the WCAS.
The standards that the WCAS is designed to test are the Next Generation Science Standards, which were adopted in fall When the Superintendent adopted the NGSS, OSPI re-named the NGSS as the Washington State 2013 K-12 Science Learning Standards. We know that this sometimes causes a little bit of confusion,--two names for the same exact thing--so we try our best to put both names in close proximity to each other at the start of any of our documents, and then refer to them simply as “the standards” throughout the rest of the document.
Washington state 2013 k-12 science learning standards
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

20. NGSS Resources Next Generation Science Standards (NGSS)
NGSS Appendices
The K-12 Framework for Science Education
SAIC Assessment Framework
If you are wondering which NGSS resources we used as we developed the Test Design and Item Specifications, the four listed on the screen are the primary resources.. We have links and descriptions of these documents on page 16 of the Test Design and Item Specifications document.

21. NGSS Performance Expectation
The structure of the standards is very important to understand before we move into explaining the structure of the WCAS.
The standards are organized into Performance Expectations, or PEs. Each PE provides a statement of what students should be able to do by the end of instruction. This screenshot shows one PE, MS-ESS1-2. The “MS” tells us that it is a middle school PE, the “ESS1” refers to the overarching concept of “Earth’s Place in the Universe,” and the “2” tells us that it is the second in a series of middle school PEs about that overarching concept.
The text of the PE starts with the PE Statement (in black font) and may be followed by a clarification statement and/or assessment boundary in red font. The clarification statements supply examples or additional clarification to the performance expectations. The assessment boundaries are meant to specify limits to large-scale assessment. They are not meant to put limits on what can be taught or how it is taught, but to provide guidance to assessment developers.
The PE statement incorporates three dimensions, shown below in the Dimension Boxes. These include a science and engineering practice (SEP), one or more disciplinary core ideas (DCI), and a crosscutting concept (CCC). The PE statement can provide some insight as to how the student is expected to utilize the SEP, DCI, and CCC together to achieve the performance expectation.

22. Physical Sciences Domain Earth and Space Sciences Domain
NGSS PEs per Grade Band
Each Performance Expectation (PE) includes a Science and Engineering Practice, a Disciplinary Core Idea, and a Cross-cutting Concept
Tested grade level
Grade Band of NGSS
Physical Sciences Domain
Life Sciences Domain
Earth and Space Sciences Domain
Engineering Domain
Total # of PEs 5
3-5 17 12 13 3 45 8
Middle school 19 21 15 4 59 11
High school 24 71
This chart groups the number of PEs by science domain (Physical, Life, and Earth and Space Sciences) and gives the total number of PEs in each grade band.
These totals presented a challenge to those thinking about implementing the standards through the curriculum lens. One of the solutions that was arrived at is called “bundling.” According to a section about bundling on the NGSS website: "Bundles are groups of standards arranged together to create the endpoints for units of instruction…. Bundling is helpful step in implementing standards because it helps students see connections between concepts and can allow more efficient use of instructional time.” This concept of bundling now plays a key role in our test design, which is a great example of how the teaching of the standards is driving the assessment of the standards.

23. Test Design Structure of the WCAS
Pages 3, 8, and 9 of the Test Design and Item Specifications document describe the structure, the test design of the WCAS. The tests are composed of item clusters and standalone items. It will take us a few slides to build the full picture of what an item cluster is and what standalone items are, and how they work to form a test.
Structure of the WCAS
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

24. Cluster Map
Item clusters that assess a PE bundle make up the core of the WCAS. A PE bundle is generally two or three related PEs that are used to explain or make sense of a scientific phenomenon or a design problem.
Developing an item cluster that fully supports the PE bundle is like constructing the pieces of a puzzle to form a complete picture. Each item within the cluster will align to two or three dimensions from one or more of the PEs in the bundle. Achieving as full coverage as possible requires developing items that target a variety of the dimensions represented in the PE bundle.
Each item is linked to a stimulus or stimuli and to the other items within the item cluster. The stimulus may be interspersed among the items to add information as needed.
A phenomenon, shown in the middle of this chart, is used to give the cluster coherence. All items must support the particular phenomenon identified. The items must progress logically and contain internal scaffolding to support the student’s progression to higher levels of complexity.
Item writers can bundle PEs in multiple ways:
PEs may be bundled across domains (e.g., Life Science and Earth and Space Science PEs) or may be domain-specific (e.g., all Life Science),
PEs may be bundled across grades in the elementary band (e.g. grade 3 Physical Science, grade 5 Physical Science) or may be grade-specific (e.g., all grade 5 Physical Science),
PEs may be bundled with dimensional overlap (e.g., all PEs have the modeling SEP) or bundled to include a range of dimensions (e.g., the PEs have no SEP, DCI, or CCC in common).
Some PEs may be robust enough to support an entire item cluster on their own, but most bundles will consist of two or three PEs. An individual PE may be used in more than one bundle, but care will be taken to ensure that no PE will be over-represented on the WCAS.

25. Performance Expectation Coverage
Stand Alone items
6-12 items per operational form
Each item must assess 2 or 3 dimensions of a single PE
Increases DCI, SEP, and CCC coverage for the whole test
Item Clusters
3-6 items per cluster
5-6 clusters per operational form
One PE from each domain (PS, LS, ESS, and ETS) in at least 1 item cluster
A minimum of 3 different SEPs across the clusters
A minimum of 3 different CCCs across the clusters
Each student test will start with 6 to 12 stand alone items. Each standalone item must assess 2 or 3 dimensions of a single performance expectation. These stand alones will increase the PE coverage for the whole test.
After the stand alone items, which cover the entire width of the computer screen, the test will turn to item clusters. Students will know that they are looking at a cluster because the screen will be split between the stimulus on the left and an item or group of items on the right. There will be 3 to 6 items per cluster. The grade 5 and grade 8 tests will have 5 clusters, and the grade 11 test will have 6 clusters. There will be at least one PE from each domain (Physical Sciences, Life Sciences, Earth and Space Sciences, and the Engineering, Technology, and Application of Science) assessed within an item cluster. There will be a minimum of 3 different Science and Engineering Practices across the clusters, and a minimum of 3 different Crosscutting Concepts across the 3 clusters.
If it feels like you are starting to put together a 4-D puzzle in your head with all of these requirements….your are on the right track for what test building feels like.

26. Possible # of PEs per test
Operational Test
Grade 5/8
Standalones
6-12 PEs
Cluster 1
1-3 PEs
Cluster 2
Cluster 3
Cluster 4
Cluster 5
Operational Test
Grade 11
Standalones
6-12 PEs
Cluster 1
1-3 PEs
Cluster 2
Cluster 3
Cluster 4
Cluster 5
Cluster 6
Here is a visual of how the many Performance Expectations could be in each part of the operational section of a test.
Remember each standalone assesses 1 PE, each cluster assesses 1-3 PEs, and a single PE isn’t over-represented is a test.
Therefore, The grade 5 and grade 8 tests are likely to assess between PEs per year, and the grade 11 could assess around PEs tested per year.
These numbers are not close to the total number of PEs eligible to be tested each year. (See slide 23 where we showed the total number of PEs per grade band).
The federal law that requires us to test both the depth and the breadth of our standards does allow us to sample from those standards each year in this manner. We did this kind of sampling with our previous science tests as well.

27. Structure and Administration
Operational Section
Count toward a student’s score
Fixed form
Field Test Section
Embedded in the online administration
Does not count toward a student’s score
A cluster or several standalone items
Administration
Can be administered in multiple sessions like the Smarter Balanced ELA and Math tests
1 to 3 sessions recommended
You likely noticed on the previous slide that we marked those stand alones and clusters as the “Operational Test.” That is the section of the test that will count towards a student score. That part of the test will be a fixed form test.
As we stated earlier, there will be items field tested as part of each years WCAS. They will be embedded in the online test and do not count toward a student score. These field test items will either be a cluster of items, or several stand alone items.
These two sections will not be separated from each other with a segment break like we had in the online tests the previous 2 years. We inserted a segment break when we had items based on the 2009 standards for the operational test, and then items based on the 2013 standards for the field test. Now that all the items are based on the same set of learning standards, there is no need for a segment break. We ask that you do not tell students that the last set of items they will see are field test items. We need them to put forth as much effort into those items as they do for the earlier items as possible, to get as accurate information about how the field test items function as possible.
Something that is new for science this year is that we will allow the WCAS to be given to students over multiple test sessions, much like the Smarter Balanced tests. We do recommend that 3 sessions be the maximum number of sessions used with students. Remember, each time a student starts a session, they have to go through all the log in procedures, so that does add to your total testing time. The first question all the Test Coordinators ask after they hear about the multiple sessions is “What is the expiration date for the tests? How many calendar days do they have to complete their test?” After talking through the possibilities, and knowing that the WCAS windows are smaller than the Smarter test windows, we have decided to make it simple and just have the tests expire at the end of the test window in June.
The first question all the Test Administrators ask is, “How do I break up the test into 2 or 3 sessions? When will I know to have the students pause their test?” We have some ideas for that on the next few slides.

28. Planning for 2 Test Sessions
Grade 5/8
Standalones
Cluster 1
Cluster 2
Cluster 3
Cluster 4
Cluster 5
Cluster or Standalones
Grade 11
Standalones
Cluster 1
Cluster 2
Cluster 3
Cluster 4
Cluster 5
Cluster 6
Cluster or Standalones
This is how you could break up the test into 2 sessions. Students could work through the stand alone items, and the first 3 clusters of items during their first session. Then, they could complete the remaining clusters and field test items during their second session.

29. Planning for 3 Test Sessions
Grade 5/8
Standalones
Cluster 1
Cluster 2
Cluster 3
Cluster 4
Cluster 5
Cluster or
Grade 11
Standalones
Cluster 1
Cluster 2
Cluster 3
Cluster 4
Cluster 5
Cluster 6
Cluster or Standalones
This is how you could break up the test into 3 sessions.
Students could work through the stand alone items, and the first 2 clusters of items during their first session. Then, during the 2nd session, they could complete 2 or 3 clusters of items depending on the grade level, and in the 3rd session they finish out the rest of their test.
There will be instructions about how to handle testing over multiple sessions in the TA Script for Student Directions. We have a few things that we still need to work through in this area (including how the locking items will interact) when we test the tests later this spring. We will include any wisdom or guidance in that TA Script, so please read it carefully if you break up the test into sessions.

30. Blueprints
The test blueprints are on page 9 of the Test Design and Item Specifications document. The blueprints for the WCAS are different than what we provided for the former science tests. They do not say how many of the items will be multiple choice or short answer like our previous science test maps showed. They don’t get down into some of the details like the Smarter Balanced Blueprints do.
The reason is simple: the standards are not simple, and there is no formula for a cluster of items. When we were writing items for our previous science tests, there was almost a cookie cutter layout for how many items to write, how many were short answer, and which short answer went with which type of stimulus. We don’t have that kind of formula anymore.
What the blueprints DO tell you is how many score points will be included in each of the reporting categories.

31. NGSS PEs per Grade Band revisited
Each Performance Expectation (PE) includes a Science and Engineering Practice, a Disciplinary Core Idea, and a Cross-cutting Concept
Tested grade level
Grade Band of NGSS
Physical Sciences Domain
Life Sciences Domain
Earth and Space Sciences Domain
Engineering Domain
Total # of PEs 5
3-5 17 12 13 3 45 8
Middle school 19 21 15 4 59 11
High school 24 71
Remember this slide from the start of our presentation? These middle columns [CLICK] are the foundation of the blueprints.

32. Grade 5 Blueprint 40% 35-45% 12-16 29% 24-34% 8-12 31% 26-36% 9-13
Reporting Claim
Percentage of PEs per Science Domain in the Standards
Percent range for the WCAS per Science Domain
Score point range for WCAS per Science Domain
Practices and Crosscutting Concepts in Physical Sciences
40%
35-45%
12-16
Practices and Crosscutting Concepts in Life Sciences
29%
24-34%
8-12
Practices and Crosscutting Concepts in Earth and Space Sciences
31%
26-36%
9-13
35 Total points
6-12 standalone items
5 clusters
Estimated testing time: 90 minutes
ETS PEs assessed but not included here.
We used those numbers to find the percentage of PEs per science domain in the standards, which is the first column here. OSPI confirmed with several teacher work groups over the past few years that the WCAS should reflect similar percentages of points per science domain. That is what you see in the middle column. The last column takes the 35 total points that will be on the grade 5 exam, and breaks them out by those percentages.
This slide shows the Grade 5 blueprint from page 9 of the Grade 5 Test Design and Item Specifications document. In addition to the reporting claims, percentage range, and point range per domain, this slide summarizes the total points, the standalone range, the number of item clusters and estimated testing time for the 5th grade test.

33. Grade 8 Blueprint 35% 25-35% 12-16 38% 33-43% 13-17 27% 22-32% 9-13
Reporting Claim
Percentage of PEs per Science Domain in the Standards
Percent range for the WCAS per Science Domain
Score point range for WCAS per Science Domain
Practices and Crosscutting Concepts in Physical Sciences
35%
25-35%
12-16
Practices and Crosscutting Concepts in Life Sciences
38%
33-43%
13-17
Practices and Crosscutting Concepts in Earth and Space Sciences
27%
22-32%
9-13
40 Total points
6-12 standalone items
5 clusters
Estimated testing time: 105 minutes
ETS PEs assessed but not included here.
This slide shows the Grade 8 blueprint information from page 9 of the Grade 8 Test Design and Item Specifications document.

34. Grade 11 Blueprint 36% 31-41% 14-18 28% 23-33% 11-15 45 Total points
Reporting Claim
Percentage of PEs per Science Domain in the Standards
Percent range for the WCAS per Science Domain
Score point range for WCAS per Science Domain
Practices and Crosscutting Concepts in Physical Sciences
36%
31-41%
14-18
Practices and Crosscutting Concepts in Life Sciences
Practices and Crosscutting Concepts in Earth and Space Sciences
28%
23-33%
11-15
45 Total points
6-12 standalone items
6 clusters
Estimated testing time: 120 minutes
ETS PEs assessed but not included here.
And, this slide shows the Grade 11 blueprint information from page 9 of the High School Test Design and Item Specifications document.
You’ll notice the red note on each of the blue print slides: “ETS PEs assessed but not included here.” We want to be clarify that even though the Engineering Design Domain (ETS) does not have its own reporting claim, at least one ETS PE will be assessed each year on the WCAS at each grade level. ETS PEs will be bundled with a PE from the Physical Science, Life Sciences, or Earth and Space Sciences domain.

35. Item Specifications
Now we are going to move into the Item Specifications part of the document.
Remember, the Item Specifications section of the current draft represents a small sample of PE specifications for each grade. The sample will continue to expand through the 2017–18 and 2018–19 school years until full Performance Expectation coverage is achieved.
The Test Design and Items Specifications documents are not intended to guide classroom instruction. Classroom instruction should align to the Washington State K–12 Science Learning Standards (NGSS), not the WCAS.
Page 13 provides an over view of the parts of an item specification. We are going to use the specification for 4-LS1-1 as the example.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

36. Page 1 Each PE Item Specification has two pages.
The first page contains the text of the PE itself [CLICK]
the dimensions information for that PE {CLICK] SEP
[CLICK] DCI
[CLICK] CCC
as well as links to the K-12 Framework for science education and NGSS Appendices where more information about the dimensions can be found.
Then the item specification lists the clarification statement [CLICK]
and assessment boundary [CLICK] from the standards.
Next, you’ll find a list of expected DCI vocabulary. Please note that the vocabulary listed in this section may be used to assess the PE, and will not be glossed in the online test delivery system. Students are expected to be able to read these words, and to use them in the context of the DCI. OSPI will not provide a list of definitions for expected DCI vocabulary.

37. Page 2
The second page contains a chart at the top with the Item Specifications [CLICL]. Each PE will have four item specifications. The first item specification has a code with a “.1” added onto the end of the PE number. For example, the code for the first item specification for PE 4-LS1-1 is 4-LS1-1.1 The first item specification listed will describe how a three dimensional item is written to assess the SEP, DCI, and CCC for the PE.
The second item specification listed--has a code with a “.2” added onto the end of the PE number. For our example, this specification is coded 4-LS The second item specification will describe how a two dimensional item is written to assess the SEP and DCI.
The third item specification listed--has a code with a “.3” added onto the end of the PE number. For our example, this specification is coded 4-LS The third item specification will describe how a two dimensional item is written to assess the DCI and CCC.
The fourth item specification listed--has a code with a “.4” added onto the end of the PE number. For our example, this specification is coded 4-LS The fourth item specification will describe how a two dimensional item is written to assess the SEP and CCC for the PE.
When you read each specification, you’ll notice key words bolded. The “Details and Clarifications” section includes information that helps unpack the those key words. [CLICK]
For example, if I want to write three dimensional WCAS item, I can see from the details and clarifications section that there are several ways to assess the SEP “Construct an argument”. For example, I could write an item in which students use data or evidence to develop an argument or use data or evidence to evaluate a claim OR use data/evidence to compare two arguments. There are several examples of System Models I could use to incorporate the CCC into my three dimensional item. In addition, there are examples of assessable structures and functions I could use to incorporate the DCI. As noted in the Details and Classifications sections, these examples are not exhaustive.
If I want to write a two dimensional item, I would use the appropriate item specification as a guide, and then use the details and clarifications sections to help put the pieces together.
As described at the beginning of today’s webinar, the item specifications are always in draft form, and will be modified annually to reflect lessons learned during the assessment development cycle. The bottom of this page has a Modification Log, which will be used to document changes to the Item Specifications over the years. [CLICK]

38. Expected SEP and CCC vocabulary
cause
energy
limitations
scale
change
evidence
mass
solution
claim
explanation
matter
stability
constraints
function
model
structure
criteria
graph
pattern
system
data
interaction
proportion
system model
effect
investigation
quantity
volume
On the last page of the Item Specification section (p. 36) you’ll find a list of expected SEP and CCC vocabulary. This list is the same at all three grade levels.
Please note that the vocabulary listed in this section may be used to assess the PE, and will not be glossed in the online test delivery system. Students are expected to be able to read these words, and to use them in the context of the SEP or CCC. This list may have words added to it as we develop more items, so be sure to check it when new drafts are released.
OSPI will not provide a list of definitions for expected DCI vocabulary.

39. Conclusion
The Test Design and Items Specifications documents are not intended to guide classroom instruction.
Classroom instruction should align to the Washington State K–12 Science Learning Standards (NGSS), not the WCAS.
The Test Design and Items Specifications documents will be added to and revised in the next few years.
To conclude our conversation about the Test Design and Items Specifications documents, I would like to remind you that they are not intended to guide classroom instruction.
They are intended to guide the writing of test items. Classroom instruction should align to the Washington State K–12 Science Learning Standards (NGSS), not the WCAS. Think of Test Design and Items Specifications documents as another resource, much like the NGSS Appendices, to help clarify our understanding of the standards.
The Test Design and Items Specifications documents will be added to in the next few years, and will be revised through the steps of the assessment development cycle.

40. Other Assessment Resources
We do have a few other resources we’d like to point out to you, some of which we referenced during the presentation.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

41. Online Training Test Support document
Detailed information about how to access the Training Tests
Suggestions for helping students work with each item type
An answer key and other information for each question
Appendices with:
Tool Button sheet
Introduction to the online calculator
Names and locations of other resources
The first is a document called the “Online Training Test Support” document that was published January 5th on our website. This pdf has detailed information about how to access the Training Tests, many suggestions for working with each item, and an answer key and other information for each question.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

42. Science Training Tests
Help students become familiar with the features and tools of online tests.
Available on the WCAP portal.
Grades 5, 8, and 11
One cluster (stimuli and items) and one standalone item at each grade
All three training tests should be used at each grade level to experience most online item types
Classroom instruction should align to the Washington State K–12 Science Learning Standards (NGSS), not the WCAS.
The second is the Training Tests. They are available on the WCAP portal. There is one training test for each tested grade. They have one cluster (stimuli and items) and one standalone item at each grade. All three training tests should be by students in each grade level to experience most online item types. Our webinar 2 weeks ago was about this topic, and most of the files from that webinar are currently on the science webpage. The recording will be posted for District Test Coordinators soon.

43. Formative Assessment Resources
Formative assessment resources are available that can help you and your students measure progress toward learning the Science K-12 Learning Standards as assessed on the WCAS
STEM Teaching Tools website
Includes tools that can help teach science, technology, engineering and math (STEM) and is currently focused on supporting the teaching of the Next Generation Science Standards (NGSS). Each tool is focused on a specific issue and leverages the best knowledge from research and practice.
Recommended tools:
Prompts for Integrating Crosscutting Concepts Into Assessment and Instruction
Integrating Science Practices Into Assessment Tasks
How can formative assessment support culturally responsive argumentation in a classroom community?
How can assessments be designed to engage students in the range of science and engineering practices?
Steps to Designing a Three Dimensional Assessment
The third is a list of formative assessment resources that we have found helpful.

44. Reminders & Wrap up
As we start to wrap up, please be sure to put any questions you have in to the Q&A box.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

45. Upcoming PD Opportunities
Event
Timing
Item Cluster Writing
Spring 2018
Content Review
Summer 2018
Field Test Range Finding and Rubric Validation
Content Review with Data
Fall 2018
Achievement Level Descriptor Development*
November 2017
Contrasting Groups Study*
Early Spring 2018
Alignment Study*
Achievement Level Setting*
August 2018**
*Special work connected to the premiere of a new test. Will only happen this school year.
**Test scores will be released slightly later than usual.
Here are the upcoming professional development opportunities for educators.

46. Other training opportunities
Sign up to get messages from the Science Assessment Development team:
Go to the Subscribe page for GovDelivery. Enter your address. On the Subscriptions page, select Content Areas > Science, then select the grade band(s) for which you would like to receive information.
Check for upcoming work groups on our Professional Development page:
If you would like to get a message when we open up the application process for these meetings, or to get other messages directly from the Science Assessment Development team please register on our list:
Go to the Subscribe page for GovDelivery, which you can access from the red envelope icon on our webpage. Enter your address. On the Subscriptions page, select Content Areas > Science, then select the grade band(s) for which you would like to receive information.

47. Contrasting Groups Study
WCAS Contrasting Groups Study (winter/spring 2018) Educators use the Achievement Level Descriptors (ALDs) to predict how their students will perform on the WCAS. Online ALD training is provided prior to this study. Classroom teachers submit predictions through EDS.
This study is how every science teacher in the state can have their voice heard when the cut scores are set at the Achievement Level Setting meeting in August.
Information from the contrasting group study, as well as performance/impact/student test data, is considered by the achievement level setting committee to inform the establishment of cut-score for the achievement level ranges.
Another great opportunity to be involved from the comfort of your own computer is the Contrasting Groups Study.
This study is how every science teacher in the state can have their voice heard when the cut scores are set at the Achievement Level Setting meeting in August. (Sorry, we can’t fit all 5,000 of you in the meeting space in August.) Educators use the Achievement Level Descriptors (ALDs) to predict how their students will perform on the WCAS. Online ALD training is provided prior to this study using OSPIs Moodle site—so you can go through it at your own pce. Classroom teachers submit predictions of their student performance through the EDS system before the test window opens.
The results of the study are one piece of information the achievement level setting panels will use to establish the cut scores.
You probably have a ton of questions about this. All we have time to say today is that more information will be provided in the next month about this study and how to participate, through the GovDelivery list we mentioned on the previous slide and through WAW messages to DCs.
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION
9/18/2018

48. Where to find the materials
Audio file (DCs only)
EDS: WAMS—Assessment Operations–File Downloads—2018 Administration
Presentation slides with script
Test Design and Item Specifications section of the Science Assessment webpage (by end of day 1/31)
FAQ document with answers to Chat questions
Test Design and Item Specifications section of the Science Assessment webpage (by end of day 2/9)
[PAUSE]
The Audio file for this webinar will only be accessible by District Test Coordinators. They will need to go into WAMS, within EDS, and follow this filepath.
EDS: WAMS—Assessment Operations–File Downloads—2018 Administration. If we have no technical difficulties, we should have it available early next week.
We will post this PowerPoint presentation, including notes with our script, on the Science Assessment webpage by a week from today. We’ll put it in the section about the “Test Design and Item Specifications.”
We will be working on the FAQ with all of your questions over the next week, and will post it next to the presentation by the end of the following week.
If you are not with a school district, contact Kara Todd next week about how to get the recording..
OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

49. Contact Information (again)
General account
Dawn Cope—Science Assessment Lead
Amy Viveiros—Science Assessment Specialist
Kara Todd—Content Coordinator for Assessment Development
Anton Jackson—Director of Assessment Development
Here is our contact information again. If you have specific questions about science content, please direct those questions to Dawn and Amy. If you have questions about accessing the files related to this webinar, please contact Kara. If you have broader questions about the state assessment system, including Smarter Balanced testing, please send those questions to Anton. If you are not sure where the question goes, please send it to our general account.
Thank you for your participation.