Presentation on theme: "Tips to Scoring the ELA Smarter Balanced Interim Assessments"— Presentation transcript:
1 Tips to Scoring the ELA Smarter Balanced Interim Assessments This training is intended for educators setting up access to and using the English Language Arts Smarter Balanced Interim Assessments through the WCAP portal.Comprehensive and BlockOffice of Superintendent of Public Instruction
2 Navigating Interim Support – WCAP Portal The Washington Comprehensive Assessment Program (WCAP) portal houses the:Interim assessmentsTeacher Hand Scoring System (THSS)Online Reporting System (ORS)Administration Manual Online - InterimThe WCAP portal is your access point for the Washington assessment system tools and resources. This is where the Smarter Balanced Interim and hand scoring resources are housed along with sign-in access for the Summative assessments. Other resources and tools on the portal include access to the digital library and administrative and teaching supports for schools and classrooms.
3 Start Here http://wa.portal.airast.org/ The first page of the portal displays recent announcements and display tabs to access all support materials. There are four access points: Students and Families, Test Administrators (TA), Test Coordinators, and Technology Coordinators. Some materials are available in multiple locations, but classroom educators will find what they need under the Test Administrators tab.
4 Manuals Interim Tests Reports ELA hand scoring training documents are also located here.Hand ScoringInterim TestsClassroom Activities for InterimOnce you’ve entered the Test Administrators (TA) interface, more options appear.The Test Administration Resources icon houses all manuals and non-secure resources.The TIDE icon, depending on your role, includes: managing user accounts, managing student information, assessment eligibilities and test settings, managing orders for testing materials, managing rosters of students, and managing appeals. Notice the lock in the bottom right corner indicating access is secure and will need a secure login to access. Secure material related to the assessments are also located here, including the summative Performance Task Classroom Activities and ELA Interim Scoring Training materials.The Test Administration icon is where educators assign, approve, and activate assessments their students are taking, both interim and summative.The Interim Assessment Classroom Activities icon houses the Classroom Activity that needs to preface the Interim Performance Tasks.Once students have completed the interim, educators access the Interim Teacher Hand Scoring System (THSS) icon to score student constructed responses and essays.Once all scores have been submitted, educators access the Online Reporting System icon to view scores for their assigned students.Reports
5 Administrator Roles within WCAP Portal Depending on your district size and configuration, these roles may be somewhat varied.District CoordinatorClassroom TeacherConfirm that student information has been rolled over into TIDEAssign educator rolesDecide which interim assessment to useVia the Test Administration Tab, select the test and activate the sessionApprove students so they may beginOnce all students are finished, stop sessionDistrict Coordinators confirm that all student information rolls over into TIDE and assigns educator roles. Classroom Teaches will assign the specific assessment, activate the session ones students are logged in and will stop the session when all students are finished.
6 Interim AssessmentsNow that the location of materials, access to tests, and user roles have been discussed, let’s take a look at the actual interim assessments.
7 Interim Assessment ICA and IAB Options There are two Interim Assessment Options available in the Test Administration icon.
8 Overview of Interim Process Decide which assessment your students should takeInterim Comprehensive Assessment (ICA)Interim Assessment Block (IAB)Check to be sure student rosters are loaded in TIDESet up the test using the Test Administration tabAdminister the test with studentsFor a successful interim administration, follow these four steps. Administering the interim is the classroom teacher’s opportunity to interact with students and the testing interface as they take the assessment.
9 Administration Characteristics ICAs and IABs can be administered multiple times during the yearTests draw from the same interim pool.Students may encounter some of the same items on the ICA as they will on the IAB.Decisions about test choice and frequency of use are determined by district or school plans.Currently both are fixed-form, online tests. Both will become adaptive over time as item bank allows.All Smarter Balanced assessment items come from the same pool. Because of the newness of the assessment, there are a limited number of test questions available. Students may encounter the same item in an ICA as they will in an IAB. Performance tasks in both interim pools (the ICAs and the IABs) are the same within a grade level. Due to the newness of the item bank, all students currently accessing a specific grade level interim will see the same assessment. This is called a fixed-form. As the item bank grows, these test will become adaptive.
10 Interim Assessment Similarities August 7, 2014Interim Assessment SimilaritiesBoth ICA and IAB options are:Available at grades 3, 4, 5, 6, 7, 8, and high schoolAdministered at teacher’s discretion, including using off grade-levelTeacher Scored Constructed ResponseMajority of items are automatically scoredShort-Text, brief writes, and full writes are hand-scored by teachers; requires scoring training using a rubric through provided materialsAdministering or offering two different types of assessments provide flexibility.This slide highlights the characteristics that are the same for both types of interim assessments.
11 Interim Comprehensive Assessment (ICA) Same design as summative assessmentAssess the same claims and standardsReports give the overall scale score, performance level designation, and claim score informationTiming of administration to mirror summativeApproximately 4 hours + 30 minute classroom activityIs arranged in two sectionsComputer Adaptive Test (CAT)Performance Task (PT)Teachers will need to hand-score short texts, brief writes and a full writeOnce teachers submit the hand scores, reports are availableReports provides teachers with information on a student’sGeneral areas of strength or needReadiness for the end-of-year summative assessmentBecause the ICA mirrors the summative test, it is divided into two sections, the Computer Adaptive Test (CAT) [Currently this portion of the assessment is non-adaptive, but it is equivalent to the CAT blueprint of the summative assessment] and Performance Task (PT). These can be given in one day, or students can log-in over multiple days to complete to complete the entire assessment.
12 A Closer Look at the ICA Reading Brief Writes Listening Research Computer Adaptive Test (CAT)*Performance Task (PT)ReadingBrief WritesListeningResearchClassroom ActivityLocated in TIDEAdministered prior to students accessing the PTPart 1: Access Sources and answer Research questionsPart 2: Full WriteThe ICAs have both the CAT portion and a Performance Task.CAT (sections 1 & 2)Reading, stand-alone Research, stand-alone Writing, and Brief Writes (section 1)Listening (section 2)Performance Task (section 3)A Classroom Activity is administered prior to students taking the PT section of the test.The PT is broken into two sections:Part 1: Read 2-5 sources and answer 2-3 Research questions related to the sources.Part 2: Students will be prompted to write a full essay (i.e. full write) using sources in Part 1 as evidence in their writing.The PT articles (or sources) are available during both parts of the PT. Students can take Global Notes during Part 1 and access and use their Global Notes in Part 2.*Currently this portion of the assessment is non-adaptive, but it is equivalent to the CAT portion on the summative assessment
13 ICA Estimated Time Content Number of Items Response Type **Computer Adaptive Test (CAT)Reading Lit13–15Mostly machine scored, a few hand scored45 min–1 hourReading Info14–16Brief Write6 brief writing tasksAll hand scoredEdit/Revise13–16All machine scored30 minListeningResearch17–18Performance Task (PT)***Classroom ActivityPart 1: Research2–3Part 2: Full Write1Hand scored11/2 –2 hours*Estimated time is based on 1–2 minutes to answer a MC question and up to 5 minutes to answer a constructed response. Plus time to read stimulus/sources.**Currently this portion of the assessment is non-adaptive, but it is equivalent to the CAT portion on the summative assessment***Located in TIDE- Administered prior to students taking the PTPlease note, these are estimated times.
14 Interim Assessment Blocks (IAB) Short, focused sets or “blocks” of items that measure similar content6-8 blocks per grade level, including ELA Performance Task blocksOffers a smaller set of targets and more flexibility for administrationReports state “Below Standard, “At/Near Standard,” and “Above Standard”Provides teacher withInformation about a student’s strengths or needsMore detailed content information for instructional purposesTeacher hand scoring, dependent on the blockSee next slideBlocks are shorter, more focused assessments. Educators can access a particular topic based on the block focus. Teachers can administer more than one in a day, or access them individually over a period of time. Reports are not as specific as the comprehensive since they are broken down into distinct blocks and no longer cover a range of ELA items.
15 ELA Blocks- Estimated time Number of ItemsResponse TypeEstimated Time*Reading Lit13–15Mostly machine scored, a few hand scored45 min–1 hourReading Info14–16Brief Write6 brief writing tasksAll hand scoredEdit/Revise13–16All machine scored30 minListeningResearch17–18All Performance Tasks2–3 research items and 1 full writeMostly hand-scored1–2 hours, including 30 minutes for the classroom activity.*Estimated time is based on 1–2 minutes to answer a MC question and up to 5 to answer a constructed response. Plus time to read stimulus/sources.Note: Some blocks require hand scoring while others will be scored completely by machine. While the summative items are professionally scored, interim hand scoring is done by local educators.
16 English Language Arts Blocks: Grades 3–5 These are the specific blocks available for grades Note that performance tasks are limited for the school year. Additional Performance Tasks are also located in each grade level of the ELA Practice Test for student use.
17 English Language Arts Blocks: Grades 6–8 These are the specific blocks available for grades Note that performance tasks are limited for the school year. Additional Performance Tasks are also located in each grade level of the ELA Practice Test for student use.* Not available in 2014–15
18 English Language Arts Blocks: High School These are the specific blocks available for High School. Note that performance tasks are limited for the school year. Additional Performance Tasks are also located in each grade level of the ELA Practice Test for student use.* Not available in 2014–15
19 Scoring the Assessments Once students have completed the comprehensive and/or block assessments, educators will have an active role in scoring students’ reading, writing, and research constructed responses plus the full write.
20 Accessing hand-scoring training materials. The training materials are in two locations. Educators may access the hand scoring training materials prior to students testing by locating them in TIDE under the Download Forms tab. This location allows educators to access the Exemplars and Training Guides prior to scoring and therefore allows more time to become familiar with the scoring process and criteria. The same hand scoring training materials are located in the Teacher Hand Scoring System (THSS) and can be accessed as each item is scored, but is not available until students have completed testing. Remember, the ICA requires hand scoring. Not all IABs require hand-scoring (see slide 15).Prior to testing, log in to TIDE and select the Download Forms tab which houses PDFs of each scoring set for every hand-scored item. These same scoring sets are also located in the THSS but cannot be accessed there until students are done testing and teachers actively start scoring.
21 Within TIDE, the ELA scoring materials are located in the Download Forms tab.
22 Teacher Hand Scoring System (THSS) Read and score student responsesAccess, download/print the Training Guide and Exemplar Sets to train on scoring as you are scoring*Access scores in the ORSELA IABs for Edit/Revise, Listening and Research do not require hand scoring.Once inside the THSS portal, educators will be able to select, read, and score students’ constructed responses.* To access materials prior to testing, log in to TIDE and select the Download Forms tab. These are the same scoring materials located in the THSS but can be accessed in TIDE prior to students testing.
23 Where are the constructed response items? ICA or IABNumber of Hand Scored ItemsResponse TypeEstimated Teacher Training Time*Reading Lit1–2Short-text45 min–1 hourReading InfoBrief Write6Brief write tasksAll Performance Tasks2–4 research items and 1 full write2-3 short-text and 1 full-write essay1–2 hours*Estimated time is based on teachers reading through the materials, practicing scoring the Prep set, and scoring the Check set on their own. This can be done in a PLC or individually.ICAThis slide identifies where and about how many hand-scored items are located in both the ICA and the IABs and shows estimates of how long it may take to train on an item for the first time.
24 Scoring Overview: Things to Consider August 7, 2014Scoring Overview: Things to ConsiderInterim assessments have the same item types as the summative assessmentMost items are machine scoredShort-text, Brief Writes, and Full Write are hand scored locally for interimsEducators use the Teacher Hand Scoring System (THSS) to score student responsesThe THSS cannot be accessed until testing is completeELA Rubrics and training materials are provided within TIDE and THSSAfter hand-scoring is complete, student scores will be available in the Online Reporting System (ORS)An overview of the scoring process
25 After students have completed the assigned test, you are ready to begin scoring. Each constructed response item has its own set of scoring materials.Reminder: materials can be found in TIDE prior to testing, or in the Exemplar and Training Guide tabs in the Teacher Hand Scoring System as tests are being scored.When you log in to the THSS, you will be taken to the Student Response List page (see next slide)Locate the student and the item you want to score.Click the Score tabTraining to score is something educators are encouraged to do prior to scoring actual student responses. To access materials prior to scoring students’ responses, locate the materials in TIDE. The same materials are available in the THSS and can be reviewed and used during scoring.
26 This is the first page of the THSS This is the first page of the THSS. Use the drop down menu to select the test and or session. You may also sort by student name, item number, session, or status. Once you are ready, click on the “Score” tab.
27 Student Response PageEach student’s response is on its own page (see next slide).Information on the page includesA rubricA Training Guide tabAn Exemplar Set tabA place to enter the scoreThe question or taskThe student’s response
28 Exemplar and Training Guide Tabs Student Response PageRubricOnce you click on “Score”, you will be directed to the individual student response page. This is where you can access the Exemplar and Training Guides (the same ones you will find in TIDE), a rubric, a drop down to enter the score, a condition code (information about the condition code can be found in the Training Guide), the item stem, and the student’s response. This particular screenshot does not show the item stem or the student response, however the response will be where this screenshot is cut-off.Student’s Response
29 What will I find in the ELA Training and Exemplar Tabs? Training GuideExemplarsInstructionsStimulusItem stemRubricCondition CodesClaim and Target SheetPrep SetSample student responses with scoresScoring commentsTrain using the rubricCheck SetSample student responses without scoresScores and comments located on a keyPractice scoring using the rubricReminder: these are the same materials also located in TIDE.
31 Online Reporting System To view student scores, back on the portal page, access the Online Reporting System.
32 Online Reporting System - ORS ICAOne overall ELA scoreClaim information also reportedBoth CAT and PT portions must be completed by student and all hand-scoring completed by teacher/proctorIABScores generated per blockIf the block includes constructed response items, hand-scoring must be completed to generate a scoreReminder – machine scoring happens immediately. Hand-scoring connected to either the ICA or IAB must be completed to generate the student’s score.
33 ICA Score ReportHome Example State Bay View School District Bay View Central High Grade 11For the ICA, each student’s score will be displayed with an overall number, and as a point on a scale of 1-4.Level 1- Below Basic (red)Level 2- Basic (yellow)Level 3- Proficient (green)Level 4- Advanced (blue)Within the Smarter Balanced Digital Library- there are score report modules available that explain how to read score reports based on the user: parents, teachers and administrators.
34 IAB Score ReportFor the IAB, students will be given one of three descriptions since the blocks do not cover a range of ELA standards:Below StandardAt or Near Standard (falls within 1 standard error of the Level 2-to-3 cut score for that block)Above Standard*This screenshot shows what scores will look like for Washington state. Individual teachers will see this by student depending on which blocks have been completed.
35 Final Thoughts The interim assessments are a valuable tool to: Prepare teachers and students for successful testing.Guide instruction aligned to standards based on student needs.