Webcast May 22, 2018 Presented by Educational Testing Service (ETS)

Webcast May 22, 2018 Presented by Educational Testing Service (ETS)
Post-Test Webcast: Principles of Scoring and Reporting Accessing Scores, Reports, Resources, and Tools Webcast May 22, 2018 Presented by Educational Testing Service (ETS)

Audience Participation: Poll Everywhere
When poll is active, respond at or 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

In-Person 2017–18 Post-Test Workshops
The Results are In—Now What? Analyzing Evidence to Inform Teaching and Learning May 23, 2018: Sacramento County Office of Education May 24, 2018: Shasta County Office of Education May 30, 2018: University of California, Irvine May 31, 2018: Los Angeles County Office of Education June 5, 2018: Santa Clara County Office of Education June 7, 2018: Fresno City College June 11, 2018: Ventura County Office of Education June 13, 2018: San Diego County Office of Education 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Agenda Agenda Item What’s New The Process of Scoring
Understanding Reports Test Results in the Test Operations Management System (TOMS) Understanding Your Student Score Report (SSR) 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

What’s New 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

What’s New (1) Electronic reporting
California Alternate Assessment (CAA) SSRs to include three years of score data Reporting of CAA for English language arts/literacy (ELA) and mathematics in August 2018 Local educational agency (LEA) bulk downloads of SSRs 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

What’s New (2) Updates to the Online Reporting System (ORS)
Changes to demographic student groups Target report calculations IDEA Indicator and LEP Status fields renamed to Disability Status and EL Status respectively LEA Student Data File Layout to include additional fields for the CAA for Science and California Science Test (CAST) 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

What’s New (3) 2017–18 California Assessment of Student Performance and Progress (CAASPP) Student Data File Layout will be available soon on CAASPP.org The Public Web Reporting site to include three years of score data for the CAA for ELA and mathematics Student Data File Layout to include the CAST and CAA for Science 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

What’s New (4) The 2016–17 and 2017–18 individual Smarter Balanced ELA Writing Extended Response (WER) scores are now available in ORS. View multiple students’ WER scores using the student data report in ORS. WER scores may not be shown in ORS. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Live Poll Question Does your LEA use a parent portal as part of your student information system (SIS)? 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

The Process of Scoring 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Demographic Data Test Assignments, Test Settings, and
Data Flow California Longitudinal Pupil Achievement Data System (CALPADS) TOMS: Test Assignments and Settings Test Delivery System ORS Scoring System Student Demographic Data Test Assignments, Test Settings, and Basic Student Data Student Responses Test Results TOMS (Reporting) 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CALPADS Reminders Confirm that your student enrollment and demographic data are up-to-date in CALPADS and TOMS prior to the end of your LEA’s selected testing window. Students who will be returning to the same school the following year should not be re- enrolled in the next grade before the LEA’s selected testing window closes. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Goals of This Section Provide an overview of computer adaptive testing (CAT) and scoring. Describe the relative contribution of the performance tasks (PTs) and CAT to the overall scores. Describe: Score scale Achievement levels Claims Assessment targets 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

An Illustration of Adaptivity (1)
There are several real-world examples in which industries deploy technology to cater to our unique tastes. For example: Search engines using cookies to recommend products and services we might like Smart-home software that learns from our daily living patterns to heat and cool our homes efficiently Music applications that allow us to create our own unique music stations based on responding to several presented music selections 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

An Illustration of Adaptivity (2)
The notion of adaptivity extends to the assessment world. We can obtain a precise estimate of an individual’s proficiency or achievement based on his or her responses to a sequence of test questions. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

A Comparison: Customized Music Service versus a CAT (1)
XXYYZZ Music Service Computer application that allows users to customize their music around their unique taste CAT Test administered on the computer that presents test items for an assessed content area that is at the right difficulty level for an individual test taker 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

XXYYZZ Music Service Depends on having large music library CAT Depends on having large item bank Item Bank 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

XXYYZZ Music Service To arrive at your own unique music taste: Music service presents you with a variety of music selections. CAT To arrive at a test taker’s estimate of proficiency or achievement: The CAT will present the test taker with a collection of test items aligned to the content area being assessed. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

XXYYZZ Music Service To arrive at your own unique music taste: You can respond positively “” if you like the selection or negatively “” if you dislike the selection. CAT To arrive at a test taker’s estimate of proficiency or achievement: Test takers provide responses to items where “R” indicates where the test taker answered the item correctly or “W” indicates the test taker answered the item incorrectly. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

XXYYZZ Music Service To arrive at your own unique music taste: Tracks your feedback on proposed music selections over an extended period of time        …….. CAT To arrive at a test taker’s estimate of proficiency or achievement: Tracks responses to test items at various levels of difficulty to “hone in” on true proficiency or achievement RRWRWWWWRR…… 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

XXYYZZ Music Service To arrive at your own unique music taste: The more information you provide, the better the estimate of your true music taste CAT To arrive at a test taker’s estimate of proficiency or achievement: The more items a test taker responds to, the better the estimate of true proficiency or achievement 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

XXYYZZ Music Service Final estimate of unique music taste Can be an ongoing process but can end when the listener stops providing responses to music selections CAT Final estimate of proficiency or achievement Based on test taker completing a sufficient number of test items associated with a test blueprint that is aligned to the content area being assessed 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

How Does a CAT Work in Practice? Example: A Student of Average Ability
Medium Med–Low Low Very Low Med–High High Very High Test Questions Answers (right/wrong) 1 2 3 4 5 6 7 8 9 10 R R W R W W W W R R Expanded 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAT: Practical Considerations (1)
Each student’s test is constrained to ensure coverage of the full range of appropriate grade- level content (e.g., the ELA test cannot consist of only reading informational items). The level of test-question exposure is constrained to maintain test security. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAT: Practical Considerations (2)
Sets of test questions based on a common passage or stimulus constrain the ability to adapt within the set. The test-question pools for a particular grade level are designed to include an enhanced pool of test questions that are more or less difficult for that grade but still match the grade’s test blueprint. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAT Scoring (1) As a student progresses through the test, his or her pattern of responses is tracked. Revised estimates of the student’s ability are calculated in real time. Successive test questions are selected to increase the precision of the student’s level-of- achievement estimate, given the current understanding of his or her ability. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAT Scoring (2) The adaptive test will select these items based on the student’s performance on prior items. Scores from the CAT portion of the test are based on the specific test questions selected as a result of the student’s responses. Scores are NOT based on the sum of the number of questions answered correctly. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Human-Scored Items in the CAT
Some items administered on the Smarter Balanced CAT component require human scoring. Since these items cannot be scored in real time by a human, the student’s performance on these items will not impact subsequent item selection. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Performance Tasks (PTs)
In all Smarter Balanced tests, a PT and a set of stimuli on a given topic are administered as well as the CAT. PTs are not adaptive and, therefore, are not targeted to a student’s specific ability level. The items associated with the PTs may be scored by a machine or by human raters. For more information on the number of machine- scored and human-scored items, please refer to the Smarter Balanced blueprints— elaliteracy-summative-assessment-blueprint.pdf 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Final Score (1) For each student, the responses from the PT and CAT portions are merged for final scoring. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Final Score (2) The resulting estimates of achievement are based on the specific test questions that a student answered and not the total number of items answered correctly. Higher estimates of achievement are associated with test takers who correctly answer difficult and more discriminating items. Lower estimates of achievement are associated with test takers who correctly answer easier and less discriminating items. It is possible for students to have the same estimate of achievement through different response patterns. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Final Scoring: Contribution of the CAT and PT Sections (1)
Number of Items Defined by Test Blueprints English Language Arts Mathematics Grade CAT PT 3–5 38–41 4–6 6–8 37–42 11 39–41 Grade CAT PT 3–5 31–34 2–6 6–8 30–34 11 33–36 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Relative contribution of the PT and CAT sections on the total score will vary across students since the overall score is based on pattern scoring; depends on performance on the CAT section; depends on performance on the PT section; and depends on the average difficulty of the PT section. Example: A PT might have a significant negative contribution on a student’s overall score if he or she did extremely well on the CAT but performed poorly on a very easy PT section. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

The guidance provided by Smarter Balanced is that the PT section could account for up to 25 percent of a student’s overall score. Upshot: All parts of the Smarter Balanced assessments are important. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Available Summative Assessment Scores
Scale scores Achievement levels Not met, nearly met, met, exceeded Claim levels Below, near, above Target scores 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Properties of the Reporting Scale
Scores are on a vertical scale. Expressed on a single continuum for a content area Allows users to describe student growth over time across grade levels Scale score ranges ELA: 2114–2795 Mathematics: 2189–2862 For each grade level and content area, there is a separate scale score range. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Smarter Balanced Scale Score Ranges and Levels for ELA
Grade Level 1 Standard Not Met Level 2 Standard Nearly Met Level 3 Standard Met Level 4 Standard Exceeded 3 2114–2366 2367–2431 2432–2489 2490–2623 4 2131–2415 2416–2472 2473–2532 2533–2663 5 2201–2441 2442–2501 2502–2581 2582–2701 6 2210–2456 2457–2530 2531–2617 2618–2724 7 2258–2478 2479–2551 2552–2648 2649–2745 8 2288–2486 2487–2566 2567–2667 2668–2769 11 2299–2492 2493–2582 2583–2681 2682–2795 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Threshold Scale Scores by Grade (1)
2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Smarter Balanced Scale Score Ranges and Levels for Mathematics
Grade Level 1 Standard Not Met Level 2 Standard Nearly Met Level 3 Standard Met Level 4 Standard Exceeded 3 2189–2380 2381–2435 2436–2500 2501–2621 4 2204–2410 2411–2484 2485–2548 2549–2659 5 2219–2454 2455–2527 2528–2578 2579–2700 6 2235–2472 2473–2551 2552–2609 2610–2748 7 2250–2483 2484–2566 2567–2634 2635–2778 8 2265–2503 2504–2585 2586–2652 2653–2802 11 2280–2542 2543–2627 2628–2717 2718–2862 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Threshold Scale Scores by Grade (2)

Claim Areas ELA Reading Writing Listening Research/Inquiry Mathematics
Concepts and Procedures Problem Solving/Modeling and Data Analysis Communicating Reasoning 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Claim Scores Reported at an individual and aggregate level
Reported for all claims in ELA and mathematics 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Achievement Levels for Claims
Achievement levels for claims are very similar to subscores. They provide supplemental information regarding a student’s strengths or weaknesses. A student must complete all items within a claim to receive an estimate of his or her performance on a claim. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning Copyright © 2009 Educational Testing Service.

Why Are There Only Three Achievement Levels of Claims?
Three achievement levels for claims were developed. Achievement levels for claims are based on the distance a student’s performance on the claim is from the Level 3 “Standard Met” criterion. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Assessment Targets Assessment targets connect the content standards to evidence that will be collected from the assessment. Targets map the Common Core State Standards (CCSS) onto assessment evidence that is required to support the content categories and claims. Targets are used to guide the development of items and tasks that will measure the CCSS. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Assessment Target Reports (1)
Assessment targets are group-level indicators (e.g., for a classroom). Target reports in the ORS indicate group- level strengths or areas for improvement for specific standards that are addressed in each claim in the ELA and mathematics Concepts and Procedures claim. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning Copyright © 2009 Educational Testing Service.

Two target reports available Strength/Weakness indicator Summarizes how a group of students performed on each target relative to overall test performance Areas where performance indicates Standard Met Summarizes whether a group of students’ performance on a each target is above or below the Standard Met criteria 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Example: Mathematics Target Report Assessment Target Reports Quick Start Guide available at: 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding the Reports

Available Reports  Report LEA School Parent/ Guardian
Initial Student Test Results  ORS Partial Aggregate Data Partial Assessment Target Reports SSRs PDF/Paper TOMS†/Paper†† TOMS†/Paper Paper Final Student Data File TOMS † PDFs of the SSRs will be available in TOMS. †† LEAs must provide the copy of the CAASPP SSR to each student’s parent/guardian within 20 working days of its delivery to the LEA or, if received after the last day of instruction, within the first 20 days of the next school year. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Example 2017–18 Timeline 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Using the ORS 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Enhancements to the ORS
For historical and new scores, we have updated the disaggregation labels and data (new and removed) of aggregate data in the ORS. Beginning with 2017–18 data, we added a new target report to provide aggregate performance data on Standard Met (Level 3). We revised the target report calculations to expand the reporting of weaknesses and strengths for schools and districts. LEAs should see less “insufficient information.” 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Partial Results Available Now in the ORS
Test results are available three weeks after a student completes both parts—CAT and PT— of a content area. Test results are added nightly. Use Caution: The results available in this reporting system are partial and may not be a good representation of your school’s or LEA’s final aggregate results. These partial results are not appropriate for public release. Final data will be released publicly by the California Department of Education (CDE) in September 2018. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

ORS Summary The ORS is a Web-based system that displays score reports for each student who has taken the following California assessments: Smarter Balanced for ELA and mathematics CAAs for ELA and mathematics Standards-based Tests in Spanish (STS) for reading/language arts (RLA) Note: All score report data, except for an individual student’s score reports, can be disaggregated into student groups for detailed analysis. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Features and Reports in the ORS
Home Page Dashboard Subject Detail Report Claim-Level Detail Report Assessment Target Report Student Listing Report Student Detail Report Manage Rosters The ORS Score Reports Quick Start Guide is available at: ORS-quick-start-guide.2016.pdf 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Users with Access to the ORS
User Role ORS Access Level LEA CAASPP coordinators All students within the LEA CAASPP test site coordinators All students within their school or entity Test administrators Test examiners Test administrators/test examiners Students assigned to them via a roster assignment by their CAASPP test site coordinator Educators/Teachers For additional information about user roles available within TOMS, refer to the 2016–17 User Role Guidance document on the CAASPP Portal. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Managing Rosters 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Manage Rosters Create customized groupings of students within a school. Example: School-level users can create a report that lists all of the students within a specific grade or in a particular classroom. Use in combination with user roles (e.g., test administrator, test examiner, test administrator/ test examiner, educators) to provide teachers with access to test results for their students. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Manage Rosters: Print, Modify, and Delete

Using Reports in TOMS 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Features and Reports in TOMS
Student Score Reports (PDFs) LEA User List Report Student Demographics Reports LEA Coordinator Report Test Assignment Reports Student Results Report—Student Score Data Extract Unlisted Resources Report Percentage Report 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Test Results in TOMS CAASPP Portal: http://www.caaspp.org/

TOMS Home Screen TOMS Downloading Student Score Report PDF Files Quick Start Guide available at: 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Download Student Score Report PDFs
Select year Select the LEA, school and grade of the reports you want to compile and name the report download file. To select contiguous grades (e.g., grades three–six), hold down the [Shift] key as you select each grade. To select specific noncontiguous grades (e.g., grade three, grade four, grade six), hold down the [CRTL] key as you select each grade. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Score Report PDF Download File
LEA CDS Code School CDS Code Grade SSID Date stamp A compressed file (.zip) will be generated. Large files may take time to process. Each PDF contains the SSR for one student. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Score Report PDF Download File
School Name School Code Grade Last Name First Name Middle Initial State Student Id Subject Code PDF file Demo School 3 LAST01 FIRST01 A ELA,MA ETSCAASPP_2_ _ _03_ _ PDF LAST02 FIRST02 B ETSCAASPP_2_ _ _03_ _ PDF LAST03 FIRST03 C ETSCAASPP_2_ _ _03_ _ PDF LAST04 FIRST04 D ETSCAASPP_2_ _ _03_ _ PDF LAST05 FIRST05 E ETSCAASPP_2_ _ _03_ _ PDF LAST06 FIRST06 F ETSCAASPP_2_ _ _03_ _ PDF Hyperlinks to the individual Student Score Reports 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report

Understanding Your SSR (1)
Student Score Report Versions Assessment Grades Smarter Balanced for ELA and mathematics 3, 4, 6, 7, and 11 Smarter Balanced for ELA, mathematics, and science (CAST field test) 5, 8, and 11 CAAs for ELA and mathematics CAAs for ELA, mathematics, and science (CAA for Science pilot year two) STS for RLA 2–11 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Score Report Types Initial—Electronic Final—Electronic and printed Additional SSRs Final 10 percent after original 90 percent are shipped Electronic and printed One report for each LEA Multiple administrations combined as well as all CAA student scores remaining Any re-reported students 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your SSR Guide Versions Available (3)
Language versions: English Spanish Armenian Chinese (simplified) Chinese (traditional) Hmong Korean Pilipino (Tagalog) Vietnamese 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Interpreting Score Change (1)
As previously highlighted, the Smarter Balanced Student Score Report tracks student progress over time. The CDE has produced tables summarizing typical changes in scale scores from one grade level to the next grade level. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Average Scale Score Change from Grade Four Mathematics to Grade Five Mathematics Scale Score Grade Four Average Scale Score Change Number of Students 2204–2299 62.09 9,764 2300–2399 26.11 99,264 2400–2499 21.81 203,945 2500–2599 29.18 122,674 2600–2659 22.22 24,251 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Example: Matthew’s scores on the Smarter Balanced mathematics test increased from 2643 in grade four to 2665 in grade five, for a scale score change of 22 points. Matthew’s prior year’s scale score was in the highest performance grouping, between 2600 and 2659. Students whose prior year’s scores were in this range had an average score change of points. Matthew’s score change was typical. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAST and CAA for Science Reporting for 2017–18

CAST Preliminary indicators Parent/Guardian letter templates
Grade ten and eleven results saved CAST field test scores and performance levels to be in LEA downloadable files in fall 2018 Aggregate data 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAA for Science Parent/Guardian letter templates
Grade ten and eleven results saved Aggregate data Preliminary indicators Determined locally using Total Score Conversion Tables Scores calculated and distributed in fall 2018 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAST and CAA for Science Reporting for 2017–18 (1)
A percent of items a student answered correctly An indicator category with text describing the student’s performance Preliminary indicator categories: Category Description 3 Student performance suggests a considerable understanding of the California Next Generation Science Standards. 2 Student performance suggests a moderate understanding of the California Next Generation Science Standards. 1 Student performance suggests a limited understanding of the California Next Generation Science Standards. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Preliminary Indicator Communication Toolkit resources include: Preliminary Indicators—Key Messages Reporting Science Assessments Results— 2017–18 Timeline Science Assessments—Transition to Operational Status Frequently Asked Questions 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

For more information on the reporting of CAST and CAA for Science preliminary indicators, access the Communication Toolkit at 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your SSR: CAST and CAA for Science
Elements of the Student Score Report—Back Page 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Other Notable Reports for 2017–18

SSR in Spanish 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Requesting a Spanish SSR
Request through TOMS. Select the [Score Status] tab on the Student Profile page. Select the check box for “Request for Spanish Student Score Report.” TOMS Spanish-Language Student Score Report Quick Start Guide available at: 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Upload Spanish Score Report Requests

Timeline for Reporting

When Will the Initial SSRs for CAASPP Appear in TOMS?
Initial SSRs will be provided as downloadable PDF files in TOMS one week after the student’s last score is reported in the ORS. Initial SSRs are created during the test administration window as the student completes the last assigned test. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Avoiding Scoring Delays for the Smarter Balanced Assessments (1)
ETS may place a temporary hold on scoring a student’s results to investigate and/or clarify information provided. Situations that would lead to a hold include, but are not limited to: Some students may have multiple Statewide Student Identifiers (SSIDs). A student answer document was returned incomplete. Notification came from CALPADS of a retired SSID, and the student has tested. SSID swap was made. Rescore request was made. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Avoiding Scoring Delays for the Smarter Balanced Assessments (2)
If a temporary hold is placed on a student’s score, it will delay the reporting of that student’s results in the ORS as well as the SSRs and data files. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

LEA Dashboard in TOMS 1 2 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Information on the LEA Dashboard
1 Progress of scoring Estimate of when to expect delivery of paper SSRs Time stamp of last update 2 Alert at 80 percent—Review and update student demographic data in CALPADS. Alert at 90 percent—Final SSRs will be processed and shipped. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Scoring Progress Dashboard (1)
TOMS includes a dashboard displaying the progress of scoring for your LEA’s administration. When 80 percent of student scores are complete for the LEA’s administration, the LEA CAASPP coordinator will receive a notification to carry out the following: Complete testing for students with processed appeals. Verify condition codes and enter any remaining condition codes. Request Spanish SSRs. Select parent/guardian address (if desired). 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Scoring Progress Dashboard (2)
When 90 percent of student scores are complete, the final SSRs will be printed, packaged, and delivered to the LEA. At the end of scoring, all remaining final SSRs will be delivered in a separate shipment. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Score Report Resources

Score Report Resources
Student Score Report Resources Estimated Availability Location Quick Start Guides Now LEA Student Data File and supporting materials Summer 2018 TOMS, CAASPP Portal, Webcast* CAASPP Post-Test Guide CAASPP Portal, Webcast* Public Web reporting September 2018 CDE Web site, Webcast** * Information about the LEA student data file and the Post-Test Guide will be provided during the same Webcast. ** Information about these resources will be provided in separate Webcasts to be timed to coincide with the release of the resource. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Results for the CAAs for ELA and Mathematics

Scores Reported The scores reported include
scale scores for ELA and mathematics; and achievement levels for ELA and mathematics. No subscores or claim scores are reported. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Scale Scores (1) “Scaling” is the statistical process that transforms scores from one set of numbers to another that allows scores to be more interpretable as well as comparable in some way. A scaled score DOES NOT mean “percent correct.” A scaled score DOES NOT mean “raw score.” 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Scale Scores (2) The reporting scale for the CAAs for ELA and mathematics is a three-digit scale, where the leading digit corresponds to the grade level, and the last two digits range from 00 to 99. Examples: Scale for grade five ELA ranges from 500 to 599. Scale for grade eleven mathematics ranges from 900 to 999. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Achievement Levels Scaled scores are mapped into three achievement levels: Level 1 Alternate—Limited Understanding Level 2 Alternate—Foundational Understanding Level 3 Alternate—Understanding 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Achievement Level Descriptors
Description 3 Students at this level demonstrate understanding of core subject matter in the content area. They are actively working with adapted grade-level content that focuses on the essential knowledge and skills and may need occasional prompts and assistance to complete tasks and activities. 2 Students at this level demonstrate foundational understanding of core subject matter in the content area when provided with frequent prompts and supports. They are actively working with adapted grade-level content that focuses on the essential knowledge and skills and may frequently need supports to complete tasks and activities. 1 Students at this level demonstrate limited understanding of adapted grade-level content that focuses on much of the basic knowledge and skills, even with extensive supports. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAAs for ELA and Mathematics: Scale Score Ranges by Grade
Level 1 Alternate Level 2 Level 3 3 300–344 345–359 360–399 4 400–444 445–459 460–499 5 500–544 545–559 560–599 6 600–644 645–659 660–699 7 700–744 745–759 760–799 8 800–844 845–859 860–899 11 900–944 945–959 960–999 Scaled score ranges are the same for both the ELA and mathematics CAA assessments. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding How Scale Scores Are Assigned
Students who did not respond or orient to the test will receive a Lowest Obtainable Scaled Score ([LOSS] e.g., 300, 400, 500, and so on). Students who responded to fewer than four questions will receive a “LOSS + 1” (e.g., 301, 401, , 901). Students responding to four or more questions could receive any score above “LOSS + 1” (e.g., 302–399, 402–499, , 902–999). 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

How to Use CAA Results (1)
Scaled scores for the CAA can be compared within grade levels and from one year to another. For example: Mean scaled scores for grade five ELA in School A may be compared with grade five ELA in School B. Mean scaled scores for grade seven mathematics in LEA A may be compared with grade seven mathematics in California. Mean scaled scores for grade three ELA in LEA “A” in may be compared with grade three ELA in LEA “A” in 2017 and 2016. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

How to Use CAA Results (2)
Scaled scores for the CAA should not be compared across different content areas and grades. There is no vertical scale for CAA scores. Therefore, you should not compute differences in scaled scores from one year to the next for the same student or group of students that move from one grade to the next. Scaled scores for the CAA should not be compared to different assessments such as: Smarter Balanced assessments California Alternate Performance Assessment 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAASPP Portal http://www.caaspp.org/

CAA Scoring and Reporting
Scores will be available in the ORS by mid-July. SSRs for the CAAs will be distributed to LEAs in August. Data files for the CAAs will be available in September. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your CAA Student Score Report

Electronic Reporting for 2018–19

Does your LEA use parent portal as part of your student information system (SIS)?

Audience Participation
When poll is active, respond at or 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Electronic Reporting Electronic reporting of SSRs begins in the 2018–19 administration. Electronic reporting communication timeline PDF SSRs will be downloadable through TOMS. PDFs can be made available to student information systems for parents. PDFs are printable. 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Reflect What word first comes to mind as you think about transitioning to electronic reporting? 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Questions and Answers 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

What word first comes to mind as you think about transitioning to electronic reporting?

Help Desk Support The California Technical Assistance Center is here to support all LEA CAASPP coordinators! Monday–Friday from 7 a.m.–5 p.m. PT Phone: Web site: 2017–18 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Webcast May 22, 2018 Presented by Educational Testing Service (ETS)

Similar presentations

Presentation on theme: "Webcast May 22, 2018 Presented by Educational Testing Service (ETS)"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Webcast May 22, 2018 Presented by Educational Testing Service (ETS)

Similar presentations

Presentation on theme: "Webcast May 22, 2018 Presented by Educational Testing Service (ETS)"— Presentation transcript:

Similar presentations

About project

Feedback