1. California Assessment of Student Performance and Progress (CAASPP)
Introduction to the
2015 CAASPP Online Reporting System (ORS)
1 – 2 p.m.
May 4, 2015

2. Purpose
To provide an introduction to the Online Reporting System (ORS), which is the reporting system local educational agencies (LEA) will use to view preliminary student results on the following CAASPP assessments:
Smarter Balanced Summative Assessments in English language arts/literacy (ELA) and mathematics
California Standards Test (CST) for Science
California Modified Assessment (CMA) for Science
California Alternate Performance Assessment (CAPA) for Science
Standards-based Tests for Spanish (STS)

3. Agenda Overview Accessing the ORS Understanding the ORS Interface
Viewing Score Reports in the ORS
Live Demonstration
Questions
Updates and Announcements

4. Overview

5. What is the Online Reporting System?
The ORS is a web-based system that displays preliminary score reports and participation data for each student who takes part in the CAASPP assessments.

6. Why Online Reporting?
Timely data is more useful for affecting learning and instruction.
Move to computer-based assessments facilitates faster reporting.
Policymakers made timeliness a priority
(Education Code 60641(b)).

7. Important Reminder
The data available in the CAASPP ORS represents partial and preliminary results that are not appropriate for public release. As a real-time system, results will change as additional data is received and relevant appeals and rescores are processed. These changes may result in final scores being higher or lower than the preliminary results posted to this system. The California Department of Education (CDE) recommends that data from the ORS only be released publically following the state level release of assessment data that occurs in August.

8. Process for Release of Preliminary Test Results
Hand score
Machine score
AI Score
Score
Confirm accuracy by grade and content area
Check
Preliminary scores available in the ORS
Report

9. Reporting Timeline for Online Smarter Balanced Summative Tests
Week 0:
Student completes a content area
Weeks 1-3:
Student responses are scored and merged; preliminary results are checked
Week 4:
LEA accesses ORS to view preliminary results

10. Why Preliminary Results?
Test administrations may still be in progress
Factors that could change results
Security breaches
Problematic test questions
Appeals processed and test rescores

11. Preliminary Test Results
Provides information to support teachers in:
Focusing instruction
Improving learning
Preliminary results include:
Student and aggregate results
Overall scores reported within one of four achievement levels
Achievement Levels
LEVEL 4
Exceeded the Standard 3
Met the Standard 2
Nearly Met the Standard 1
Has Not Met the Standard

12. Estimated Timeline for Release of Preliminary Results
Grade and Content Area
Estimated Date
Smarter Balanced Grade 3 mathematics
Smarter Balanced Grade 5 mathematics
Smarter Balanced Grade 7 ELA
May 3 (today) for students completed testing on or around April 1
Smarter Balanced Grade 6 ELA and mathematics
Smarter Balanced Grade 7 mathematics
Smarter Balanced Grade 8 ELA and mathematics
TBD
Smarter Balanced Grade 3 ELA
Smarter Balanced Grade 4 ELA
Smarter Balanced Grade 5 ELA
Smarter Balanced Grade 4 mathematics
Smarter Balanced Grade 11 ELA and mathematics
CST/CMA/CAPA Science Grades 5, 8 and 10
STS RLA Grades 2-11

13. Reporting Timeline for Final Results
Week 0:
Student completes a content area
Weeks 1-3:
Student responses are scored and merged; preliminary results are checked
Week 4:
LEA accesses ORS to view preliminary results
4 Weeks After Test Administration Window Closes:
LEAs accesses and downloads final student data file from TOMS

14. Examples of Recommended Uses
Use preliminary overall scores and claim information to focus instruction
Combine with other academic information about the student to identify learning opportunities

15. Not Recommended
Adding these results to your local student information system
Presenting these results as final scores
Using these results as the sole basis for determining a student’s learning opportunities
Comparing these results to last year’s STAR results

16. Accessing the ORS

17. User Roles and Access in the ORS
ORS operates in a secure, role-based structure
Access to reports and data in ORS depends on your user role in the Test Operations Management System (TOMS)
Example: LEA-level users can view data for all schools and students in their LEA; school-level users can view data only for students in their school.

18. User Roles and Access in the ORS (cont.)
Access Level
LEA CAASPP Coordinator
Test Site Coordinator (SC)
Test Administrator (TA)
School Listing 
Student Listing
Preliminary Student Results Report
Summary Statistics
Retrieve Student Results
Plan and Manage Testing
Test Completion Rates
Search Students

19. Accessing the ORS
Note: To log onto ORS, users must have an authorized TOMS username and password.
Select the [Online Reporting System] button on the CAASPP portal at
Log onto ORS using your
TOMS logon credentials.

20. Accessing the ORS
Upon logging on, the Welcome page will appear.

21. Accessing the ORS
Note: ORS contains students’ personally identifiable information (PII). For security purposes, it is important that you log out to ensure that the reports cannot be accessed by unauthorized users following your session.
ORS has a timeout feature that automatically logs you out if you have been inactive for 30 minutes.

22. Understanding the ORS Interface

23. Understanding the ORS Interface
Two primary features of ORS:
Test Management Center (Completion Status System)
Score Reports

24. ORS Welcome Page Takes you to the Completion Status System
Takes you preliminary student test scores

25. Test Management Center
Same as the Completion Status System
Provides participation data for students taking tests
Can be used to determine which students need to complete and/or start testing
Can be used to view a summary statistics report and download student data files
A detailed overview of accessing and using student participation reports is provided in the Completion Status User Guide

26. Score Reports Provide score data for each administered test
Compare score data between individual students and the school, LEA, or overall state average scores
Provide information about performance on claims

27. ORS Banner Completion Status Reports Preliminary student test results
Opens a pop-up window where you can search for a student and access his/her preliminary scores
Completion Status Reports
Preliminary student test results
Displays Retrieve Student Results page, where you can access student performance data files
Links to the ORS User Guide (forthcoming)
Exports the data displayed on the page in a .xls file

28. Viewing Score Reports in the ORS

29. Viewing ORS Reports
From the Select drop-down list, select the LEA or school whose reports you want to view.
A list will appear only if you are associated with more than one school or LEA
Select [Score Reports].
Note: All score reports data, except for individual students’ score reports, can be disaggregated into subgroups for detailed analysis. For example, you can view a Grade 5 Mathematics report for a for an entire school or for a LEA.

30. Available Preliminary Score Reports in the ORS
District Level
School Level
Student Level
Home Page Dashboard
Summary of performance (to date) across grades and subjects or courses for the current administration. 
Subject Detail
Data for a subject within a particular grade or course for the current administration.
Claim Level Detail (Smarter Balanced)
Claim/Block-level percentage at each performance level for a subject within a particular grade or course for the current administration.
Student Listing
Roster of all students who belong to a school with their associated subject or course (and claim, if applicable) scores for the current administration.
Student Detail
Detailed information about a selected student’s performance in a specified subject or course (and claim, if applicable).

31. Home Page Dashboard Report
Displays the overall summary of score data and testing progress for your LEA and/or school
Starting point for data analysis
Define the students whose aggregated scores you want to view
Navigate to more detailed score reports from the Home Page Dashboard
Note: The score data you see are dependent on your role

32. Home Page Dashboard Report (cont.)

33. Subject Detail Report
The aggregation tables on the Home Page Dashboard display score data for students by grade and subject and provide access to Subject Detail Reports.

34. Subject Detail Report (cont.)

35. Subject Detail Report (cont.)
Each Subject Detail Report consists of the following components:
Report descriptor
Test name (subject and grade)
Administration year
Entity (e.g., LEA, school)
The title of the score report table
Name, number of students, average scale scores, percent in each achievement level
All data are based on the total number of students who have taken and completed the test and submitted it for scoring

36. Claim Level Detail Report2 1

37. Claim Level Detail Report (cont.)

38. Claim Level Detail Report (cont.)
The Claim Level Detail Report consists of the following components:
Report name
[Entity] Performance for Each Claim. What are my [entity’s] strengths and weaknesses in [Subject or Course]?
Test name (subject and grade)
Administration year
Entity (e.g., LEA, school, or roster)
The title of the score report table
Name, number of students, average scale score, claims, and percentage in each claims achievement level

39. Student Listing Report2 1

40. Student Listing Report (cont.)

41. Student Listing Report (cont.)
Students’ SSIDs are displayed
Scores for individual students are not mean scale scores
Different procedure for viewing scale score data by demographic subgroup
Print on the Student Listing Report prints the current page and also generates a PDF file of individual preliminary student results for all the students in the roster

42. Student Detail Report 2 1

43. Student Detail Report (cont.)

44. Student Detail Report (cont.)
Displays the breakdown of the student’s preliminary scale score, achievement level for the selected subject, and performance and claim description for each claim
Includes average scale scores for the state and LEA for comparison purposes
Note: State-level scale score averages will not be available until formally released by the CDE.

45. System Enhancements under Consideration
Target reports
Creating roster groups

46. Live Demonstration

47. Questions?

48. Submitting Feedback on the ORS
New systems require patience, we appreciate your understanding and welcome your feedback:

49. Updates and Announcements

50. Updates and Announcements
The Online Reporting System User Guide is forthcoming.
Next Webcast: 2015 CAASPP Post-Test Workshop
Date: Wednesday, May 20, 2015
Time: 1–3:30 p.m.

51. Resources and Support

52. Help Desk Support
The California Technical Assistance Center (CalTAC) is here to support all LEA CAASPP coordinators!
Monday–Friday from 7 a.m.–5 p.m. PT
Phone:
Web site: