1. California Assessment of Student Performance and Progress (CAASPP) 2015 Online Test Administration Site Coordinator’s Training March 3, 2015

2. 2015 CAASPP Online Test Administration Workshop 2 Agenda Topic Overview Roles and Responsibilities Using the Test Operations Management System (TOMS) Test Administration Test Security Practice Tests and Training Tests Questions

3. Overview

4. 2015 CAASPP Online Test Administration Workshop 4 Student Participation All students in grades 3–8 are required to take the SBAC summative assessments. The only students exempt from participating are: −Students participating in the California Alternate Assessments (CAA) in English language arts/literacy (ELA) and mathematics −English learners (ELs) who have been attending school in the United States for less than 12 months (for the ELA test only)

5. 2015 CAASPP Online Test Administration Workshop 5 About the Summative Assessments Two content areas: –English language arts/literacy (ELA) –Mathematics Two components per content area: –Computer adaptive test (CAT) –Performance task (PT)

6. 2015 CAASPP Online Test Administration Workshop 6 About the Summative Assessments (cont.) Q:What is the CAT component of the online summative assessment? A:The computer adaptive portion of the online test includes a wide variety of item types. As students progress throughout a CAT, the computer program adjusts the difficulty of questions throughout the assessment based on their responses. For example, a student who answers a question correctly will receive a more challenging item, while an incorrect answer generates an easier question.

7. 2015 CAASPP Online Test Administration Workshop 7 About the Summative Assessments (cont.) Q:What is the PT component of the online summative assessment? A:The PT is designed to provide students with an opportunity to demonstrate their ability to apply their knowledge and higher-order thinking skills to explore and analyze a complex, real-world scenario. Prior to the PT, teachers or instructional staff conduct a Classroom Activity for all students to ensure that a lack of understanding of the context of the task does not interfere with a student’s ability to address the content of the task.

8. 2015 CAASPP Online Test Administration Workshop 8 Systems and Processes for Online Administration Test Administrator Interface & Student Testing Site Administer online Smarter Balanced assessments Test Administrator Interface & Student Testing Site Administer online Smarter Balanced assessments TOMS Set up test administration windows Verify student demographic data Configure online test settings (designated supports and accommodations) TOMS Set up test administration windows Verify student demographic data Configure online test settings (designated supports and accommodations) CALPADS (California Longitudinal Pupil Achievement Data System) Manage student demographic data CALPADS (California Longitudinal Pupil Achievement Data System) Manage student demographic data 123

9. Roles and Responsibilities

10. 2015 CAASPP Online Test Administration Workshop 10 Test Site Coordinator Ensure that Test Administrators have submitted signed CAASPP Test Security Affidavits Ensure TAs are properly trained so they can administer the assessments Enter and/or verify student test settings in TOMS Create or approve testing schedules and procedures for the school

11. 2015 CAASPP Online Test Administration Workshop 11 Test Site Coordinator (cont.) Work with technology staff to ensure that necessary secure browsers are installed and any other technical issues are resolved. Ensure that necessary hardware is available and working (i.e., headphones, mice) Monitor testing progress and ensure that all eligible students participate Address testing problems at the school site Mitigate and report all test security incidents to the LEA CAASPP coordinator Oversee CAASPP administration activities in the school Review Test Site Coordinator Checklist for other activities TAM 84- 88).

12. 2015 CAASPP Online Test Administration Workshop 12 Test Administrator Read and sign the CAASPP Test Security Affidavit Complete training and review all Smarter Balanced policy and administration documents prior to administering any Smarter Balanced assessments Review student information prior to testing to ensure that the correct student receives the proper test with the appropriate supports/test settings Administer the Smarter Balanced assessments Report all potential test security incidents to the SC and LEA CAASPP Coordinator in a manner consistent with Smarter Balanced, state, and LEA policies Review Test Administrator Checklist (TAM 89-94)

13. Using the Test Operations Management System (TOMS)

14. 2015 CAASPP Online Test Administration Workshop 14 Tasks to Complete in TOMS for the Online Administration  TOMS can be accessed from the CAASPP portal at http://caaspp.orghttp://caaspp.org  Review student data −Demographic data from CALPADS −Test enrollment  Assign online test settings to students

15. 2015 CAASPP Online Test Administration Workshop 15 Reviewing Student Data Each student profile in TOMS includes: −Demographic data  Read-only; must be modified in CALPADS −Test enrollment  All students in grades 3–8 are automatically enrolled in the online Smarter Balanced summative assessments −Online test settings  Universal tools, designated supports, and accommodations

16. 2015 CAASPP Online Test Administration Workshop 16 Reviewing Student Data (cont.) Students’ grade levels must be correct in TOMS before testing −Grade levels are only correctable in CALPADS All students who are going to test must appear in TOMS. (Brand new students may not appear for 24/48 hours) If a student’s designated supports and accommodations information is incorrect, it must be updated in TOMS before he or she tests.

17. 2015 CAASPP Online Test Administration Workshop 17 Importance of CALPADS CALPADS is the sole source for student demographic, enrollment and program participation data LEAs cannot update student demographic data in TOMS It is critical for LEAs to update CALPADS regularly Processing time from CALPADS to TOMS is 24–48 hours Student accessibility supports are uploaded and maintained in TOMS, not in CALPADS

18. 2015 CAASPP Online Test Administration Workshop 18 Student Profile in TOMS: Demographics All demographic data is read- only

19. 2015 CAASPP Online Test Administration Workshop 19 Student Profile in TOMS: Test Mode All students in grades 3–8 are automatically enrolled in the online Smarter Balanced summative tests

20. 2015 CAASPP Online Test Administration Workshop 20 Student Profile in TOMS: Test Settings

21. 2015 CAASPP Online Test Administration Workshop 21 Student online test settings must be configured at least 24 hours prior to testing –Allow up to 24 hours from when you configure test settings in TOMS to when the changes appear in the TA and Student Testing Interface LEA CAASPP coordinators and SCs are authorized to set accessibility supports for students in TOMS –Two methods: 1.One-by-one through student profile 2.Template upload Assigning Accessibility Supports in TOMS

22. 2015 CAASPP Online Test Administration Workshop 22 What are Accessibility Supports? Universal Tools: –Available to all; based on student preference and selection (e.g., strikethrough, highlighter) Designated Supports: –Need to be indicated by an educator or guardian (e.g., color contrast, read aloud, glossary, magnification) Accommodations: −Based on individualized education programs (IEPs) or Section 504 plans.

23. 2015 CAASPP Online Test Administration Workshop 23 What are Accessibility Supports? (cont.) The online assessments include embedded and non-embedded accessibility supports. –Embedded: part of the computer administration (e.g., digital notepad, calculator) –Non-embedded: provided outside the computer administration (e.g., scratch paper, multiplication table)

24. 2015 CAASPP Online Test Administration Workshop 24 *Must be activated in the test delivery system. EmbeddedNon-Embedded Designated SupportAccommodationDesignated SupportAccommodation Color Contrast American Sign Language (ELA listening and math items) Bilingual Dictionary (ELA performance task full writes) Abacus MaskingBrailleColor Contrast Alternate Response Options Text-to-Speech (math and ELA items) Closed Captioning (ELA listening items) Color OverlaysCalculator Translated Test Directions (math items) Streamlined InterfaceMagnificationMultiplication Table Translations (glossaries) (math items) Text-to-speech (ELA reading passages) Noise BuffersPrint on Demand Translations (stacked) (for math items)* Read Aloud (math and ELA items) Read Aloud (ELA reading passages) Turn off any universal tools* Scribe (ELA non-writing and math items) Scribe Permissive ModeSeparate SettingSpeech-to-Text Print SizeTranslated Test DirectionsIndividualized Aids Translation (glossaries) (math items) Special Lighting or Acoustics Translated Test Directions (math items) What are Accessibility Supports? (cont.)

25. Test Administration

26. 2015 CAASPP Online Test Administration Workshop 26 Scheduling Testing Time Estimated testing times: Content AreaGrades Computer Adaptive Test (CAT) items hrs : mins Performance Task (PT) hrs : mins Total hrs : mins Classroom Activity (administered prior to the PT)* hrs : mins Total hrs : mins English Language Arts/Literacy 3–51:302:003:30:304:00 6–81:302:003:30:304:00 112:00 4:00:304:30 Mathematics 3–51:301:002:30:303:00 6–82:001:003:00:303:30 112:001:303:30:304:00 Both 3–53:00 6:001:007:00 6–83:303:006:301:007:30 114:003:307:301:008:30

27. 2015 CAASPP Online Test Administration Workshop 27 Scheduling Testing Time (cont.) The estimates do not account for any time needed to start computers, load secure browsers, and log in students To help estimate number of days and associated network bandwidth required, schools should access the Smarter Balanced Technology Readiness Calculator at http://www3.cde.ca.gov/sbactechcalc/ http://www3.cde.ca.gov/sbactechcalc/ −Based on number of students, computers, and hours per day computers are available

28. 2015 CAASPP Online Test Administration Workshop 28 Scheduling Testing Time (cont.) Scheduling is a collaborative effort Recommended: CAT first, PTs second –The classroom activity should be completed within 3 days prior to administering a PT  ELA PTs have 2 parts and can be completed over 2 days  Mathematics PTs have 1 part and can be completed in 1 day Recommended: PTs should not be administered on the same day as the CAT CAT Classroom Activity Performance Task

29. 2015 CAASPP Online Test Administration Workshop 29 Example 1: Example 2: Example Testing Schedules MondayTuesdayWednesdayThursdayFriday All Math CAT Math classroom activity Math PT All ELA CAT ELA classroom activity Next MondayNext TuesdayNext WednesdayNext ThursdayNext Friday ELA PT Part 1ELA PT Part 2 MondayTuesdayWednesdayThursdayFriday Math CAT Session 1 Math CAT Session 2 Math PTELA CAT Session 1 ELA CAT Session 2 ELA classroom activity Math classroom activity Next MondayNext TuesdayNext WednesdayNext ThursdayNext Friday ELA PT Part 1ELA PT Part 2 Note: Review Table 10 (TAM p.29) and Table 11 (TAM p. 30) for additional scheduling guidance.

30. 2015 CAASPP Online Test Administration Workshop 30 Establishing Appropriate Testing Conditions Room should have good lighting, ventilation, and freedom from noise and interruptions. Remove or cover displayed information (on bulletin boards, chalkboards, or dry-erase boards, etc.) that might be used by students to help answer questions –Rubrics, vocabulary charts, student work, posters, graphs, charts, etc.

31. 2015 CAASPP Online Test Administration Workshop 31 Establishing Appropriate Testing Conditions (cont.) Make appropriate seating arrangements prior to test administration –Students should be seated so that they cannot view the answers of others –In a lab setting, use dividers to block the view of adjacent monitors. Establish procedures to maintain a quiet testing environment –Remember that some students will finish more quickly than others

32. 2015 CAASPP Online Test Administration Workshop 32 Establishing Appropriate Testing Conditions (cont.) Cell phones or other electronic devices are not allowed to be used during testing (e.g., turned off and put away) Post an “Unauthorized Electronic Devices May Not Be Used at Any Time During the Testing Session” sign so that it is clearly visible to all students. Students may not use Kindles or tablets for reading when they have finished. Bring a book! Place a “TESTING—DO NOT DISTURB” sign on the door

33. 2015 CAASPP Online Test Administration Workshop 33 General Test Rules Students must answer all test items on a page before moving on to the next page Students may not return to a test segment once it has been completed and submitted. (Students will receive a notification when they reach the end of a segment (See TAM p.52) For ELA computer-adaptive tests there will be 2 segments For mathematics computer-adaptive tests there will be 2-4 segments.

34. 2015 CAASPP Online Test Administration Workshop 34 General Test Rules Students must answer all test items before the test can be submitted Students may not return to a test once it has been completed and submitted TAs must follow the “SAY” script exactly each time a test session is administered −Refer to the “SAY” boxes of the Online Test Administration Manual and/or Online Directions for Administration

35. 2015 CAASPP Online Test Administration Workshop 35 Secure Browser Students must use the secure browser to log on to take a summative test Prevents students from accessing other hardware and software functions Prior to testing, TAs should verify that: −External user applications are closed. −The secure browser will not work if the computer detects that a forbidden application is running Note: Test Administrator computers will use a standard Web browser

36. 2015 CAASPP Online Test Administration Workshop 36 Additional Resources Headphones are required for the listening portion of the ELA assessment. Scratch paper should be provided for note taking if necessary. For math, scratch paper is required for all grades. Graph paper is also required for grades 6-8.

37. 2015 CAASPP Online Test Administration Workshop 37 Administering a Test Session 1.The TA logs on to the TA Interface from http://caaspp.org on a standard Web browser http://caaspp.org Note: The pop-up blocker needs to be disabled on the computer used by the TA.

38. 2015 CAASPP Online Test Administration Workshop 38

39. 2015 CAASPP Online Test Administration Workshop 39 Administering a Test Session (cont.) 2.The TA selects the test category or the grade and content area

40. 2015 CAASPP Online Test Administration Workshop 40 Administering a Test Session 3.The TA selects the tests to administer –Additional tests can be added to a session; you will receive a confirmation message box 4.The TA starts a test session

41. 2015 CAASPP Online Test Administration Workshop 41 Administering a Test Session (cont.) 5.The TA informs students of the test Session ID –Write or display clearly in a place all students can see –Session IDs are automatically generated upon selecting the [Start Session] button Additional tests can be added to a session; you will receive a confirmation message box

42. 2015 CAASPP Online Test Administration Workshop 42 Distributing Student Logon Information TAs need to provide students with the Session ID, the student’s Statewide Student Identifier (SSID), and confirmation code (student first name in CALPADS) for logon −May be provided on a card or piece of paper −TAs may write the Session ID on a board or where students can see it Important: Student personal information is secure and must be collected and securely stored at the end of a test session, then securely destroyed at the end of the test administration.

43. 2015 CAASPP Online Test Administration Workshop 43 Administering a Test Session (cont.) 6.Students log on through the secure browser –Recommended: Secure browsers are already launched on testing devices before students sit down to test −To log on, students need:  First Name (Confirmation Code): Student’s legal first name as spelled in CALPADS  State-SSID: “CA” followed by a hyphen “-” and the student’s SSID  Session ID: TA-generated Session ID

44. 2015 CAASPP Online Test Administration Workshop 44 Administering a Test Session (cont.) 7.Students verify their identity –After logging on, students will see the “Is This You?” screen –Students should select [Yes] to continue

45. 2015 CAASPP Online Test Administration Workshop 45 Administering a Test Session (cont.) 8.Students selects the test –All grade-level tests that the student is eligible to take are displayed –Only the tests that the TA selected for the test session and those that have not been completed are selectable by students

46. 2015 CAASPP Online Test Administration Workshop 46 Administering a Test Session (cont.) 9.Upon selecting a test, students will wait for TA approval 10.As students select tests, the TA will see approvals populate in the Approvals queue of the TA Interface

47. 2015 CAASPP Online Test Administration Workshop 47 Administering a Test Session (cont.) 11.The TA selects the [Approvals (#)] button to access the Approvals and Student Test Settings screen

48. 2015 CAASPP Online Test Administration Workshop 48 Administering a Test Session (cont.) 12.The TA verifies the student has selected the appropriate test and reviews the students’ test settings by selecting [See/Edit Details] button next to each student

49. 2015 CAASPP Online Test Administration Workshop 49 Administering a Test Session (cont.)

50. 2015 CAASPP Online Test Administration Workshop 50 Administering a Test Session (cont.) 13.The TA selects [Set] to confirm the current test settings and return to the list of students awaiting approval –The TA will still have to approve the student for testing OR 14.The TA selects [Set & Approve] to establish the existing settings and approve the student for testing Note: If a student’s test settings are incorrect, TAs should deny the student, contact the LEA CAASPP coordinator or SC to correct the test settings in TOMS, and test the student on another day

51. 2015 CAASPP Online Test Administration Workshop 51 Administering a Test Session (cont.) 15.Upon reviewing test settings for all students, the TA approves students to test on the Approvals and Student Test Settings screen –Select [Approve] to approve a specific student for testing –Select [Approve All Students] to approve all students

52. 2015 CAASPP Online Test Administration Workshop 52 Administering a Test Session (cont.) 16.After the TA has approved a student for testing, the student will verify the test information and settings –The student will select [Yes, Start My Test] to begin testing or [No] if their settings are incorrect

53. 2015 CAASPP Online Test Administration Workshop 53 Administering a Test Session (cont.) 17.If a student has a text-to-speech accommodation or is taking a test with ELA listening items, he or she will verify the audio checks through headphones Text-to-speech audio check Test with listening items audio check

54. 2015 CAASPP Online Test Administration Workshop 54 Administering a Test Session (cont.) 18.Finally, students will see the Test Instructions and Help page before they begin testing –Students will select [Begin Test Now] to begin testing

55. 2015 CAASPP Online Test Administration Workshop 55 Monitoring Student Progress TAs can monitor student testing through the Students in Your Test Session table on the TA Interface –Displays students who have logged in and been approved for testing

56. 2015 CAASPP Online Test Administration Workshop 56 Pausing Tests TAs may pause an individual student’s test in the Students in Your Test Session table −This will not affect other students’ tests

57. 2015 CAASPP Online Test Administration Workshop 57 General Pause Rules If there is a technical issue (e.g., power outage or network failure), students will be logged out and the tests will automatically be paused If a test is paused, the student must log in again to resume testing Highlighted text and global notes will not be preserved

58. 2015 CAASPP Online Test Administration Workshop 58 Pause Rules: Computer Adaptive Test If a CAT is paused for more than 20 minutes, the student is: −Presented with the test page containing the test item he or she was last working on (if the page contains at least 1 unanswered item) OR −Presented with the next test page (if all items on the previous test page were answered) −Not permitted to review or change any test items on previous test pages If a CAT is paused for less than 20 minutes, the student is: –Presented with the test item or passage he or she was working on when the test was paused –Permitted to answer previously shown items within a segment

59. 2015 CAASPP Online Test Administration Workshop 59 Pause Rules: Performance Task There are no pause rules for PTs −Even if a PT is paused for more than 20 minutes, the student can return to the current section and continue ELA PTs are divided into two parts −After a student completes the first part, he or she cannot return to it

60. 2015 CAASPP Online Test Administration Workshop 60 Test Timeout Due to Inactivity As a security measure, students are automatically logged out of the test after 30 minutes of inactivity Activity means: −Selecting an answer −Using a navigation option in the test (e.g., selecting [Next] or [Back], using the Questions drop-down list) −Note: Moving the mouse or selecting an empty space on the screen is not considered activity Before the system logs out, a warning message will be displayed

61. 2015 CAASPP Online Test Administration Workshop 61 Stopping an Entire Test Session TAs can stop a test session (and therefore pause tests for all students in the session) –Select the [Stop Session] button in the upper-left corner of the TA Interface –An “Important!” box will appear, requesting verification to end the session and log students out –Select [OK] to continue or [Cancel] to keep the test session open

62. 2015 CAASPP Online Test Administration Workshop 62 Reaching the End of the Test 1.After students answer the last item on the test, the [End Test] button will appear in the upper-left corner of the screen

63. 2015 CAASPP Online Test Administration Workshop 63 Reaching the End of the Test 2.Upon selecting the [End Test] button, an “Attention” message will appear –Students will select [Yes] to proceed with reviewing answers and submitting the test

64. 2015 CAASPP Online Test Administration Workshop 64 Reaching the End of the Test 3.Students may review answers before submitting a test

65. 2015 CAASPP Online Test Administration Workshop 65 Reaching the End of the Test 4.When a student is ready to submit the test, he or she will receive a confirmation screen

66. 2015 CAASPP Online Test Administration Workshop 66 Test Expiration Rules A student’s CAT test remains active until the student completes and submits the test or 45 calendar days after the student has begun the CAT A student’s PT remains active for 10 calendar days after the student has begun it

67. 2015 CAASPP Online Test Administration Workshop 67 Classroom Activity Administration Guidelines Purpose: To provide students with important context, concepts, and key terms prior to taking the PT All students will have a classroom activity before they complete a PT (Absent students will require a make-up CA before completing a PT) May be administered in a classroom or any other appropriate space Recommended: –ELA Classroom Activity should take place on a separate day from the PT –No more than a 3-day lapse between the classroom activity and the PT administration

68. 2015 CAASPP Online Test Administration Workshop 68 Classroom Activity Administration Guidelines (cont.) Computers, projectors, and other technology are allowed but not required for the classroom activity Involves the participation of all students in an instructional task PT will be completed by individual students on the testing device Students may take notes during the classroom activity, but the notes must be collected before proceeding to the PT

69. 2015 CAASPP Online Test Administration Workshop 69 Classroom Activity Administration Guidelines (cont.) Designed to fit into a 30-minute window –Will vary due to complexity of topic and individual student needs The SC should download the assigned classroom activity and provide to TAs at least 1–2 days prior After receiving the activity from the SC, the TA or classroom teacher should administer the classroom activity

70. Test Security

71. 2015 CAASPP Online Test Administration Workshop 71 Test Security Ensuring Test Security – Review TAM 13-15 There are three different types of test security incidents (Review TAM 17-19): 1.Impropriety 2.Irregularity 3.Breach

72. 2015 CAASPP Online Test Administration Workshop 72 Test Impropriety Unusual circumstance that has a low impact on the testing individual or group of students Low risk of affecting student performance, test security, or test validity Correctable and containable at local level Must be reported to the LEA CAASPP Coordinator or SC using the Test Administration Testing Incident or Impropriety Report Form on the CAASPP Forms Web page at http://caaspp.org http://caaspp.org May be reported in the Appeals module should an Appeal be required Example: Students talking during testing

73. 2015 CAASPP Online Test Administration Workshop 73 Test Irregularity Unusual circumstance that impacts the testing individual or group of students May affect student performance, test security, or test validity Correctable and containable at the local level Must be reported to the LEA CAASPP Coordinator or SC using the online Test Administration Testing Incident or Impropriety Report Form at http://caaspp.org, who must then report it to the CDE (caasppirreg@cde.ca.gov) within 24 hourshttp://caaspp.orgcaasppirreg@cde.ca.gov Submitted in the Appeals module for resolution Examples: Student cheating or providing answers to another; student accessing or using electronic equipment (e.g., cellphone)

74. 2015 CAASPP Online Test Administration Workshop 74 Test Breach Event that threatens test validity (e.g., release of secure materials) External implications for the Consortium A breach incident must be reported to the CDE within 24 hours (caasppirreg@cde.ca.gov)caasppirreg@cde.ca.gov Examples: − Administrator or coordinator modifying student responses or record at any time − Administrator allowing students to take home test items, passage, prompts, or scratch paper

75. Practice Tests and Training Tests

76. 2015 CAASPP Online Test Administration Workshop 76 Preparing Students and Staff There are multiple ways to prepare students and staff for the Smarter Balanced summative assessments: –Practice Tests –Training Tests

77. 2015 CAASPP Online Test Administration Workshop 77 Important Notes About Practice and Training Tests Practice Tests and Training Tests do not: –Assess all of the standards that will be in the operational tests –Predict or gauge a student’s success on the operational assessments –Produce scores

78. 2015 CAASPP Online Test Administration Workshop 78 Practice Tests vs. Training Tests Practice TestTraining Test Purpose Provide students with a grade specific testing experience similar in structure and format to the summative assessments Provide students with an opportunity to quickly become familiar with the software and interface features. There are no performance tasks (PTs). Grade Levels Each grade: 3–8, 11 Grade bands: 3–5 6–8 High school Number and Types of Items Approximately 30 items in ELA and 30 items in math per grade level Includes 1 ELA PT and 1 math PT per grade level Approximately 15 items per grade band (6 in ELA and 8–9 in math); PTs are not available Universal Tools, Designated Supports, and Accommodations All Scoring Results are not scored, however answer keys and scoring rubrics are available Results are not scored

79. 2015 CAASPP Online Test Administration Workshop 79 There are two means for accessing the Practice and Training Tests: 1.Through http://caaspp.org on a Web browserhttp://caaspp.org 2.Through the secure browser −Simulates operational online testing environment TAs may create test sessions for the Practice Tests or Training Tests on the TA Practice and Training Site −Students may return to a Practice Test and/or Training Test session Accessing the Practice and Training Tests

80. 2015 CAASPP Online Test Administration Workshop 80 Accessing the Practice and Training Tests 1 2 3 Students and parents who wish to access the Practice Test and/or Test via an Internet browser will select the button below

81. 2015 CAASPP Online Test Administration Workshop 81 TA Training Resources http://caaspp.org/training/caaspp/index.html http://caaspp.org/ta-resources/summative.html

82. Questions