1. CAASPP Workshop: Test Security and Filing Appeals 2016-17
Students | Teachers | Instructional Content
Assessment

2. Objective
By the end of this training, the site coordinator will be able to
Understand the CAASPP Summative test security guidelines and procedures for the 2016–17 administration of computer-based and paper-pencil tests
File STAIRS and Appeals applicable to their site 2

3. Agenda Introductions Requirements in the Testing Regulations
General Test Security Guidelines
Specific Test Security Guidelines
Social Media Breaches
Procedures for Reporting Testing Incidents
Changes to STAIRS for
Security and Test Administration Incident Reporting System (STAIRS) Live Demo
Test Security Auditor Activities 3

4. Security of the Test Environment

5. Security of the Test Environment
A secure online testing environment is a state in which a device is restricted from accessing prohibited computer applications (local or Internet-based), or copying and/or sharing test data.
The purpose of this environment is to maintain test security and provide a stable testing experience for students across multiple platforms.
The security of assessment instruments and the confidentiality of student information are vital to maintaining the validity, reliability, and fairness of the results.
TAM Pgs 5

6. Security of the Test Environment
All summative test items and test materials are secure and must be appropriately handled.
Secure handling protects the integrity, validity, and confidentiality of assessment items, prompts, and student information.
Any deviation in test administration must be reported as a test security incident to ensure the validity of the assessment results.
TAM Pgs 6

7. Security of the Test Environment
It is the responsibility of each person participating in the administration of the Smarter Balanced summative assessments to immediately report any violation or suspected violation of test security or confidentiality.
The CAASPP test site coordinator is responsible for immediately reporting any security violation to the local educational agency (LEA) CAASPP coordinator.
TAM Pgs 7

8. Security of the Test Environment - Student Cheating
In the case of a student cheating, the test administrator must stop the cheating; however, the student must be allowed to complete the remainder of the test.
After testing, the test administrator must notify the CAASPP test site coordinator.
Do not call the California Department of Education (CDE) or the California Technical Assistance Center (CalTAC) to report student cheating.
The LEA CAASPP coordinator or CAASPP test site coordinator must, instead, file a report using the Security and Test Incident Reporting System (STAIRS) process.
TAM Pgs 8

9. Requirements in the Testing Regulations

10. CAASPP Test Security Agreement
Agreement must be read and signed ANNUALLY.
Applies to both computer-based and paper-pencil testing
All CAASPP test site coordinators must submit electronically a signed copy of the form; the LEA CAASPP coordinator will receive a notification upon completion.
Make copies for site records
Send original copies to the Assessment office ATTN: Mariela Aguilar
* Pursuant to California Code of Regulations, Title 5 (5 CCR), Section 859 (a)
TAM Pgs 10

11. CAASPP Test Security Affidavit
Affidavit must be read and signed ANNUALLY by:
Applies to both computer-based and paper- pencil testing
All test administrators, test examiners, proctors, translators, scribes, and CAASPP test site coordinators, and any other persons having access to any of the summative tests and test materials, assessment technology platform, registration system, or adaptive engine
Make copies for site records
Send original copies to the Assessment office ATTN: Mariela Aguilar
* Pursuant to California Code of Regulations, Title 5 (5 CCR), Section 859 (a)
TAM Pgs 11

12. General Test Security Guidelines

13. General Test Security Guidelines
All secure test materials must be handled and stored securely.
For computer-based tests, lock any printed rosters in secure storage with limited access, and train staff on the security procedures.
For paper-pencil tests, lock materials in secure storage with limited access, count test booklets upon receipt as well as before and after a testing session, and train staff on the security procedures.
Seat students so that they cannot easily view each other’s work.
TAM Pgs 13

14. General Test Security Guidelines
Cover or remove materials on the classroom walls that may provide information to students during testing.
Actively monitor students during testing.
Report only summative testing incidents using the STAIRS form.
Test administrators and test examiners report to the CAASPP test site coordinator and/or the LEA CAASPP coordinator.
TAM Pgs 14

15. General Test Security Guidelines
Securely destroy secure test materials that do not need to be returned to Educational Testing Service (ETS).
For computer-based summative assessments, these materials include:
Student logon information cards
Administrator logon information for the test delivery system
Rosters of students scheduled to take the test
Scratch paper with students’ work
TAM Pgs 15

16. General Test Security Guidelines
For paper-pencil tests:
Return test booklets
Securely destroy answer documents (blank with Pre-ID labels and any with student responses recorded)
Return Standards-based Tests (STS) in Spanish Grade 2 Directions for Administration
Securely destroy scratch paper with students’ work per the guidelines listed in Chapter 3 Test Security of the Online Test Administration Manual.
TAM Pgs 16

17. General Test Security Guidelines
Printed materials must be kept in a securely locked room or locked cabinet that can be opened only with a key or keycard by staff responsible for test administration who have signed a CAASPP Test Security Affidavit. These include:
Documents with student information
Student logon cards
Pre-ID labels
Scratch paper
TAM Pgs 17

18. Test Security Guidelines
Scratch paper must be collected after the first part of the full write performance task (PT) and securely stored until it is returned to the student during the second part. Scratch paper must be shredded according to LEA and/or California policies or procedures after the second part of the full write has been completed.
TAM Pg. 32 18

19. Specific Test Security Guidelines

20. What’s New for 2016–17
Change in reporting of Smarter Balanced and STS paper-pencil tests
A Restore appeal is no longer reported via the STAIRS form or the Appeals interface. If an LEA CAASPP coordinator would like to request a restore, he or she must call CalTAC.
A Restore request will not be granted unless the test was inappropriately reset.
For example, a restore request will be rejected if the test has been started, submitted, or reported. 20

21. What’s New for
A Grace Period Extension will be granted only for technical issues that resulted in expiration of the 20-minute pause rule.
A Grace Period Extension will not be granted to extend the availability of an expired or submitted test. To re-open an expired or submitted test, you must submit a re-open appeal request.
For example, Unless there was a technical difficulty, a grace period extension will not be granted for cases where the school allowed the students to exceed the 20-minute pause rule.

22. Preventing Security Violations
LEA staff, school staff, test administrators, and test examiners play a critical role in monitoring the testing session and adhering to directions for standardized administration.
Become familiar with the test security protocols outlined in Chapter 3 of the CAASPP Online Test Administration Manual:
TAM Pgs 22

23. Before and During Testing
All test items and test materials must remain secure and must be appropriately handled.
Includes creating a secure testing environment for what students can see, hear, and/or access
The test administrator is ultimately responsible for monitoring and reporting test security issues.
Inappropriate Internet access
Any other improper displaying, printing, photographing, duplicating, and/or sharing of test questions
TAM Pgs 23

24. As Testing Starts and During Testing
A logon card should include the student’s legal first name and Statewide Student Identifier (SSID).
A logon card may also include the student’s last name
Before handing out the cards, use the Test Administrator Interface or the Test Operations Management System (TOMS) to verify that the SSIDs on the cards are correct.
Ensure that the students have properly logged on and are taking the test for which they are scheduled.
Monitor students taking the test for any breach of the secure browser and that it has not allowed students to access external sites or other resources on their testing device during the assessment.
TAM Pgs 24

25. Student Logon Information for the Computer-Based Tests
Student logon information is considered secure material, so it must be provided to, and viewable only by, the student to whom the logon information pertains.
Student logon information must be returned to the CAASPP test site coordinator, stored in a secure location between testing sessions, and securely destroyed immediately after testing has been completed.
Losing student login information is no longer a reportable incident in STAIRS (previously classified as a breach).It is still however, considered of student PII and should be handled at the local level.
“(4) (A) I will keep all assigned, generated, or created usernames, passwords, and logins secure and not divulge pupil personal information to anyone other than the pupil to whom the information pertains for the purpose of logging on to the test delivery system.” CAASPP TEST SECURITY AFFIDAVIT
“(4) I will securely destroy all print-on-demand papers, scratch paper, and other documents as prescribed within the contractor’s(s’) or consortium’s administrative manuals and documentation.” CAASPP TEST SECURITY AGREEMENT 25

26. Test Security Guidelines for Computer-Based Tests
Ensure there is adequate space between students so that they cannot see each other’s work.
If using a computer lab, consider using temporary dividers (e.g., folders taped to the sides of the monitors).
Test the equipment and network to be used during testing.
Report any work-arounds to the secure browser to CalTAC.
Some software (e.g., teacher-monitoring software, Apple AirPlay) may need to be disabled or monitored locally. 26

27. Test Security Guidelines for the Paper-Pencil Tests—
Test Administrators and Test Examiners
Receive and return materials to the CAASPP test site coordinator.
Make sure test booklets and answer documents are distributed to the correct students.
Do not allow unauthorized electronic devices (e.g., cell phones) to be used in the classroom during testing. 27

28. Social Media Breaches
Social media and other Web sites will be monitored daily for postings of secure material.
A process is in place to review and investigate all test security incidents related to social media to determine what action needs to be taken. 28

29. Item Security and Information Bar
New bar code has been added that can be decoded and tracked if someone takes a picture of the screen and posts it to social media
CDE Slide 121 29

30. Procedures for Reporting Testing Incidents

31. Test Security and Reporting Impacts for 2017
Invalidated appeals will continue to result in an invalidated score.
Paper-pencil tests marked with a “C” in box A2 to indicate Student Observed Cheating will continue to result in an invalidated score.
Invalidated scores are reported in the Online Reporting System and Student Score Reports, but are not included as valid scores in aggregated test results.
Security issues for Smarter Balanced paper-pencil tests and STS paper-pencil tests do not need to be reported in STAIRS; however, you must notify your LEA CAASPP Coordinator so we can document it for our files. 31

32. CAASPP STAIRS Process
STAIRS and appeals are used for the summative assessments only.
The online CAASPP STAIRS form is the starting point for LEA CAASPP coordinators and CAASPP test site coordinators reporting a test security incident or other testing issue that interferes with the administration and completion of the summative assessment.
Coordinators access the CAASPP STAIRS form in TOMS by selecting the [STAIRS/Appeals] button.
You cannot submit an appeal request without first submitting the CAASPP STAIRS form. 32

33. What’s New in STAIRS for 2016–17
The “Other” category has been removed.
If your incident does not fit any of the reasons provided, please contact CalTAC.
There is no longer a restore option in STAIRS.
There is now an onscreen display of further action to file an appeal.
The system now presents the CDE’s review decision prior to the final submission of the STAIRS report.
STAIRS includes an integrated messaging system.
The user is now able to send the CDE questions or comments related to their particular STAIRS case within the STAIRS module.
You now have the ability to select your name from the “identify yourself” page rather than having to manually enter the information. 33

34. Commonly Reported Issues in 2015–16 and How to Avoid Them
How to minimize issues that lead to the most common appeal types
To avoid having to submit a Reset appeal, make sure all test settings are properly set before the test is started.
Double check that all selected settings are needed for each student before the test is started. If the student did not need the selected settings and they need to be changed after the test has started, the LEA will have to resort to the appeals process which will add time for processing and changes needed. 34

35. How to Avoid Commonly Reported Issues
To avoid having to submit a Re-open appeal, be sure to properly schedule testing times to ensure all students can test in a timely manner.
Please keep in mind the ORS system does not report in real time, which may make it difficult for LEAs to pull information regarding who has or has not tested in a timely manner.
Only report a Restore appeal if the test has been inadvertently or improperly reset.
Only report a Grace Period Extension appeal if a technical issue has occurred that resulted in expiration of the 20-minute pause rule. 35

36. Which Issues Should Not Be Reported in STAIRS
Question skipping—The system does not allow students to skip questions or submit a segment with questions that have no response.
Incorrect SSID (swap request)—Please call Gina Simpson.
Media use after testing is complete—The use of electronic devices is prohibited but the issue does not need to be reported unless the student was actively engaged in a testing session or the student completed testing but the LEA can verify that the device was being used to help other students that are engaged in a test session. 36

37. Which Issues Should Not be Reported in STAIRS
If an English learner (EL) enrolled in a U.S. school less than 12 months as of April 15 and accidentally took the English language arts/literacy (ELA) test (still required to take mathematics). This student’s score will not be used for federal accountability purposes and therefore need not be reported.

38. Procedures for Reporting Testing Incidents—Impropriety
Unusual circumstance that has a low impact on the testing individual or group of students
Low risk of affecting student performance, test security, or test validity
Correctable and containable at the local level
Must be reported to the LEA CAASPP coordinator or CAASPP test site coordinator
Example: Students talking during testing
TAM Pgs 38

39. Process Flow for Reporting an Impropriety
TAM Pgs 39

40. Procedures for Reporting Testing Incidents—Irregularity
Unusual circumstance that impacts the testing individual or a group of students
May affect student performance, test security, or test validity
Correctable and containable at the local level
LEA CAASPP coordinator or CAASPP test site coordinator must enter testing incident into STAIRS within 24 hours
California regulations require that irregularities be reported to the California Department of Education (CDE) within 24 hours (5 CCR Section 859 [e]).
Example: Student accessing or using an unauthorized electronic device (e.g., a cell phone)
TAM Pgs. 36 40

41. Test Security Chart—Irregularity
MEDIUM Irregularity
Student(s) cheating or providing answers to each other, including passing notes, giving help to other students during testing, or using hand-held electronic devices to exchange information
Student(s) accessing the Internet or any unauthorized software or applications during a testing event
Student(s) accessing or using unauthorized electronic equipment (e.g., cell phones, PDAs, iPods, or electronic translators) during testing
Disruptions to a test session, such as a fire drill, school-wide power outage, earthquake, or other act(s)
Administrator or coordinator failing to ensure administration and supervision of the Smarter Balanced assessments by qualified, trained personnel
Administrator giving incorrect instructions that are not corrected prior to testing.
Administrator or coordinator giving out his or her username/password (via or otherwise), including to other authorized users
Administrator allowing students to continue testing beyond the close of the testing window. For the PT, this is 10 calendar days. For the CAT, this is 45 calendar days. For a paper-pencil assessment, this is 3 weeks.
Administrator providing a student access to another student’s work/responses (unintentional access granted)
Student not receiving accessibility support or accommodation as required by IEP or Section 504 plan
TAM Pgs. 36 41

42. Test Security Chart—Irregularity
MEDIUM Irregularity
Student without IEP or Section 504 plan did not get a designated support.
Administrator or teacher coaching or providing any other type of assistance to students that may affect their responses. This includes both verbal cues (e.g., interpreting, explaining, or paraphrasing the test items or prompts) and nonverbal cues (e.g., voice inflection, pointing, or nodding head) to the correct answer. This also includes leading students through instructional strategies such as think-aloud, asking students to point to the correct answer or otherwise identify the source of their answer, or requiring students to show their work.
Administrator providing students with nonallowable materials or devices during test administration or allowing inappropriate designated supports and/or accommodations during test administration
Administrator allowing designated supports not indicated by an educator (or team of educators with parent/guardian and student input) and that are not in the student’s IEP or Section 504 plan
Administrator allowing inappropriate accommodations (which are not in the student’s IEP or Section 504 plan) during test administration
Administrator allowing anyone other than a student to log on to the test unless prescribed as an allowable accommodation in the student’s IEP. This includes test administrators or other staff using student information to log on or allowing a student to log on using another student’s information.
Administrator providing a student access to another student’s work/responses (intentional access granted)
TAM Pgs. 36 42

43. Process Flow for Reporting a Testing Irregularity
TAM Pgs. 36 43

44. Procedures for Reporting Testing Incidents—Breach
A breach is an event that poses a threat to the validity of the test.
The LEA CAASPP coordinator or CAASPP test site coordinator must immediately telephone the CDE and then submit a report in STAIRS within 24 hours.
California regulations require that irregularities be reported to the CDE within 24 hours (5 CCR Section 859 [e]).
Example: Release of secure materials or a security/system risk
To qualify as a breach, the circumstances have an external implication to the Smarter Balanced Assessment Consortium.
May result in the decision to remove an item from scoring
TAM Pgs 44

45. Test Breach
External implications for the Consortium and other statewide assessments
Example: Test administrator or CAASPP test site coordinator modifying student responses or records at any time
Example: Test administrator allowing students to take home test questions, passages, prompts, or scratch paper
CDE Slide 120
TAM Pgs

46. Test Security Chart—Breach
HIGH
BREACH
Administrator or coordinator modifying student responses or records at any time
The live student interface or Test Administrator Interface being used for practice instead of the training or practice tests
Adult or student posting items or test materials on social media (Twitter, Facebook, etc.)
Administrator allowing students to take home printed test items, reading passages, writing prompts, or scratch paper that was used during the test or failing to otherwise securely store test materials
Adult or student copying, discussing, or otherwise retaining test items, reading passages, writing prompts, or answers for any reason. This includes the use of photocopiers or digital, electronic, or manual devices to record or communicate a test item. This also includes using secure test items, modified secure test items, reading passages, writing prompts, or answer keys for instructional purposes.
Secure test materials being shared with the media (such as the writing prompts, test items, or reading passages) or allowing the media to observe a secure test administration
Adult or student improperly removing secure testing materials, such as test items, stimuli, reading passages, writing prompts, or scratch paper from the testing environment
TAM Pgs 46

47. Process Flow for Reporting a Testing Breach
TAM Pgs 47

48. Invalidate Reset Re-open Grace Period Extension Restore
Types of Appeals
Invalidate
Reset
Re-open
Grace Period Extension
Restore

49. Invalidate Appeal Invalidate Appeal Reasons:
Test Security Breach by student
Validity Issue
TA coaching
modifying student responses
providing nonallowable materials
deliberate inappropriate response to items
Student Cheating or Accessing Unauthorized Devices  
Accidental Summative Access 49

50. Reset Appeal Reset Appeal Reasons: Accessibility Issue
Incorrect Assessment Type Opened 
Incorrect SSID (no swap)
Accidental Summative Access 
A reset because of accidently administering a summative test will only be granted if one of the following is true: 
The student completed less than 10 CAT or less than 1 PT; or
The student did not receive a designated support or accommodation as prescribed by IEP or Section 504 plan 50

51. Re-Open Appeal Re-Open Appeal Reasons:
Test expiration or accidental submission by student 
A student starts a test unintentionally and the student is unable to complete the test before it expires
Technical issue that results in test expiration or submission prior to completion
Accidental summative access (reset criteria not met) 51

52. Grace Period Extension Appeal
Grace Period Extension Reasons:
Used for computer-adaptive tests only. Not applicable to the performance task.
There were disruptions to a test session, such as a technical difficulty, fire drill, school wide power outage, earthquake, or other act beyond the control of the test administrator (Test must be in a paused state.)
Only available for technical issues that resulted in expiration of the 20-minute pause rule. 52

53. Restore (from a Reset) Appeal
Restore Appeal Reasons:
If a test was inadvertently or inappropriately reset, LEAs will have to contact CalTAC to verify eligibility.
Restore appeal would be requested by the LEA and CalTAC would have to determine whether it meets the requirements for a restore.
The LEA cannot request a restore request in the Appeals module and must request that CalTAC perform the restore. 53

54. Test Security Webcast STAIRS Live Demo

55. Test Security Auditor Activities

56. Test Security Audits Audits are conducted throughout the state.
Auditors will:
Observe test security processes being used.
Interview the CAASPP test site coordinator or designee before, during, and after testing.
Summary reports will be provided to the CDE and will be shared with the LEA CAASPP coordinator. 56

57. Questions and Answers