National Conference on Student Assessment June 2014 Balancing Test Security and Accessibility on Next Generation Online Assessments: One State’s Perspective Shelley Loving-Ryder Virginia Department of Education

National Conference on Student Assessment June 2014 Considerations Provide accessibility without advantaging students Ensure that assessment responses are student- generated Maintain security of test items Maintain constructs being measured Permit use of assistive technology that may not interact well with online test delivery software.

National Conference on Student Assessment June 2014 Additional Challenges for Next Generation Tests Greater challenges in capturing some students’ responses for technology-enhanced or constructed response items. Assistive technology is ever changing Accessibility vs accommodations

National Conference on Student Assessment June 2014 Virginia’s Requirements for All School Staff Involved in Testing Signed security agreements Required training Affidavits

National Conference on Student Assessment June 2014 Accommodation Procedures Test session monitored by additional proctor Video tape or audio taped Documented procedures for providing accommodations (e.g., “read aloud” and interpreter guidelines) Supervised access to test prior to administration may be permitted Individual testing may be required Additional affidavits for interpreters

National Conference on Student Assessment June 2014 Security Requirements for Testing Accommodations Allowed AccommodationsSecurity Requirements Interpreting/Transliterating Guidance Document for Interpreters Examiner training requirement Access to test 24 hours prior with supervision Videotaping or proctoring of test session Interpreter and Proctor affidavits Communication Board, Choice Cards Student response transcribed into answer document or entered online. A second adult verifies the transcription to ensure that no errors have occurred Videotaping or proctoring of test session Examiner and Proctor affidavit

National Conference on Student Assessment June 2014 Security Requirements for Testing Accommodations Allowed AccommodationsSecurity Requirements Read-Aloud Guidance Document for Read-Aloud Access to test 24 hours prior as authorized by the Division Director of Testing Audio taping or proctoring of test session Written verification by Proctor to remain on file until scores are verified If audio tape, video must remain on file until scores are verified Flexible Schedule (includes breaks and multiple test sessions) Breaks must be supervised Test materials must not be accessible or viewable by other students or school personnel Students cannot be exposed to previously completed items or items not yet accessed.

National Conference on Student Assessment June 2014 Assistive Technology Challenges While online test delivery systems are now being built to provide certain accessibility features, current systems often do not allow the use of assistive technology software that is not provided as a part of the online delivery system One solution is to use two computers—one that delivers the online test and the other that is loaded with the assistive technology.

National Conference on Student Assessment June 2014 Security Requirements for Testing Accommodations Allowed AccommodationsSecurity Requirements Word Prediction Software (Online Short Paper Writing Test) Use of two work stations (one with word prediction software and the other logged into Test Nav) Student response typed into response area in TestNav by student or entered by scribe with a second adult verifying Student response must be deleted from first workstation

National Conference on Student Assessment June 2014 The Ever Changing Landscape of Accommodations…. Assistive technology is constantly changing How to deal with requests for accommodations that aren’t yet on the “approved” list and with those that are individualized for particular students? Virginia uses a “special accommodations” request process

National Conference on Student Assessment June 2014 Special Accommodation Requests Allows the review of requests from local school divisions for the small number of students with disabilities who require assessment accommodations beyond those routinely allowed. Each request is reviewed by Virginia Department of Education staff and consider the access needs of the student and standardization and test security requirements. Staff members who are experts in assistive technology may be involved as well as content specialists to ensure that the construct of the test items are not changed by the accommodations. Assessment accommodations may be “Approved,” “Approved with Conditions,” or “Not Approved.”

National Conference on Student Assessment June 2014 Special Accommodation Request Process Step 1 Completion and submission of Special Accommodation request form to VDOE by local school division Step 2 Review of Request by staff with experience in online testing, special education and assistive technology Consideration of student’s disability, access needs and test security Step 3 Draft response to request and determine if request is “Approved” “Approved with Conditions”, or “Denied” Step 4 Approval of Response by Assistant. Superintendent Step 5 Return of written response to school division

National Conference on Student Assessment June 2014 Sample Special Accommodation Request Online - Geometry Test Accommodation request for non-verbal student with physical disabilities – Use of iPad to communicate with scribe – Use of Smart Notebook program on laptop with document camera and LCD projector. Smart Notebook program includes tools such as compass, ruler, straightedge, pen) – Projection of test onto Smart Board to perform calculations and constructions Approved with Conditions – Test student individually – Guided Access required with iPad to ensure there is no access to the Internet or other iPad applications – Student responses entered in online test by scribe. Verified by a second school official to ensure that errors have not occurred – Test session proctored – Test items and scratch work must be deleted from Smart Board, but a copy of the scratch work must be securely maintained until scores are received and verified – Written statements required by Examiner and Proctor indicating testing according to conditions.

National Conference on Student Assessment June 2014 Special Accommodation Request: Prompts to Maintain Focus Accommodation request for student with processing disabilities and difficulties maintaining focus – Use statements – “____, stay focused on what you are doing”, and “Keep focused”. – Use of a simple tap on his desk when student is off-task. Approved with Conditions – Student must be test individually so other test-takers are not disturbed. – Only the following prompts may be used: “____, stay focused on what you are doing”, and “Keep focused”. – Test session proctored or video- recorded. – Written statements required by Examiner and Proctor indicating testing according to conditions.

National Conference on Student Assessment June 2014 Security Requirements for Evidence- Based Alternate Assessments Allowed Accommodations Security Requirements Accommodations must be documented on student’s IEP or 504 plan Evidence must be collected under the supervision of the teacher or other school personnel. Testing conditions must be used in which the student does not have access to hints, clueing, or prompts that would provide answers or an unfair advantage. Scoring rules require that individual student work that has not been copied Teacher and Principal Affidavits Note: Affidavits states that the Alternate/Alternative Assessment are a part of the Virginia Accountability System and inappropriate administration is subject to the same consequences and repercussions as violations of test security for the Standards of Learning Assessment)

National Conference on Student Assessment June 2014 Accommodation Resources Accommodations, including assistive technology devices, provided as part of the instructional and testing/assessment process will allow students with disabilities to access the Standards of Learning (SOL) assessments. Explanation of Testing Accommodations for Students with Disabilities- Math Aids- Accommodation Code 19 ccommodation_code19.pdf ccommodation_code19.pdf Explanation of Testing Accommodation for Students with Disabilities- Assistive Technology Accommodations ogy_accommodations.pdf ogy_accommodations.pdf

National Conference on Student Assessment June 2014 Accommodation Resources Participation of Students with Disabilities in Virginia’s Accountability System va_accountability_system.pdf va_accountability_system.pdf Limited English Proficient Students: Guidelines for Participation in the Virginia Assessment Program

National Conference on Student Assessment June 2014 Contact Information Shelley Loving-Ryder (804)