Presentation is loading. Please wait.

Presentation is loading. Please wait.

Colorado Online Testing Site Readiness Fall Field Test 2013.

Similar presentations


Presentation on theme: "Colorado Online Testing Site Readiness Fall Field Test 2013."— Presentation transcript:

1 Colorado Online Testing Site Readiness Fall Field Test 2013

2 Attendance Please take a moment and submit a quick attendance form You may also send an to Collin Bonner at include: –Full Name –District –Title or role –Training session attended A record of who attends will help Pearson & CDE identify and reach out to districts who have not been trained 2

3 Agenda Site Readiness Training Introduction –Online Testing Components –Site Readiness Checklist Preparing for Testing –Firewalls & Content Filters –Proctor Caching –System Check –TestNav –Configuring Proctor Caching in PearsonAccess Providing Support During Testing –Early Warning System New Technologies –Virtualization & Online Testing –Emerging Devices & Online Testing Resources 3

4 Site Readiness Training Introduction

5 Testing Overview 5

6 6 Site Readiness - High Level Checklist 1.DTC identify School Technology Coordinators 2.STC view Site Readiness Technical Training 3.Configure Firewall/Content filter 4.Identify testing labs and Proctor Caching Machines 5.Install Proctor Caching 6.Complete System Check verification on workstations and wireless testing labs 7.Update workstations to comply with TestNav hardware/software requirements 8.Verify Assessment Environments in the PearsonAccess Training Center. 9.Enter TestNav Configuration(s) in PearsonAccess Production. 10.Notify CDE’s Collin Bonner at of the results of step 8 for each district testing

7 Configure Firewall & Content Filter

8 Configure Firewall/Content Filter Takeaways How should I configure our Firewalls & Content Filters? Section Resources –Hardware and Software Requirements link for TestNav 7 –TestNav 7 Technology Guidelines Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.8 Step 3

9 Firewalls, Content Filters & Spam Filters The following domains and ports must be allowed to pass through your Internet firewalls, content filters or spam filters. *testnav.com 80, 443 *pearsonaccess.com 80, 443 Note: Districts using IP-based content filters must update the allowed IP addresses as part of this change. These districts will need to contact our Technical Support team to receive the allowed IP addresses and change a setting in PearsonAccess so that the correct URL prints on the Student Authorization Tickets. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.9

10 Preparing for Testing: Proctor Caching for TestNav Overview

11 Proctor Caching Takeaways What is Proctor Caching? Where is Proctor Caching placed within the Network? What are the Proctor Caching Requirements? What are the steps to setup Proctor Caching? What information is provided on the Proctor Caching Diagnostics screens? Section Resources –Hardware and Software Requirements link for Proctor Caching –TestNav 7 Proctor Caching User Guide –Proctor Caching media training Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.11 Steps 4 & 5

12 Proctor Caching Proctor Caching is Pearson-supplied software that is used in conjunction with TestNav to reduce bandwidth requirements and accelerate the delivery of test content. Allows you to pre-cache test content to your local network before a test Reduces the burden on your ISP by eliminating redundancy in requests for test content Stores an encrypted local copy of all pre-cached tests Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.12

13 Diagram of How Proctor Caching Works 13Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

14 Proctor Caching Requirements Runs on Windows, Mac OS X Does not require an underlying server-based operating system Proctor Caching hardware requirements can be found at ctorcaching_2013_1.html ctorcaching_2013_1.html TCP Ports: 80 (Internet), 4480, & 4481 (Local Network) Proctor Caching requires a fixed internal IP address Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.14

15 Proctor Caching Setup Download Proctor Caching installer from PearsonAccess. –Log into PearsonAccess and click on Support->Downloads Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.15 Run the installer and Start Proctor Caching if not already started by the installer. Use System Check from client computer to verify that Proctor Caching is functional. Create a Proctor Caching setup or add Proctor Machine within PearsonAccess at District or School level.

16 Proctor Caching Diagnostics Contents tab for content visibility Pearson On-Line Testing integration Client List tab for monitoring client computer connectivity Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.16

17 Contents Tab Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.17 Content is displayed at the Test and Form level Functionality is available to Refresh, Reload or Purge selected test content Clicking on a test name will display the Content Details screen which displays individual items in a test

18 Contents Tab – Status Ok Ok – Content is successfully cached Not Loaded Not Loaded – Content is not cached Waiting… Waiting… - Content is waiting to be loaded Loading… Loading… - Content is currently loading Failed to load content Failed to load content – There was a failure to load content MD5 Check Invalid MD5 Check Invalid – MD5 comparison could not be completed MD5 Mismatch MD5 Mismatch – MD5 comparison was completed but files did not match In general, any status in red is a critical problem. Any content with a red status should be checked and Reloaded. If the problem is not resolved contact Pearson Support Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.18

19 Client List Tab Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.19 Clients which have connected to TestNav are listed by IP address Clicking on a computer will display the Client Details screen which displays information regarding a particular computer connecting to Proctor Caching

20 Client tab – Status Ok Ok – Client is active Idle Idle – 5 to 30 minutes since client was active Long Idle Long Idle – 30 minutes to 12 hours since client was active Clients are removed after 12 hours of inactivity Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.20

21 Additional Technical Notes For setting up an upstream proxy refer to the Windows or Mac “Tips on Proctor Caching” section in the TestNav 7 Proctor Caching User Guide Use System Check for TestNav capacity planning in your network environment Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.21

22 Preparing for Testing: System Check

23 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.23 System Check Takeaways What is System Check? How do I load System Check? What functionality is provided by System Check? How do I use the TestNav 7 tab? How do I use the Network User Capacity tab? How do I use the Testing Volume Calculator tab? Section Resources –System Check User Guide Step 6

24 System Check for TestNav Overview System Check was developed as a tool for customers to use to validate their computers to ensure that TestNav software could run on their computers. Support can also use System Check to validate the computer for any technical problems. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.24 To gather realistic results System Check must be run from a student testing machine on the same day of week/time of day when you anticipate performing online testing.

25 Loading System Check System Check can be loaded from: ng.com/Colorado and clicking on the link for the System Check tool ng.com/Colorado System Check is Java based and requires that you run and accept an applet Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.25

26 System Check Tabs Functionality TestNav 7 Tab –Validates testing workstations for TestNav Network User Capacity Tab –Bandwidth testing for Internet and Proctor Caching connections Testing Volume Calculator –Testing Volume Calculator for capacity planning Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.26

27 TestNav 7 Tab Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.27 Validates computer workstation to ensure that minimum requirements are met

28 Network User Capacity Tab Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.28 Internet bandwidth testing to the Pearson & Akamai distributed content network Click Add Caching Computer to add your potential Proctor Caching machine for a local network bandwidth test

29 Network User Capacity Tab: Add Caching Computer Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.29 Add a name for the caching computer, the IP address and port (4480)

30 Network User Capacity Tab: Results Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.30 Click on Start to begin the test Based on the bandwidth test System Check can provide the number of students that can test concurrently.

31 Testing Volume Calculator Tab Input number of students testing concurrently. –How many testing labs and student testing workstations are available? Modify network links and network utilization sliders as necessary. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.31

32 Testing Volume Calculator Tab - Interpretation The graph will display the results based on your inputs. The background should remain white to ensure your planned testing volume can be supported by your available network capacity. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.32 Planned testing is feasible with available network capacity Planned testing is dangerously close to available network capacity Planned testing exceeds the available network capacity

33 Preparing for Testing: TestNav Online Testing Platform

34 TestNav Takeaways What are the TestNav Technology Dependencies? What are TestNav Best Practices and Recommendations? What do I need to know about wireless testing? What are Student Response Files (SRFs)? What directory permissions are needed for TestNav? Section Resources –Hardware and Software Requirements link for TestNav 7 –TestNav 7 Technology Guidelines Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.34 Step 7

35 TestNav Technology Dependencies Firewalls, content filters and spam filters must allow the Pearson domains Minimum Screen Resolution of 1024 x 768 is required Browser Dependencies: –Adobe Flash Player version 10 and higher –Java runtime plugin version 1.5 and higher –Windows Firewall configured to allow javaw.exe to communicate –Allow pop-ups for Pearson sites –Accept Java applet –Allow Local File access to home directory Complete requirements can be found at Note: Pearson continually monitors and evaluates the hardware and software requirements. As vendors release newer versions of their products, we update the recommended requirements accordingly. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.35

36 Best Practices and Recommendations Performance: –Use Proctor Caching for best network performance. –Use System Check extensively to test Proctor Caching and network environment before live testing. –Use System Check to test wireless network capacity if using wireless clients. Testing Computers: –Use System Check (TestNav Verification Test) to validate that TestNav will load on machines. –Freeze browser, Adobe Flash Player, & Java environment if possible. –Energy Saving features can interrupt testing, so the recommendation is to set those to not trigger during the length of the test. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.36

37 Best Practices and Recommendations Testing Computers: –Applications that launch automatically  Any applications that may automatically launch on a computer should be configured not to launch during testing sessions. Common applications that may launch automatically include: –Anti-virus software performing automatic updates –Power management software on laptops warning of low batteries –Screen Savers – with Auto message notification Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.37

38 For wireless testing: Wireless access points – more is better Distance – closer is better Limit the number of computers per wireless access point Try a small number first for proof of concept – Do not roll out a few hundred on the first attempt (baby steps) Limit obstructions Use the System Check Utility to verify that you have the appropriate amount of bandwidth for online testing 38Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved. Best Practices and Recommendations

39 TestNav Directories TestNav creates a Pearson directory in the home directory of the testing computer: –Windows XP C:\Documents and Settings\username\Pearson\... –Windows Vista and 7 C:\Users\username\Pearson\... –Mac //Users/username/pearson/... Student Log files: –Pearson/student/logs SRF file default location: –Pearson/student/srf TestNav and test content is temporarily stored locally: –Pearson/temp Pearson recommends that the Pearson directory be excluded if desktop restoration tools such as DeepFreeze are used Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.39

40 Student Responses TestNav will save student responses to an encrypted backup file so the student can either continue testing or exit the system without losing response data. –TestNav requires at least one working save location –TestNav by default saves student response data to: /Pearson/student/srf –Create a TestNav Config to specify custom Primary and Alternate file locations  Pearson recommends designating two Response File Save Locations  Best Practice: Primary Response File Location should be on the student workstation. The alternate Response File Location should be on a shared network location –Students must have full access to designated Response File Locations Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.40

41 Response file will be saved to both the individual student’s testing workstation and a network location Student may resume on any workstation that has access to the shared network save location 41 Student Response Files (SRF) SRF file configured for network and local save locations Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

42 Student Response Files (SRF) SRF file configured for local save locations only All response files will save only to each individual student’s testing workstation Student must resume on the same workstation or SRF files must be moved to the workstation where the student will resume the test If the SRF files are not placed in the same location on the new workstation TestNav will prompt the student to navigate to where the SRF files are saved when they resume testing 42Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

43 Response files will save to one or more network locations Student may resume on any workstation that has access to the shared network save location(s) 43 Student Response Files (SRF) SRF file configured for network save location(s) only Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

44 Using PearsonAccess to Configure Proctor Caching

45 Proctor Caching Takeaways How do I use PearsonAccess to Configure TestNav for Proctor Caching Setup? Section Resources –TestNav 7 Proctor Caching User Guide –Proctor Caching media training Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.45 Steps 9

46 Using PearsonAccess to Configure Proctor Caching Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.46

47 Providing Support during Testing: Early Warning System & Quick Reference Guide

48 Early Warning System Takeaways What is the Early Warning System? What are the TestNav Early Warning System Triggers? How do I resolve Early Warning System alerts? Section Resources –Early Warning System User Guide Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.48

49 Early Warning System (EWS) The Early Warning System (EWS) is integrated functionality that gives TestNav a high degree of fault tolerance and provides additional fail-safes in the event of unexpected network disruptions during online testing. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.49

50 Early Warning System Design Minimal interruptions for students Multiple physical file backups to safeguard against data loss Notification Screens designed for Test Proctors Early Warning System notifications often involve IT staff so an understanding is necessary to correct any issues during testing Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.50

51 Student Responses – SRF Files The EWS writes continuously in the background to the student response files. Both the Primary and Alternate files are written to at the same time. The SRF file has a response data threshold that once reached will trigger TestNav to send response data to Pearson. Uploading of response data is continuous. If an upload fails TestNav will cycle and attempt another upload. If the response data upload is successful, TestNav will create a new empty SRF file and begin the process again. TestNav will only delete an SRF file once it is successfully uploaded. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.51

52 Student Responses - Identification A combination of the test ticket ID combined with the session token is used to uniquely identify a Student Response File (SRF). TestNav can identify the correct student response file if a test is successfully resumed. The SRF can only be used the next time the student resumes a test. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.52

53 TestNav Early Warning System - Triggers Scenario 1: TestNav determines that neither Saving Response File Location is viable. –Will result in immediate Test Proctor Click Here notification. Scenario 2: TestNav is unable to download portions of the test. –Will result in immediate Test Proctor Click Here notification. Scenario 3: TestNav is unable to upload student responses to Pearson. –Will result in Test Proctor Click Here notification at the time of test submission. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.53

54 TestNav EWS – Test Proctor Click Here Screens One of the following screens will appear when scenario 1, 2 or 3 occurs. Students should ALWAYS raise their hand when presented with either of the Test Proctor Click Here screens. They should NEVER click the Test Proctor-Click Here button. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.54

55 TestNav EWS – Scenario 1 In this scenario there is an issue with both of the Response File Locations. Either the student does not have full access to the location(s), OR the location(s) are temporarily unavailable. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.55 Resolve by entering and saving a viable Response File Location. If the issue is temporary, reselect the original location. Then, click the Save and Continue button. Best Practice: Contact your local Technology Coordinator to determine the appropriate course of action.

56 TestNav EWS – Scenario 2 In this scenario, TestNav cannot access the test content on the student work station. If Proctor Caching is being utilized, either the Proctor Caching Server is not functioning properly, OR the local network and Pearson are not communicating. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.56 The ONLY way to resolve this situation is to click the “Close TestNav” button. The student’s testing status will remain as STARTED in PearsonAccess™. Once communication is restored, Resume the student in PearsonAccess and have the student log back into TestNav. Best Practice: Your local Technology Coordinator should be contacted to determine the appropriate course of action. TestNav makes several attempts to access the test content. This message indicates all attempts to retrieve test content have failed.

57 TestNav EWS – Scenario 3 In this scenario not all of the student responses have been received by the Pearson testing server. However, responses not received by Pearson have been saved locally to the Primary Response File Location. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.57 You may try to resolve this scenario by clicking the Retry button until the submission of responses is successful or click on Close TestNav Best Practice: If clicking the Retry button does not resolve the situation, contact your local Technology Coordinator to determine the appropriate course of action. TestNav makes several attempts to communicate with the Pearson Testing Server. This message indicates attempts to submit some or all of the student responses have failed.

58 Quick Reference Guide The Quick Reference Guide should be available to all test proctors to help support students as they test. The Guide provides a list of all the warning messages that TestNav may show a student and how to resolve them. Available through PearsonAccess.com->Colorado->Support->Reference Guides->Online Test Administrator Quick Reference Guide Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.58

59 New Technologies: Virtualization & Online Testing

60 Virtualization Takeaways What is virtualization? What are the concerns with virtualization and online testing? Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.60

61 What is it? –Using your monitor/keyboard/mouse to look at and run software on some other computer –“Thin Clients” are one type of virtualization –VDI (Virtual Desktop Infrastructure) is becoming popular as a way of turning aging computers into terminals, instead of retiring them –There are almost as many different solutions as there are solutions providers Virtualization

62 Virtualization often appeals to schools looking to lower the student to computer ratio within limited IT budgets Virtualization presents unique security considerations when used for high-stakes online testing –Schools & districts may already be using virtualization technology without Department of Education knowledge Virtualization

63 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.63 Virtualization Network Virtualized environment runs on a central server and streams the user interface to workstations

64 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.64 Virtualization Security Concerns

65 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.65 Virtualization Concerns Virtualization security settings on the server are typically unable to be accessed or controlled by the test delivery engine Streamed data between the virtualization server and workstations may not be encrypted and could be intercepted by “man in the middle” exploits –Secure test content –Student keystrokes and interactions with TestNav –Student responses Virtualization on student workstations may not properly lock down the environment and provide a secure online testing environment –Unable to prevent student use of other applications –Unable to prevent capturing screen and test content –Lack of TestNav visibility to virtualization “modal” windows Virtualization performance issues could negatively impact students’ ability to test

66 VDI – Keeping Aging Computers In Service Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.66

67 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.67 A Worst-Case VDI Scenario

68 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.68 TestNav and Virtualization Pearson is working with individual virtualization vendors to qualify their solutions for use with TestNav Approved virtualization solutions will be listed on: Non-approved virtualization solutions may expose online testing to exposure of secure content

69 New Technologies: Emerging Devices & Online Testing

70 Takeaways What technology changes are taking place in schools? What does comparability mean in online testing? Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.70

71 Online Testing & Technology Innovation Technology Markets – Always Changing –Innovation drives continual change in technology markets –As technology evolves, the testing environment also changes  Incremental updates: OS’s, browsers, Flash, Java, HTML  New hardware & devices: netbooks, tablets, smartphones  New network architectures: thin clients, virtualization, cloud computing  New human/computer interfaces: mouse, touch, speech recognition, motion-controlled interaction (eg, Wii, Xbox, Kinect) –A transition to technology-based assessments is not a one-time change

72 Diversity in the School Technology Environment

73 Comparability & “Standardized” Testing Comparability between Paper and Online Tests – Decades of research looking at differences in experience — and impact to student performance — has resulted in:  Scaled score adjustments  Content & design changes  Certain technology requirements – Typically has treated online as the same experience for all students Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

74 New Comparability Research Focus Comparability between Devices –Differences between devices are becoming more pronounced –New research is beginning to look at differences within the “online” condition Areas of Research –Display size –Text input – keyboards vs. on- screen typing –Interface – mouse vs. touch  Cursor-based feedback  Control & fine-point precision  Ability to see through/around cursor Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

75 Display Size Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

76 Text Input

77 Mouse- & Touch-based Interfaces

78 Managing “Ongoing Change” in Testing Supporting Research –Only way to know comparability impact is through research studies and field tests –Expect that technology adoption will always be slowest in the realm of “high-stakes” testing Communication Always Helps –Don’t hesitate to contact Pearson with questions –Please share what is happening with technology in your schools –Help educate your local technology staff about the critical connection between technology at the student level and statewide comparability Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

79 Resources

80 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.80 Documents and Links DTC Academy: https://sites.google.com/a/elearningbydesign.net/cde-dtc-field- test/home or https://sites.google.com/a/elearningbydesign.net/cde-dtc-field- test/homehttp://tinyurl.com/goDTCs CDE DTC webpage: PearsonAccess Colorado Support Page –System Check User Guide –System Check tool link –Hardware and Software Requirements link for TestNav 7 –Hardware and Software Requirements link for Proctor Caching –TestNav 7 Technology Guidelines –TestNav 7 Proctor Caching User Guide –Proctor Caching media training –Early Warning System User Guide

81 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.81 Support Contacts Collin Bonner –CDE Assessment Technology Specialist – Pearson Support – –Districts/Schools should use Pearson as the first contact for support

82 Questions & Answers?

83 CDE Site Readiness Training Evaluation 2013 Please take a moment to complete the evaluation of this training: or


Download ppt "Colorado Online Testing Site Readiness Fall Field Test 2013."

Similar presentations


Ads by Google