Presentation on theme: "Georgia's Online Options for Assessment: CRCT, EOCT, and the OAS GAETC 2011 November 2, 11:00-12:00 Joe Blessing Michael Huneke."— Presentation transcript:
Georgia's Online Options for Assessment: CRCT, EOCT, and the OAS GAETC 2011 November 2, 11:00-12:00 Joe Blessing Michael Huneke
Benefits of On-line Testing Ease of test administration Security Delivery of Accommodations Preparation for future of testing
Test Administration Easier to administer without inconvenience and labor intensive demands of paper documents Students view one item at a time Items are shown larger than on paper/pencil Research shows comparability between online and paper/pencil testing Students motivated by online format
Research CRCT comparability study showed equity in scores between on-line and paper/pencil Paek (2005) presented a table summarizing the results of recent comparability studies. Adding up the number of grade/subject combinations, we see that out of 97 cases the results for 74 were deemed comparable, in 8 the computer- administered test appeared more difficult, and in 15 the paper test seemed more difficult. – Paek, P. (2005). Recent trends in comparability studies (PEM Research Report 05-05). Available from http://www.pearsonedmeasurement.com/downloads/research/RR_05_05.pdf Studies asked students who took computerized tests whether they would prefer to take future tests on computer or on paper. In all such studies located for this review, the majority of students indicated their preference to test on computer (Bridgeman, Lennon, & Jackenthal, 2001; Higgins, Russell, & Hoffman, 2005; Glassnapp, Poggio, Poggio, & Yang, 2005; Ito & Sykes, 2004; Johnson & Green, 2004; OMalley et al., 2005; Richardson et al., 2002; Sim & Horton, 2004).
Security No paper/pencil tests in building/district No answer documents handled Read-aloud delivered by computer rather than the test administrator With paper/pencil, students can potentially view multiple answers on another students answer sheet. Online, they can only see one question at a time and the answer only after it is chosen.
Security Online testing offers unique security issues. Test examiners should evaluate testing environment for security concerns. – Create seating arrangements that keep students focused on their own computers. – Use monitor shields to maintain privacy. – Make certain that there are enough proctors in the room to respond to technical issues and allow active monitoring.
Accommodations Accommodations available Online Large Print Oral reading of test questions (Text Reader) Audio amplification device in conjunction with text reader Basic four-function calculator Untimed Test Accommodations not available Online Braille Color overlays, templates, or place markers Oral reading of reading passages Low vision aids All other accommodations are administered same as Paper/Pencil mode
Accommodations Many accommodations delivered by computer and assigned by test coordinator leads to more consistency in administration. Read-aloud accommodations delivered – Read-aloud is pre-assigned and available for student. – Reading of test items is standardized – Reading of content is personalized and student-directed Allows for individual student pacing Multiple accommodation groups possible in a single setting
CRCT Online Retest Updates CTB improvements DOE changes that would need to be considered to promote large-scale usage Additional marketing
Updates to On-Line testing program Two-day report turnaround after closing of test sessions School test coordinator has rights to re-open session No more wait time after network disruptions Allow for underlining/highlighting of text Multiple sessions per day (already allowed) Enhanced test examiner experience
Sample On-line Testing Schedules LocationSessionMondayTuesdayWednesdayThursdayFriday Lab 1 1 Reading Gr 3 Reading Gr 5Math Gr 5Makeup Lab 1 2Reading Gr 8 Math Gr 8 Makeup Lab 2 1 Reading Gr 3Reading Gr 5 Math Gr 5 Makeup Lab 2 2 Reading Gr 8Math Gr 8 Makeup
Test Features One item displayed at a time Passage shown at all times Click on text to read aloud. Option for speed of delivery (Accommodation) Return to review items
Test Features Option Eliminator Scroll to end of passage
Overview of Online Retest Roles System Test Coordinator Assessment policies, procedures and possible test scheduling School Test Coordinator / Test Examiner Test scheduling, student management, test administration, and invalidations School Technology Coordinator Setting up hardware and installing software for each school computer System Technology Coordinator Equipment and ISP, proxy servers, firewalls and WAN connections at each school
Roles and Responsibilities Technology Coordinator Before Test Receive Login and Password Emails Complete Site Readiness Activities Review Installation Guide and Checklists Verify network and workstation configuration Run Network Utility software Install test software Download test content Complete practice test Submit site certification During Test Assist Test Examiners with technical issues After Test Use the Uninstall Utility to remove test software from workstations
CTB servers Student computers Customer proxy, firewall, virus scanning, web content filtering Customer network Student Workstations MS Windows based computers with 1.0 GHz CPU, with 512MB RAM and 200MB free space Mac computers with 1.8 GHz CPU, 512MB RAM and 200MB free space Linux workstations High speed internet connection, minimum 1.5 Mbps per 150 workstations upload and download Architecture Overview
Network Requirements Bandwidth T1, high-speed cable, or high-speed DSL recommended 1.5 Mbps upload and download Firewalls Port 80 and 443 must be open Proxy Servers Dynamic pages are not to be cached Allow both HTTP and HTTPS traffic for *.ctb.com Prioritize traffic to and from *.ctb.com Users must have credentials if proxy authentication is enabled Web Content Filtering
Administrator / Student Workstation Hardware and Memory Processor 1.3 GHz Memory 512 MB Disk space 200 MB Monitor Minimum 800x600 pixels Operating System Windows 2000 SP4, or XP SP3, Vista SP1 Mac: OS 10.4.8, 10.5.4, or 10.6 Intel Linux: Linux Fedora Release 11 (Red Hat), OpenSUSE 11.1, or Ubuntu 9.0 Additional Software Adobe® Air® 1.5; Adobe® Reader® 7.0 or later Java Runtime Environment JRE 5.0, Update 16 Web Browser ( specific for Administrator only ) Microsoft Internet Explorer 6.0 or 7.0 Mozilla Firefox 2.0 Internet Access High-speed internet connection: Minimum 1.5 Mbps upload and download –T1, high-speed cable, or high-speed DSL Peripherals Headphones for students with read-aloud accommodation
Administration Key Functions Manage Users Manage Students Schedule Tests Print Tickets Monitor Status Mark Invalid/ Irregular Test
Overview End-of-Course Tests are administered via paper and pencil and/or online. – Paper/ pencil assessments are only available during the winter, spring and summer administrations. – Online assessments are available for all main administrations, mid-month administrations, and (beginning in Winter 2011) retest administrations.
Suggested Schedule One-day Administration: Distribute materials, Student Authorization Ticket, Reading Directions Section 1........................10 min. Test Administration – Section 2.....................45-60 min. Break between sections......................................5 min. Test Administration – Section 3....................45-60 min. Collect test materials.........................15 min.
Suggested Schedule Two-day Administration: Day One Distribute materials, Student Authorization Ticket, Reading Directions Section 1........................10 min. Test Administration – Section 2..................................................45-60 min. Collect test materials........................................................................15 min. Day Two Distribute Student Authorization Tickets.............................................5 min. Test Administration – Section 3...................................................45-60 min. Collect test materials.........................................................................15 min.
TestNav TestNav is Pearsons proprietary test delivery application that is used to access the Internet and online tests. TestNav must be installed either on a file server or on each test delivery workstation. Test Delivery workstations are used to deliver tests to students. Students access online tests by means of TestNav. TestNav does not require a browser.
System Requirements The minimum and recommended hardware requirements for all TestNav test delivery workstations are listed below. Hardware RequirementsWindowsApple / Macintosh Processor Pentium III 733 MHz (PIII/4) 1.3 GHz recommended) G3 500 MHz (G4 800 MHz recommended) Processor (for TestNav only) PIII 500 MHz (PIII 800 MHz recommended) G4 500 MHz (G4 800 MHz recommended) Memory 512 MB RAM512 MB RAM (1 GB RAM recommended) Devices Mouse/Pointing Devices Audio Headphones/Speakers Screen Resolution 800 x 600 (1024 x 768 recommended)
Suggested Movement to Testing On-line – 2011-2012 At least one school re-testing online per district – 2012-2013 More than one school re-testing online in every district (dependent upon districts size) – 2013-2014 Every school that administers re- testing online in a district should have some online testing
www.georgiaoas.org 6/6/2014 Kathy Cox, State Super intend ent of Scho ols We will lead the nation in impro ving stude nt achie veme nt. 34
Teacher Use of the OAS A web-based system that allows teachers to create, administer, and monitor assessments using the same kind of items that appear on the end-of-year assessments. All items and reports are aligned to the Georgia State Standards and easily linked to georgiastandards.org. A tool for preparing students for the end of year assessments. More importantly, a tool for formative assessment throughout the school year.
District Wide Implementation Allows administrators to implement large scale district-wide benchmark assessments using a unique bank of items Three sets of pre-built comprehensive benchmark assessments for GPS Reading/ELA, Math, and Science in grades 1-8 Two sets of Framework Assessments for Reading/ELA, Math, and Science in grades 6-8 One set of EOCT assessments
Benchmarks and Frameworks For CRCT content areas for Math, Science, ELA/Reading Comprehensive 3 per grade/content area Benchmarks One test each for Biology, Physical Science, Combined Am. Lit/9 th Grade Lit (comprehensive) Math I Midterm, Units 1-3 (follow framework units) EOCT Benchmarks Grades 6,7,8 Math, Science, ELA/Reading 1 st test covers roughly 1 st 12 weeks or 3 units 2 nd test covers roughly 2 nd 12 weeks or 2 to 3 units Frameworks
To create a test using the Item Bank, select the Tests tab
Kathy Cox, State Super intend ent of Scho ols We will lead the nation in impro ving stude nt achie veme nt. 56
Kathy Cox, State Super intend ent of Scho ols We will lead the nation in impro ving stude nt achie veme nt. 57
Kathy Cox, State Super intend ent of Scho ols We will lead the nation in impro ving stude nt achie veme nt. 58 A = An attempt at area B = pi x radius x height C = radius 2 x height D = pi x radius 2 x height M6M3 Students will determine the volume of fundamental solid figures (right rectangular prisms, cylinders, pyramids, and cones)
Assessment for Learning Identification of students needing support (screenings), ongoing progress monitoring (formative assessment), process for common assessments (creating as a team, review results as a team, make instructional decisions as a team), modification of instructional techniques based on data.
Tiered Approach to Formative Assessment Teacher Grade or Content Team School District
School Benchmark District Benchmark School Benchmark District Benchmark School Benchmark District Benchmark Qtly System Data Meeting Monthly School Data Meeting
OAS Support Michael Huneke firstname.lastname@example.org 404 232-1208 Support Hours and Contact Information Website: http://www.georgiaoas.org Live Remote Assistance is found on the Support page Software Support Phone Number - (866) 215-6881 Software Support Hours of Operation - Monday through Friday 7am -9pm EST Support Email Address - email@example.com
Assessment Support CRCT Joe Blessing 404 656-2589 firstname.lastname@example.org http://www.gadoe.org/ci_testing.as px?PageReq=CI_TESTING_CRCT CTB Help Desk 866 282-2249 GeorgiaHelpDesk@CTB.com www.ctb.com/gacrct EOCT Melissa Faux email@example.com 404 656-5975 http://www.gadoe.org/ci_testing.as px?PageReq=CI_TESTING_EOCT Pearson Customer Call Center 888 705-9414 firstname.lastname@example.org