Presentation is loading. Please wait.

Presentation is loading. Please wait.

Test Item Bank System And Computer Adaptive Test System

Similar presentations


Presentation on theme: "Test Item Bank System And Computer Adaptive Test System"— Presentation transcript:

1 Test Item Bank System And Computer Adaptive Test System
Race to the Top Test Item Bank System And Computer Adaptive Test System Maryland Assessment Group Conference November 2011

2 The Race to the Top initiative requires that we pursue a new path towards the goal of preparing all students to be college and career ready. What processes and systems will have to be developed to support the new path?

3 The process chart for the work to be done to eliminate achievement gaps is shown here. At what point would you imagine that the teacher would interact with a technology system?

4 This diagram from the RttT Grant proposal identifies how the teachers will interact with the technology systems. What systems do you have in place in you district to assist you in a similar way?

5 These are RttT technology systems being developed to assist teachers

6 The Online Instructional Toolkit is provided to equip teachers with curriculum, model lessons, formative assessment practices, item bank and computer adaptive systems, and professional development opportunities

7 Formative Assessment Practices Assessment for Learning
A System of Assessment STUDENT Standards Annual Summative Quarterly Unit The Test Item Bank and Computer Adaptive Test s are a part of the overall system of assessments available for teachers. As shown in the diagram, the formative practices are those used as teachers assess for learning, during instruction. They will be used more frequently, daily, minute by minute throughout instruction. The Test Item Banks and Computer Adaptive testing is done periodically, whether given as a skill check, a unit test, quarterly as benchmark or interim testing or summatively at the end of the year. All of these assessment systems point towards the Common Core State Standards as a goal. Weekly Minute by Minute / Daily Formative Assessment Practices Assessment for Learning Test Item Banks and Computer Adaptive Tests

8 Formative Assessment Practices (Assessment for Learning)
To gauge student understanding so teacher can adjust instruction To improve learning and achievement and involved students in their own learning Collaborative, teacher and student engaged in a dialogue around ungraded learning Daily, ongoing, on the fly Teacher incorporates into instruction Students evaluate own learning Summative Assessment Processes (Assessment of Learning) To measure what students have learned after the teaching-learning cycle To determine if a student has reached a certain level of competency Teacher directed assessment Periodic Diagnostic/Progress checks ( district and teacher determine) Computer –based, paper and pencil, performance skills demo with rubric scoring I f you compare the Formative assessment practices with the Summative Assessments Processes you can see how they are different. These both have a place in the overall instructional practices and working together throughout the academic year can give teachers a rich array of data about each student.

9 Traditional Grading Practices
More Large Scale Interim, benchmarks (district tests, purchased or constructed from item banks) Accountability (state tests, national reporting, regulations) More Formative Purpose More Summative Purpose Classroom Formative Assessments (clarify learning goals, feedback, teacher and student questioning) Traditional Grading Practices When we are looking at the quadrants in this diagram we see that interim tests, benchmarks, or teacher made summative tests are still used for formative purposes. Does this reflect the practices in your district? How do teachers use the benchmark tests and resulting data? More Classroom Focused

10 Testing Systems Projects Overview
Implement a Test Item Bank System Implement a Computer Adaptive Test System Test Item Load Provide Online Adaptive Testing Units for High Schools The test item bank project incorporates 3 projects in one. The Test Item Bank system will provide the framework for the implementation of the computer adaptive test system and the online adaptive testing units.

11 Test Item Bank System A repository for test items aligned to the Common Core State Standards and used to build assessments either online or for paper/pencil tests Teacher tool used to assess student learning Central office tool to build interim benchmark assessments Student tool to take online assessments The test item bank will have test items for grades 3 – 11 in reading/ English and mathematics. The items will be aligned to the common core state standards. The student information such as student ID, name, teacher name, course, and section will provide the framework for setting up the online tests and the paper and pencil answer sheets. If answer sheets are created then the school will need a scanner to score the test and upload the results. We will be collecting information from the your districts to identify the infrastructure you have in place for connectivity and capacity to work with the system. Information collected will provide a basis for the requirements in the RFP. Districts will be able to use the Item Bank System to create standardized interim benchmark assessments for their schools. Districts will also be able to make a copy to keep at their own districts if they already have an item bank system. A process for accepting items into the item bank will be set up so that teachers can submit their own items for review.

12 Location and Access MSDE Teacher Online Toolkit
Interoperable with the Curriculum Management System Links from Formative Assessments Role based access Remotely or within the schoolhouse The item bank system will be available to teachers when they log into the online toolkit. It is the current vision that teachers have what they will need in one place. Access to tools for instruction, tools for assessment and links to data reports will be with one set of log in procedures.

13 Items Aligned to Common Core State Standards
English language arts/literacy and mathematics Grades 3 – 11 Able to be searched and filtered using a meta-tag designation

14 Item Design Grade level appropriate Universal design
Accessible Portable Item Profile (APIP) Sensitivity Variety of item formats Supports online student interactive tools, links, hot spots

15 System Requirements Produce an assessment based on a set of skills or other designation as chosen by the teacher Produce assessments available to students online or printable for paper pencil/answer sheet Allow for flexible scheduling of test windows Allow for interruptions during online administrations

16 Functions of the System
Keeps track of item usage Provides space for teachers to save tests for future use Provides teachers with immediate assessment results such as scores, item analysis, skills mastery Provides a student profile report – for parents or students conferences

17 Online Test System Requirements
Secure (locking feature to prevent students from accessing other sites during testing) Accessible from a variety of test taking tools such as iPads, tablets, net books, laptops Remotely accessible (secure logon procedures on or off site via LEA roles and permissions)

18 The teacher will be able to:
Generate answer sheets with student ID, course, and section. Manipulate test in “MS Word” Spacing Enlarge font Provide a worksheet format for student work Choose how many questions per page Print test in PDF format

19 Computer Adaptive Test System
An online test system that has the ability to upload any content items Governed by software that selects questions for students based on their previous answers Results show progress and provide results in specific areas The computer adaptive testing system will be available for teachers to use with their students to obtain pre and post test measures. The tests will be available online only. The tests are governed by an algorithm that will direct the software to choose the student’s test items based on the response to the previous questions.

20 How it Works Based on algorithm built into the software
Picks questions based on the student response to a base set of questions ( think about an interactive video game experience) Chooses the next question to challenge students or to elicit a response to an easier question if needed Has an extensive Item bank covering the span of abilities, but students experience a select number of questions at their level ( 10 x rule) The item bank for a computer adaptive test system usually contains ten times the number of items as in a typical benchmark test. The test must represent the differing skill levels in its function to assess the student .

21 Adaptive Experiences for the Test Taker
Shorter test Challenging Rewarding Equivalent content

22 Adaptive Experience for Teachers
Receive information that is specific to the ability of each student See progress if equated pre-test and post- test forms are given Access results based on the test purposes ( skill based clusters, overall performance, grade level performance)

23 Specifications and Constraints
Content driven tests – spiraled content so that the test taker experiences the full extent of the content Relies on elaborate content classifications Batteries of adaptive tests provide diagnostic results using different subtests for distinct but related abilities

24 Advantages Shorter test taking time
Minimizes the ability of students to borrow answers from other students (overall test security is easier) Can be fast paced or slow – speededness can be reported to the teacher Simple retesting with comparable, equated results Ability to interject field test items (district decision) Interoperable with Test Item Bank System

25 Online Adaptive Testing Units
Various Wi-Fi connected portable units Compatible with LEA internal network configuration to access the internet Enable the students to take online tests within high schools or remotely One time procurement of limited sets of units for district implementation of online tests Year 3 implementation with LEA leadership for distribution and support The online adaptive test units project will provide a limited number of testing units ( hand-helds, net books, tablets, etc) to LEAs. The purpose of this project is to assist high schools with the online testing.

26 Project Timeline Year 1 – ‘10 – ‘11 Project planning Requirements gathering Year 2 – ’11 – ‘12 RFP development Procurement of systems Year 3 – ’12 – ‘13 Item load, testing and full implementation Adaptive testing units procure and deploy We are entering the year two now. We still need information from LEAs to finalize our requirement .

27 Project Update Process Status Outcome Create plan Complete
Communicate plan to stakeholders Requirement development In Progress Identify business, functional, and system requirements for item test bank Data discovery 2012 Identify data elements and related dictionary Review process for infrastructure design Establish hierarchy of user access roles Define technical specifications for connectivity Alignment of items Item alignment with Common Core Standards Procure adaptive test units 2013 Deployment of ATUs to LEAs Identify LEAs to participate in UAT User Acceptance Testing results and risk mitigation Rollout 2014 IBS “Go-Live” all systems Most of our work is still ahead in the next two years. We will need to partner with Local Educational Agencies for input during the incremental phases of the software development lifecycle (SDLC). None of the projects is working in isolation. Project representatives meet regularly to collaborate and to assure that systems will work together for the teachers.

28 Stakeholder Participation
Participate in webinars Participate in requirements gathering Participate in early adoption and testing Participate in roll-out planning

29 Contact Information Subject Matter Expert/Business Analyst: Barbara Holman Or Mailbox for Item Bank Systems For further information you can contact me personally or your designated IBS representative to collaborate on this effort at IBS Collaborative MSDE.state.md.us.


Download ppt "Test Item Bank System And Computer Adaptive Test System"

Similar presentations


Ads by Google