Presentation on theme: "Test Item Bank System And Computer Adaptive Test System"— Presentation transcript:
1 Test Item Bank System And Computer Adaptive Test System Race to the TopTest Item Bank SystemAndComputer Adaptive Test SystemMaryland Assessment Group Conference November 2011
2 The Race to the Top initiative requires that we pursue a new path towards the goal of preparing all students to be college and career ready. What processes and systems will have to be developed to support the new path?
3 The process chart for the work to be done to eliminate achievement gaps is shown here. At what point would you imagine that the teacher would interact with a technology system?
4 This diagram from the RttT Grant proposal identifies how the teachers will interact with the technology systems. What systems do you have in place in you district to assist you in a similar way?
5 These are RttT technology systems being developed to assist teachers
6 The Online Instructional Toolkit is provided to equip teachers with curriculum, model lessons, formative assessment practices, item bank and computer adaptive systems, and professional development opportunities
7 Formative Assessment Practices Assessment for Learning A System of AssessmentSTUDENTStandardsAnnual SummativeQuarterlyUnitThe Test Item Bank and Computer Adaptive Test s are a part of the overall system of assessments available for teachers. As shown in the diagram, the formative practices are those used as teachers assess for learning, during instruction. They will be used more frequently, daily, minute by minute throughout instruction. The Test Item Banks and Computer Adaptive testing is done periodically, whether given as a skill check, a unit test, quarterly as benchmark or interim testing or summatively at the end of the year. All of these assessment systems point towards the Common Core State Standards as a goal.WeeklyMinute by Minute / DailyFormative Assessment PracticesAssessment for LearningTest Item Banks and Computer Adaptive Tests
8 Formative Assessment Practices (Assessment for Learning) To gauge student understanding so teacher can adjust instructionTo improve learning and achievement and involved students in their own learningCollaborative, teacher and student engaged in a dialogue around ungraded learningDaily, ongoing, on the flyTeacher incorporates into instructionStudents evaluate own learningSummative Assessment Processes(Assessment of Learning)To measure what students have learned after the teaching-learning cycleTo determine if a student has reached a certain level of competencyTeacher directed assessmentPeriodic Diagnostic/Progress checks ( district and teacher determine)Computer –based, paper and pencil, performance skills demo with rubric scoringI f you compare the Formative assessment practices with the Summative Assessments Processes you can see how they are different. These both have a place in the overall instructional practices and working together throughout the academic year can give teachers a rich array of data about each student.
9 Traditional Grading Practices More Large ScaleInterim, benchmarks (district tests, purchased or constructed from item banks)Accountability(state tests, national reporting, regulations)More Formative PurposeMore Summative PurposeClassroom Formative Assessments (clarify learning goals, feedback, teacher and student questioning)Traditional Grading PracticesWhen we are looking at the quadrants in this diagram we see that interim tests, benchmarks, or teacher made summative tests are still used for formative purposes. Does this reflect the practices in your district? How do teachers use the benchmark tests and resulting data?More Classroom Focused
10 Testing Systems Projects Overview Implement a Test Item Bank SystemImplement a Computer Adaptive Test SystemTest Item LoadProvide Online Adaptive Testing Units for High SchoolsThe test item bank project incorporates 3 projects in one. The Test Item Bank system will provide the framework for the implementation of the computer adaptive test system and the online adaptive testing units.
11 Test Item Bank SystemA repository for test items aligned to the Common Core State Standards and used to build assessments either online or for paper/pencil testsTeacher tool used to assess student learningCentral office tool to build interim benchmark assessmentsStudent tool to take online assessmentsThe test item bank will have test items for grades 3 – 11 in reading/ English and mathematics. The items will be aligned to the common core state standards.The student information such as student ID, name, teacher name, course, and section will provide the framework for setting up the online tests and the paper and pencil answer sheets. If answer sheets are created then the school will need a scanner to score the test and upload the results.We will be collecting information from the your districts to identify the infrastructure you have in place for connectivity and capacity to work with the system. Information collected will provide a basis for the requirements in the RFP.Districts will be able to use the Item Bank System to create standardized interim benchmark assessments for their schools. Districts will also be able to make a copy to keep at their own districts if they already have an item bank system.A process for accepting items into the item bank will be set up so that teachers can submit their own items for review.
12 Location and Access MSDE Teacher Online Toolkit Interoperable with the Curriculum Management SystemLinks from Formative AssessmentsRole based accessRemotely or within the schoolhouseThe item bank system will be available to teachers when they log into the online toolkit. It is the current vision that teachers have what they will need in one place. Access to tools for instruction, tools for assessment and links to data reports will be with one set of log in procedures.
13 Items Aligned to Common Core State Standards English language arts/literacy and mathematicsGrades 3 – 11Able to be searched and filtered using a meta-tag designation
15 System RequirementsProduce an assessment based on a set of skills or other designation as chosen by the teacherProduce assessments available to students online or printable for paper pencil/answer sheetAllow for flexible scheduling of test windowsAllow for interruptions during online administrations
16 Functions of the System Keeps track of item usageProvides space for teachers to save tests for future useProvides teachers with immediate assessment results such as scores, item analysis, skills masteryProvides a student profile report – for parents or students conferences
17 Online Test System Requirements Secure (locking feature to prevent students from accessing other sites during testing)Accessible from a variety of test taking tools such as iPads, tablets, net books, laptopsRemotely accessible (secure logon procedures on or off site via LEA roles and permissions)
18 The teacher will be able to: Generate answer sheets with student ID, course, and section.Manipulate test in “MS Word”SpacingEnlarge fontProvide a worksheet format for student workChoose how many questions per pagePrint test in PDF format
19 Computer Adaptive Test System An online test system that has the ability to upload any content itemsGoverned by software that selects questions for students based on their previous answersResults show progress and provide results in specific areasThe computer adaptive testing system will be available for teachers to use with their students to obtain pre and post test measures. The tests will be available online only. The tests are governed by an algorithm that will direct the software to choose the student’s test items based on the response to the previous questions.
20 How it Works Based on algorithm built into the software Picks questions based on the student response to a base set of questions ( think about an interactive video game experience)Chooses the next question to challenge students or to elicit a response to an easier question if neededHas an extensive Item bank covering the span of abilities, but students experience a select number of questions at their level ( 10 x rule)The item bank for a computer adaptive test system usually contains ten times the number of items as in a typical benchmark test. The test must represent the differing skill levels in its function to assess the student .
21 Adaptive Experiences for the Test Taker Shorter testChallengingRewardingEquivalent content
22 Adaptive Experience for Teachers Receive information that is specific to the ability of each studentSee progress if equated pre-test and post- test forms are givenAccess results based on the test purposes ( skill based clusters, overall performance, grade level performance)
23 Specifications and Constraints Content driven tests – spiraled content so that the test taker experiences the full extent of the contentRelies on elaborate content classificationsBatteries of adaptive tests provide diagnostic results using different subtests for distinct but related abilities
24 Advantages Shorter test taking time Minimizes the ability of students to borrow answers from other students (overall test security is easier)Can be fast paced or slow – speededness can be reported to the teacherSimple retesting with comparable, equated resultsAbility to interject field test items (district decision)Interoperable with Test Item Bank System
25 Online Adaptive Testing Units Various Wi-Fi connected portable unitsCompatible with LEA internal network configuration to access the internetEnable the students to take online tests within high schools or remotelyOne time procurement of limited sets of units for district implementation of online testsYear 3 implementation with LEA leadership for distribution and supportThe online adaptive test units project will provide a limited number of testing units ( hand-helds, net books, tablets, etc) to LEAs. The purpose of this project is to assist high schools with the online testing.
26 Project TimelineYear 1 – ‘10 – ‘11Project planningRequirements gatheringYear 2 – ’11 – ‘12RFP developmentProcurement of systemsYear 3 – ’12 – ‘13Item load, testing and full implementationAdaptive testing units procure and deployWe are entering the year two now. We still need information from LEAs to finalize our requirement .
27 Project Update Process Status Outcome Create plan Complete Communicate plan to stakeholdersRequirement developmentIn ProgressIdentify business, functional, and system requirements for item test bankData discovery2012Identify data elements and related dictionaryReview process for infrastructure designEstablish hierarchy of user access rolesDefine technical specifications for connectivityAlignment of itemsItem alignment with Common Core StandardsProcure adaptive test units2013Deployment of ATUs to LEAsIdentify LEAs to participate in UATUser Acceptance Testing results and risk mitigationRollout2014IBS “Go-Live” all systemsMost of our work is still ahead in the next two years. We will need to partner with Local Educational Agencies for input during the incremental phases of the software development lifecycle (SDLC). None of the projects is working in isolation. Project representatives meet regularly to collaborate and to assure that systems will work together for the teachers.
28 Stakeholder Participation Participate in webinarsParticipate in requirements gatheringParticipate in early adoption and testingParticipate in roll-out planning
29 Contact InformationSubject Matter Expert/Business Analyst:Barbara HolmanOrMailbox for Item Bank SystemsFor further information you can contact me personally or your designated IBS representative to collaborate on this effort at IBS Collaborative MSDE.state.md.us.