Presentation on theme: "TCAP Achievement Transition"— Presentation transcript:
1TCAP Achievement Transition Laura AtkinsOffice ofAssessment, Evaluation, &Research
2Questions to ExploreWhen the new TCAP achievement test is administered in 2010?Will it be a criterion-referenced assessment or a norm-referenced assessment?Will the test administered in 2010 have a close correlation to the state’s new curriculum standards?It is criterion-referenced.
3Questions to Explore Will the test administered in 2010 be NAEP like? Will roll-outs be conducted during the spring of 2009 to prepare leadership teams in school systems for working with teachers as assessment processes change?
4Questions to ExploreWould you consider holding a state testing conference or regional roll-outs for this unique transitional period resulting from implementation of new curriculum standards and a new assessment process in ?
5Questions to ExploreHow can TVASS information be used most effectively to develop the instructional program to meet student needs as curriculum standards change and added rigor is applied?Issues related to transition.
6Questions to ExploreWhat plans (if any) are in place to change the K-2 assessment process?What plans (if any) are in place to change the assessment process for ELL and SPED students?Discuss security procedures.
8New Achievement Contract Grades K-2 (Optional NRT)Grades 3-8 (Mandated CRT)Modified Academic Achievement Standards Assessment (MAAS)English Linguistically Simplified Assessment (ELSA)This is the last week to order K-2.
9New Vendors Pearson Educational Measurement (PEM) Reporting Logistics Pearson AccessK-2 testingEducational Testing Service (ETS)Write custom items for TN grades 3-8Curriculum & Instruction involved in item development.
10New Achievement Contract October 1, 2007: New content standards to AssessmentDecember 2007: Release of Achievement RFPJanuary 2008: SBE approval of content standardsJanuary to March 2008: Creation and review of New Items by CTBTimeline
11New Achievement Contract March 2008: New Achievement Contract beginsMarch to August 2008: Continued creation and review of New Items by Pearson/ETS (Vendor Transition including Item Bank transfer)June 2008: Review of New Passages (ETS)Contract finalized. Field testing 20 extra items per grade, per content without making test longer, just more versions of test.
12New Achievement Contract August 2008: Review of New Items (ETS)December 2008: Passage Review (ETS)February 2009: Item Review (ETS)Need ELL and SpEd teachers for passage review. Passage review only involves reading/language arts. Item review involves all content areas. Bias review can be anyone who has had classroom experience.
13New Achievement Contract TCAP T: Spring 2009 (Operational items assess current SPIs)TCAP A: Spring 2010 New Operational Assessment (Assess revised SPIs for RLA, Math, & Science)TBA: New Cut Scores Established (Standard Setting Required)This spring’s test still on old standards. Have a conversation with teachers concerning the pilot items and how they present this information to students. Math has some materials that is pushed down to lower grade levels. Algebra will be more present on this test in the middle grades. There are few math SPIs in new standards, but more SPIs in reading/language arts. The state is working to make the test shorter. Social studies has not been affected by new standards so test material will not change. Cut scores have not been established yet, hopefully summer 2010.
15TCAP T Time Limits Grade 3 Grades 4-8 Same as Form S (Spring 2008) SUBTESTTEST ITEMSTIME LIMITScience, Part 13955Science, Part 238Social Studies, Part 1Social Studies, Part 2Grades 4-8SUBTESTTEST ITEMSTIME LIMITScience, Part 13950Science, Part 238Social Studies, Part 149Social Studies, Part 248New TAM will be available in January. Time limits are same as previous test.Same as Form S (Spring 2008)
16State Performance Indicator Chart TCAP Achievement (3-8)State Performance Indicator ChartSpring 2009SPIs have not chagned. A new chart will be up in the fall.
17TCAP Achievement (3-8) Third Grade Pilot Pilot results are in… Results are positive!ALL students grades 3 – 8 will now use a separate answer document.ALL students (3 – 12) may mark in test booklets.Third grade will use separate answer document. They can now mark in their test booklets. Still cannot write in K-2 documents.
18TCAP Achievement (3-8) Practice Answer Document Practice answer document created as a resource for teachers of grades 3-8.Designed for practice in filling in bubbles on a document similar to the TCAP Achievement answer document.Available online at:
20TCAP Achievement (3-8) Unique Identification Numbers Beginning with the Summer 2008 Gateway/EOC test administration, both the student social security number and the State unique identification number (EIS) will be captured in an effort to phase out the use of social security numbers.For one more year, social security number will be requested but not reported on any reports.
21TCAP Achievement (3-8)For information about Curriculum roll-out sessions contact the Office of Curriculum and Instruction.615/Roll-outs are planned for summer. Feb in middle TN counselor conference will provide some roll-out training related to connect new curriculum, resources, assessment, and accountability. Share ways to be proactive. Sessions will be on grade span scope. Sessions will be available for each content area.
22TCAP AchievementThe Office of Curriculum and Instruction (TDOE) is developing companion documents to the new curriculum standards.User’s Guide to the Tennessee Mathematics Curriculum FrameworkScott Eddins 615/This document is available online.
23User’s Guide to the Tennessee Mathematics Curriculum Framework TCAP AchievementUser’s Guide to the TennesseeMathematics Curriculum FrameworkOverviewGeneral Organizational FormatWebb Levels and the TN CurriculumVision of Standards Based Science,Technology Engineering & Mathematics EducationImplementation
24User’s Guide to the Tennessee Mathematics Curriculum Framework TCAP AchievementUser’s Guide to the TennesseeMathematics Curriculum FrameworkFormative and Summative Assessment MappingWebb Depth of Knowledge LevelsAlignment with ACT & NAEPComparison of Former & New StandardsVocabulary in FocusContent in Focus
26TCAP Achievement (3-8) Online Resources Item Samplers & Practice Tests (Current)Educators Guide to Test Interpretation (will be revised)Teacher Directions (will be revised)Frequently Asked Questions (will be revised)Item samplers still valid this year. There will be a full sampler for each content area for the new test (one item for each SPI). Practice test are printed, but that may be cut with the budget issues, and they may be only available online this year.
33New Assessments Modified Academic Achievement Standards (MAAS) English Linguistically Simplified Assessment (ELSA)Moving on with these new assessments. These are for AYP reporting. ELSA is almost identical to the TCAP test, the only difference is that items will have simplified language.
34Achievement - ELSA English Linguistically Simplified Assessment (ELSA) Available for ACH - Spring 2010Intended for Limited English Proficient (LEP) studentsACH test in “simplified” English
35Achievement – ELSA Characteristics of simplified language assessments Reduce wordinessUse common wordsAvoid words with multiple meaningsUse concrete wordsReduce complexity of sentence structureUse the simplest verb formsUse less complex contextThis is different from modified items in that modified items are completely different.
36Achievement– ELSAReduce linguistic barriers to measurement of the intended knowledge or skillThe knowledge or skill being assessed must not be changedChallenging vocabulary should be used only when it is relevant to the knowledge or skill being assessedSame ACH test (e.g., large print and Braille versions)This test is to measure what they know. Reduce barriers language.
37Achievement– MAASApril 2007, federal regulations finalized an option for an alternate assessment based on modified academic achievement standardsResource: National Center on Educational Outcomes (NCEO)
38Achievement– MAASTDOE adopts Modified Academic Achievement Standards (MAAS) AssessmentPearson/ETS contracted to field test MAAS items in Spring 2009MAAS to be operational in Spring 2010MAAS becomes one component of the NCLB Adequate Yearly Progress (AYP)Will be used for AYP reporting. This is a modified test. The performance levels (below prof, prof, advanced) can be changed for this test. The names will be the same but the cut scores will be different. The content must be the same.
39Achievement– MAASAvailable for all 3-8 content areas (Reading and Language Arts, Mathematics, Science, and Social Studies)
40Achievement– MAAS Assess same content standards as ACH Modify test formatModify test questionsModify achievement standardsModifications: 3 answer choices only, the SPI cannot be modified. Cannot make curriculum easier. If feds do not approve this test, these students must take TCAP.
41Achievement– MAAS Referred to as the 2% solution Students for whom the alternate assessment is too easyStudents who cannot demonstrate achievement on TN’s ACHDesign an assessment with more valid score interpretation for the 2% populationPortfolio is too easy for these students, but the TCAP test does not fairly measure their performance.
42Achievement– MAASDevelop modified achievement standards (performance levels)For on-grade-level content standardsChallenging for eligible studentsAssess a student’s mastery for grade-level contentBut at a less difficult level than grade-level achievement standards (performance levels)Who are these students? The field test will include both regular and sped students. If enough regular ed students are assessed, parental permission is not required to test sped students on the field test.
43Achievement– MAAS Eligibility Guidance from State Department of Education (Special Education Division)IEP TeamContent Standards (same as ACH)Achievement Standards (specific to MAAS)
44Achievement– MAASState must establish clear and appropriate guidelines for IEP teams to determine eligibility.Students may be assessed on the modified assessment in one or more subjects.Eligible students may be from any of the disability categories listed in the IDEA.For testing purposed functionally delayed is not recognized by IDEA. If they have another disability, mark sped hours for them. If they do not, you bubble functionally delayed, and they do not count in the sped population for NCLB. Functionally delayed (only) students do not take the MAAS test.
45Achievement– MAASState must ensure students’ IEPs include goals for content standards for the grade in which the student is enrolled.Students must have access to the curriculum and instruction for the grade in which the student is enrolled.Info coming in January (WebEx) on IEP teams. These students do not give up the option of a regular ed diploma because they took the MAAS.
46Achievement– MAASStudents cannot be precluded from attempting to complete the requirements for a regular high school diploma.“A student’s IEP Team continues to determine how the student will participate in State and district-wide assessments.” (OESE)
47Achievement– MAASAchievement expectations may be modified to provide a better method for assessing growth over time for the 2% population.Achievement standards should promote challenging expectations for this population.
48Achievement– MAASAfter performance level descriptors for each achievement level are obtained, appropriate cut scores can be set for each proficiency level.Educator committees will be convened to recommend cut scores for this population on this new assessment.
49Achievement– MAAS Test design changes include: Three answer option choicesShorter reading passagesReduced reading loadSimplified languageDecreased cognitive complexityThese are changes made in the state’s review.
50TCAP Achievement– MAAS Format changes include:Print styles simplified (modified use of italics, underlining, bold-facing)Charts, graphs, and tables simplifiedPrint type size enlargedEssential information reinforcedFewer items on each pageModifications made to this test.
52Pearson Access November/December 2008 TDOE will upload student data to PEMSolutions (pre-id)Spring Testing Coordinators will use the PEMSolutions tool of Pearson Access to:manage student data(before and after testing)access reportsDec. 17 is first trial upload from EIS. Data must be clean.
53Pearson Access November/December 2008 TDOE will upload student data to PEMSolutions (pre-id)Last name, First name, MIStudent ID numberBirthdateGenderEthnicityCode A/BGradeThese are the fields that are being uploaded. There will be no bubbles filled in on this year’s answer document. This year, there will be a bar code on the answer document which will have student’s name and ID number in order to distribute answer docs to students. Pearson access will provide an opportunity to review data. A cut-off date will be established. Username login for Pearson access will be the same, but the password will change. Answer docs will come sorted by grade level, but working with Pearon to identify by teacher name.
54PearsonAccess PEMSolutions Visit the Computer Lab for a demonstrationLogin/Logout & Reset Password (use same passwords as order entry)Administrative ManagementStudent Demographic & Enrollment DataTest ResultsSupportFAQ & much moreThat administrative login per system.
55RANDA Solutions Office of Assessment, Evaluation, & Research New data vendor. TurnLeaf data should be downloaded.Office ofAssessment, Evaluation, &Research
56TCAP Achievement Program Transition TurnLeaf (CTB) available through June 2009Our online applications site is down for upgrades:https://tnors.turnleaf.comhttps://www.eval.k-12.state.tn.us/login
57TCAP Achievement Program Transition New Tech Vendor (upgraded & enhanced)Achievement OrdersRIsSGLsTraining Videoshttps://tdoe.randasolutions.com
58TCAP Achievement (3-8) Achievement Orders: Please verify all quantities (check Braille and LP).For late orders or changes contact:Freddie Summers 615/Order #s and EIS upload information will be used to decide how many tests to order for each school. Systems must continuously updateEISYou may be contacted by Freddie Summers in relation to over orders of Braille test. The number of students in EIS will supply the number of documents, plus a modest overage, plus the Central Office box. Numbers should be accurate if info in EIS is correct.
60Spring 2009 Test WindowFor the school year , school systems can now select 6 consecutive school days to administer the TCAP Achievement Test.This change has been made to accommodate school systems with large ELL and SPED populations.The six consecutive school days must fall within the state testing window.Extra day will assist ELL and SpEd population extended time accommodations.
61Spring 2010 Test WindowGuidance from the U.S. Department of Education concerning the reporting of AYP information states that beginning with the school year , the State Department of Education will be required to report AYP information no later than 14 days before the start of the school year.Since some schools (year round schools) open July 14, the state must move up the testing window.
62Spring 2010 Test WindowThis new regulation will require reporting AYP information at least two weeks earlier than the current AYP reporting date.Since some Tennessee schools open as early as July 14, we must adjust the TCAP Achievement Test dates in order to comply with the new directive.
63Spring 2010 Test WindowWe have drafted two possible solutions that will have the least impact on the Spring 2010 administration of the TCAP Achievement Test:Option 1: March 29 - April 9, 2010(two-week window)Option 2: March 22 - April 9, 2010 (three-week window)
64Spring 2010 Test Window Easter Sunday – April 4, 2010 Good Friday – April 2, 2010Option 1: March 29 - April 9, 2010(two-week window)Option 2: March 22 - April 9, 2010 (three-week window)
65Spring 2010 Test Window Sun Mon Tues Wed Thurs Fri Sat March2122232425262728293031April12Good Friday34Easter Sunday5678910Achievement WindowMarch 30 – April 24
67Test Security T.C.A. 49-1-607 states: Any person found not to have followed security guidelines…placed on immediate suspensiongrounds for dismissalincludes dismissal of tenured employeesgrounds for revocation of state license
68State Test Security Guidelines The Public School Systems, State Special, and Non-Public Schools MUST:1) Adopt a locally monitored test security policy that incorporates, at a minimum, these State Test Security Guidelines. This policy should include a Testing Code of Ethics for personnel to sign and leave at the district office for documentation.Teachers are now filed suit against their districts for improper training.
69Test Security2) Train all personnel involved in the testing process on State Test Security Law, Security Guidelines, local policy, and test administration procedures; retain training documentation for system records.3) Implement check-in, check-out, and quantity verification procedures for all test materials at the system level, at the school level, and foreach test session.
70Test Security4) Restrict handling of test materials to authorized personnel at all times.5) Implement policies and procedures to prohibit all personnel from obtaining knowledge of test items or passage content before, during, and after testing.
71Test Security6) Return test materials immediately after each test session and when the entire administration is completed. Store test materials in a centrally located locked room that is inaccessible to unauthorized persons.7) Create a secure, yet positive, environment for testing.
72Test Security8) Conceal or remove all instructional or reference materials in the test setting that are related to the content area being assessed, such as maps, posters, student samples, bulletin board items, familiar study aids such as graphic organizers, models, or number lines that relate to subject content.Audits by the SDE will be conducted again this year. This is the most common problem…instructional items left on the wall. If in doubt, cover it up.
73Test SecurityAny information regarding the content being measured on the test MUST be concealed during the test administration. Including but not limited to the following:MapsChartsPostersStudent WorkBulletin Board ItemsWriting FormulasWord ListsParts of SpeechGraphic OrganizersNumber LinesDefinitionsWord WallsDisplaysModels
74Test Security9) Turn off all electronic communication devices (cell phones, pagers, PDAs, etc.) in the test setting. Ensure proper calculator use as outlined in the Test Administration Manual, making sure that calculators are cleared before and after administration of each test.
75Test Security10) Confirm each student is the person named on the answer document for every testing session. A photo ID may be required if administrators are not responsible for normal classroom instruction.11) Require Test Administrators and Proctors to carefully adhere to all test administration and accommodation instructions, following appropriate schedules and time limits,outlined in all testdirections.
76Test Security12) Require Test Administrators and Proctors to remain with the students and be observant and nondisruptive throughout the testing session.13) Prohibit coaching students in any way during State assessments.14) Prohibit reading test items and passages by anyone other than those reading to students being tested through read aloud, unless indicated in test instructions or accommodations.Read aloud issues should be addressed with extra day. Assure that everyone getting accommodations they are supposed to get. Sped teachers should understand what they are supposed to read aloud. Do not block classroom door windows. Everyone should be able to look in. Teachers should not read test book.
77Test Security15) Ensure students respond to test without assistance from anyone.16) Ensure that test items are not reproduced, duplicated, or paraphrased in any way, for any reason, by any person.17) Maintain confidentiality of student-specific accountability demographic information and test results at all times.Make sure you have identified version numbers. Call the state if this is not done.
78Test Security18) Document test security concerns, including missing materials, on the Breach of Testing Security Report form.19) Failure to report a breach of security compromises the integrity of the testing process and should be treated as a breach of testing security.If you learn of a breach and do not report it, that is a breach.
79Item Review Office of Assessment, Evaluation, & Research Items are scrutinized by teachers and then goes through a bias review, a Q&A review by vendors, and then another review by the State.Office ofAssessment, Evaluation, &Research
80Item Review Purpose: Obtain Tennessee educator input Support the content validity of the Achievement TestTwo sets of committeescontent reviewsbias and sensitivity reviews
82Item Development Cycle All steps in the item development cycle are completed under the direction of staff at the TDOE, and all items and test forms are approved by TDOE.LD Note to TDOE: Don’t mean to leave TDOE out of this process. Do we want to add TDOE role as separate steps on this slide in approval role? Or do you prefer as is with a speaking note to TDOE’s role.
83Item ReviewETS brings a range of items developed for all performance indicators.Following a structured processeducators read all items individuallythe committee discusses itemsEducators mark their individual comment form with input on items
84Item ReviewEach committee reviews all ACH items and all MAAS items for a grade span (3-4; 5-6; 7-8).Each committee is comprised of teachers of regular and special education students, as well as teachers of limited English proficient (LEP/ELL) students.ELL, SpEd, Regular Ed on Committee for review.
85Item Review For each item, educators will recommend: accept “as is” accept with editsreject
86Item Review Guiding Questions Does the item measure the SPI? Does the item reflect sound instructional practice at the grade level?Is the item clearly and concisely stated?Does the item have one, and only one, clearly correct answer?
87Item Review Guiding Questions Does the item measure the SPI? Does the item reflect sound instructional practice at the grade level?Is the item clearly and concisely stated?Does the item have one, and only one, clearly correct answer?
88Item Review Guiding Questions Are the other answer options clearly incorrect?Are all stimuli and/or graphics clear and understandable, and important to answering the item?Is the item appropriate for all groups of students?