TCAP Achievement Transition Laura Atkins Office of Assessment, Evaluation, & Research

Questions to Explore When the new TCAP achievement test is administered in 2010? Will it be a criterion-referenced assessment or a norm-referenced assessment? Will the test administered in 2010 have a close correlation to the state’s new curriculum standards? It is criterion-referenced.

Questions to Explore Will the test administered in 2010 be NAEP like? Will roll-outs be conducted during the spring of 2009 to prepare leadership teams in school systems for working with teachers as assessment processes change?

Questions to Explore Would you consider holding a state testing conference or regional roll-outs for this unique transitional period resulting from implementation of new curriculum standards and a new assessment process in 2009-2010?

Questions to Explore How can TVASS information be used most effectively to develop the instructional program to meet student needs as curriculum standards change and added rigor is applied? Issues related to transition.

Questions to Explore What plans (if any) are in place to change the K-2 assessment process? What plans (if any) are in place to change the assessment process for ELL and SPED students? Discuss security procedures.

New Achievement Contract & Timeline Office of Assessment, Evaluation, & Research

New Achievement Contract Grades K-2 (Optional NRT) Grades 3-8 (Mandated CRT) Modified Academic Achievement Standards Assessment (MAAS) English Linguistically Simplified Assessment (ELSA) This is the last week to order K-2.

New Vendors Pearson Educational Measurement (PEM) Reporting Logistics Pearson Access K-2 testing Educational Testing Service (ETS) Write custom items for TN grades 3-8 Curriculum & Instruction involved in item development.

New Achievement Contract October 1, 2007: New content standards to Assessment December 2007: Release of Achievement RFP January 2008: SBE approval of content standards January to March 2008: Creation and review of New Items by CTB Timeline

New Achievement Contract March 2008: New Achievement Contract begins March to August 2008: Continued creation and review of New Items by Pearson/ETS (Vendor Transition including Item Bank transfer) June 2008: Review of New Passages (ETS) Contract finalized. Field testing 20 extra items per grade, per content without making test longer, just more versions of test.

New Achievement Contract August 2008: Review of New Items (ETS) December 2008: Passage Review (ETS) February 2009: Item Review (ETS) Need ELL and SpEd teachers for passage review. Passage review only involves reading/language arts. Item review involves all content areas. Bias review can be anyone who has had classroom experience.

New Achievement Contract TCAP T: Spring 2009 (Operational items assess current SPIs) TCAP A: Spring 2010 New Operational Assessment (Assess revised SPIs for RLA, Math, & Science) TBA: New Cut Scores Established (Standard Setting Required) This spring’s test still on old standards. Have a conversation with teachers concerning the pilot items and how they present this information to students. Math has some materials that is pushed down to lower grade levels. Algebra will be more present on this test in the middle grades. There are few math SPIs in new standards, but more SPIs in reading/language arts. The state is working to make the test shorter. Social studies has not been affected by new standards so test material will not change. Cut scores have not been established yet, hopefully summer 2010.

Resources Office of Assessment, Evaluation, & Research

TCAP T Time Limits Grade 3 Grades 4-8 Same as Form S (Spring 2008) SUBTEST TEST ITEMS TIME LIMIT Science, Part 1 39 55 Science, Part 2 38 Social Studies, Part 1 Social Studies, Part 2 Grades 4-8 SUBTEST TEST ITEMS TIME LIMIT Science, Part 1 39 50 Science, Part 2 38 Social Studies, Part 1 49 Social Studies, Part 2 48 New TAM will be available in January. Time limits are same as previous test. Same as Form S (Spring 2008)

State Performance Indicator Chart TCAP Achievement (3-8) State Performance Indicator Chart Spring 2009 http://www.state.tn.us/education/assessment/tsachrcspi.shtml SPIs have not chagned. A new chart will be up in the fall.

TCAP Achievement (3-8) Third Grade Pilot Pilot results are in… Results are positive! ALL students grades 3 – 8 will now use a separate answer document. ALL students (3 – 12) may mark in test booklets. Third grade will use separate answer document. They can now mark in their test booklets. Still cannot write in K-2 documents.

TCAP Achievement (3-8) Practice Answer Document Practice answer document created as a resource for teachers of grades 3-8. Designed for practice in filling in bubbles on a document similar to the TCAP Achievement answer document. Available online at: http://www.state.tn.us/education/assessment/doc/Grade3_Ansdoc.pdf

TCAP Achievement (3-8)

TCAP Achievement (3-8) Unique Identification Numbers Beginning with the Summer 2008 Gateway/EOC test administration, both the student social security number and the State unique identification number (EIS) will be captured in an effort to phase out the use of social security numbers. For one more year, social security number will be requested but not reported on any reports.

TCAP Achievement (3-8) For information about Curriculum roll-out sessions contact the Office of Curriculum and Instruction. 615/532-6300 Roll-outs are planned for summer. Feb. 8-10 in middle TN counselor conference will provide some roll-out training related to connect new curriculum, resources, assessment, and accountability. Share ways to be proactive. Sessions will be on grade span scope. Sessions will be available for each content area.

TCAP Achievement The Office of Curriculum and Instruction (TDOE) is developing companion documents to the new curriculum standards. User’s Guide to the Tennessee Mathematics Curriculum Framework Scott Eddins 615/741-3073 scott.eddins@state.tn.us This document is available online.

User’s Guide to the Tennessee Mathematics Curriculum Framework TCAP Achievement User’s Guide to the Tennessee Mathematics Curriculum Framework Overview General Organizational Format Webb Levels and the TN Curriculum Vision of Standards Based Science, Technology Engineering & Mathematics Education Implementation

User’s Guide to the Tennessee Mathematics Curriculum Framework TCAP Achievement User’s Guide to the Tennessee Mathematics Curriculum Framework Formative and Summative Assessment Mapping Webb Depth of Knowledge Levels Alignment with ACT & NAEP Comparison of Former & New Standards Vocabulary in Focus Content in Focus

State Performance Indicator Chart Spring 2009 http://www.state.tn.us/education/assessment/tsachrcspi.shtml

TCAP Achievement (3-8) Online Resources Item Samplers & Practice Tests (Current) Educators Guide to Test Interpretation (will be revised) Teacher Directions (will be revised) Frequently Asked Questions (will be revised) Item samplers still valid this year. There will be a full sampler for each content area for the new test (one item for each SPI). Practice test are printed, but that may be cut with the budget issues, and they may be only available online this year. http://www.state.tn.us/education/assessment/tsachhome.shtml

K-2 Testing Office of Assessment, Evaluation, & Research

K-2 Testing New Stanford 10 Grades K-2 “Shelf Test” (Norm-Referenced) Optional No longer use TerraNova Use Stanford Achievement Test Spring 2009 SAT 10. Kid-friendly test, colorful.

K-2 Testing New Stanford 10 Levels SESAT 2 (Kindergarten) Primary 1 (First Grade) Primary 2 (Second Grade) Three levels of test.

K-2 Testing New Stanford 10 Subtests Reading Mathematics Language Spelling Listening Science Social Studies You can give it all or choose the subtest, price is the same.

K-2 Testing New Stanford 10 Flexible time limits Content reflects national standards Full-color Easy-to-navigate Normed in 2002 Must order before Thanksgiving! You decide the time limits.

New Assessments Office of Assessment, Evaluation, & Research

New Assessments Modified Academic Achievement Standards (MAAS) English Linguistically Simplified Assessment (ELSA) Moving on with these new assessments. These are for AYP reporting. ELSA is almost identical to the TCAP test, the only difference is that items will have simplified language.

Achievement - ELSA English Linguistically Simplified Assessment (ELSA) Available for ACH - Spring 2010 Intended for Limited English Proficient (LEP) students ACH test in “simplified” English

Achievement – ELSA Characteristics of simplified language assessments Reduce wordiness Use common words Avoid words with multiple meanings Use concrete words Reduce complexity of sentence structure Use the simplest verb forms Use less complex context This is different from modified items in that modified items are completely different.

Achievement– ELSA Reduce linguistic barriers to measurement of the intended knowledge or skill The knowledge or skill being assessed must not be changed Challenging vocabulary should be used only when it is relevant to the knowledge or skill being assessed Same ACH test (e.g., large print and Braille versions) This test is to measure what they know. Reduce barriers language.

Achievement– MAAS April 2007, federal regulations finalized an option for an alternate assessment based on modified academic achievement standards Resource: National Center on Educational Outcomes (NCEO) http://cehd.umn.edu/AAMAS.html

Achievement– MAAS TDOE adopts Modified Academic Achievement Standards (MAAS) Assessment Pearson/ETS contracted to field test MAAS items in Spring 2009 MAAS to be operational in Spring 2010 MAAS becomes one component of the NCLB Adequate Yearly Progress (AYP) Will be used for AYP reporting. This is a modified test. The performance levels (below prof, prof, advanced) can be changed for this test. The names will be the same but the cut scores will be different. The content must be the same.

Achievement– MAAS Available for all 3-8 content areas (Reading and Language Arts, Mathematics, Science, and Social Studies)

Achievement– MAAS Assess same content standards as ACH Modify test format Modify test questions Modify achievement standards Modifications: 3 answer choices only, the SPI cannot be modified. Cannot make curriculum easier. If feds do not approve this test, these students must take TCAP.

Achievement– MAAS Referred to as the 2% solution Students for whom the alternate assessment is too easy Students who cannot demonstrate achievement on TN’s ACH Design an assessment with more valid score interpretation for the 2% population Portfolio is too easy for these students, but the TCAP test does not fairly measure their performance.

Achievement– MAAS Develop modified achievement standards (performance levels) For on-grade-level content standards Challenging for eligible students Assess a student’s mastery for grade-level content But at a less difficult level than grade-level achievement standards (performance levels) Who are these students? The field test will include both regular and sped students. If enough regular ed students are assessed, parental permission is not required to test sped students on the field test.

Achievement– MAAS Eligibility Guidance from State Department of Education (Special Education Division) IEP Team Content Standards (same as ACH) Achievement Standards (specific to MAAS)

Achievement– MAAS State must establish clear and appropriate guidelines for IEP teams to determine eligibility. Students may be assessed on the modified assessment in one or more subjects. Eligible students may be from any of the disability categories listed in the IDEA. For testing purposed functionally delayed is not recognized by IDEA. If they have another disability, mark sped hours for them. If they do not, you bubble functionally delayed, and they do not count in the sped population for NCLB. Functionally delayed (only) students do not take the MAAS test.

Achievement– MAAS State must ensure students’ IEPs include goals for content standards for the grade in which the student is enrolled. Students must have access to the curriculum and instruction for the grade in which the student is enrolled. Info coming in January (WebEx) on IEP teams. These students do not give up the option of a regular ed diploma because they took the MAAS.

Achievement– MAAS Students cannot be precluded from attempting to complete the requirements for a regular high school diploma. “A student’s IEP Team continues to determine how the student will participate in State and district-wide assessments.” (OESE)

Achievement– MAAS Achievement expectations may be modified to provide a better method for assessing growth over time for the 2% population. Achievement standards should promote challenging expectations for this population.

Achievement– MAAS After performance level descriptors for each achievement level are obtained, appropriate cut scores can be set for each proficiency level. Educator committees will be convened to recommend cut scores for this population on this new assessment.

Achievement– MAAS Test design changes include: Three answer option choices Shorter reading passages Reduced reading load Simplified language Decreased cognitive complexity These are changes made in the state’s review.

TCAP Achievement– MAAS Format changes include: Print styles simplified (modified use of italics, underlining, bold-facing) Charts, graphs, and tables simplified Print type size enlarged Essential information reinforced Fewer items on each page Modifications made to this test.

Pearson Access Office of Assessment, Evaluation, & Research

Pearson Access November/December 2008 TDOE will upload student data to PEMSolutions (pre-id) Spring 2009 - Testing Coordinators will use the PEMSolutions tool of Pearson Access to: manage student data (before and after testing) access reports Dec. 17 is first trial upload from EIS. Data must be clean.

Pearson Access November/December 2008 TDOE will upload student data to PEMSolutions (pre-id) Last name, First name, MI Student ID number Birthdate Gender Ethnicity Code A/B Grade These are the fields that are being uploaded. There will be no bubbles filled in on this year’s answer document. This year, there will be a bar code on the answer document which will have student’s name and ID number in order to distribute answer docs to students. Pearson access will provide an opportunity to review data. A cut-off date will be established. Username login for Pearson access will be the same, but the password will change. Answer docs will come sorted by grade level, but working with Pearon to identify by teacher name.

PearsonAccess PEMSolutions Visit the Computer Lab for a demonstration Login/Logout & Reset Password (use same passwords as order entry) Administrative Management Student Demographic & Enrollment Data Test Results Support FAQ & much more That administrative login per system.

RANDA Solutions Office of Assessment, Evaluation, & Research New data vendor. TurnLeaf data should be downloaded. Office of Assessment, Evaluation, & Research

TCAP Achievement Program Transition TurnLeaf (CTB) available through June 2009 Our online applications site is down for upgrades: https://tnors.turnleaf.com https://www.eval.k-12.state.tn.us/login

TCAP Achievement Program Transition New Tech Vendor (upgraded & enhanced) Achievement Orders RIs SGLs Training Videos https://tdoe.randasolutions.com

TCAP Achievement (3-8) Achievement Orders: Please verify all quantities (check Braille and LP). For late orders or changes contact: Freddie Summers 615/532-1249 freddie.summers@state.tn.us Order #s and EIS upload information will be used to decide how many tests to order for each school. Systems must continuously update EIS You may be contacted by Freddie Summers in relation to over orders of Braille test. The number of students in EIS will supply the number of documents, plus a modest overage, plus the Central Office box. Numbers should be accurate if info in EIS is correct.

Test Window Office of Assessment, Evaluation, & Research

Spring 2009 Test Window For the school year 2008 - 2009, school systems can now select 6 consecutive school days to administer the TCAP Achievement Test. This change has been made to accommodate school systems with large ELL and SPED populations. The six consecutive school days must fall within the state testing window. Extra day will assist ELL and SpEd population extended time accommodations.

Spring 2010 Test Window Guidance from the U.S. Department of Education concerning the reporting of AYP information states that beginning with the school year 2009-2010, the State Department of Education will be required to report AYP information no later than 14 days before the start of the school year. Since some schools (year round schools) open July 14, the state must move up the testing window.

Spring 2010 Test Window This new regulation will require reporting AYP information at least two weeks earlier than the current AYP reporting date. Since some Tennessee schools open as early as July 14, we must adjust the TCAP Achievement Test dates in order to comply with the new directive.

Spring 2010 Test Window We have drafted two possible solutions that will have the least impact on the Spring 2010 administration of the TCAP Achievement Test: Option 1: March 29 - April 9, 2010 (two-week window) Option 2: March 22 - April 9, 2010 (three-week window)

Spring 2010 Test Window Easter Sunday – April 4, 2010 Good Friday – April 2, 2010 Option 1: March 29 - April 9, 2010 (two-week window) Option 2: March 22 - April 9, 2010 (three-week window)

Spring 2010 Test Window Sun Mon Tues Wed Thurs Fri Sat March 21 22 23 24 25 26 27 28 29 30 31 April 1 2 Good Friday 3 4 Easter Sunday 5 6 7 8 9 10 2008-2009 Achievement Window March 30 – April 24

Test Security Office of Assessment, Evaluation, & Research

Test Security T.C.A. 49-1-607 states: Any person found not to have followed security guidelines… placed on immediate suspension grounds for dismissal includes dismissal of tenured employees grounds for revocation of state license

State Test Security Guidelines The Public School Systems, State Special, and Non-Public Schools MUST: 1) Adopt a locally monitored test security policy that incorporates, at a minimum, these State Test Security Guidelines. This policy should include a Testing Code of Ethics for personnel to sign and leave at the district office for documentation. Teachers are now filed suit against their districts for improper training.

Test Security 2) Train all personnel involved in the testing process on State Test Security Law, Security Guidelines, local policy, and test administration procedures; retain training documentation for system records. 3) Implement check-in, check-out, and quantity verification procedures for all test materials at the system level, at the school level, and for each test session.

Test Security 4) Restrict handling of test materials to authorized personnel at all times. 5) Implement policies and procedures to prohibit all personnel from obtaining knowledge of test items or passage content before, during, and after testing.

Test Security 6) Return test materials immediately after each test session and when the entire administration is completed. Store test materials in a centrally located locked room that is inaccessible to unauthorized persons. 7) Create a secure, yet positive, environment for testing.

Test Security 8) Conceal or remove all instructional or reference materials in the test setting that are related to the content area being assessed, such as maps, posters, student samples, bulletin board items, familiar study aids such as graphic organizers, models, or number lines that relate to subject content. Audits by the SDE will be conducted again this year. This is the most common problem…instructional items left on the wall. If in doubt, cover it up.

Test Security Any information regarding the content being measured on the test MUST be concealed during the test administration. Including but not limited to the following: Maps Charts Posters Student Work Bulletin Board Items Writing Formulas Word Lists Parts of Speech Graphic Organizers Number Lines Definitions Word Walls Displays Models

Test Security 9) Turn off all electronic communication devices (cell phones, pagers, PDAs, etc.) in the test setting. Ensure proper calculator use as outlined in the Test Administration Manual, making sure that calculators are cleared before and after administration of each test.

Test Security 10) Confirm each student is the person named on the answer document for every testing session. A photo ID may be required if administrators are not responsible for normal classroom instruction. 11) Require Test Administrators and Proctors to carefully adhere to all test administration and accommodation instructions, following appropriate schedules and time limits, outlined in all test directions.

Test Security 12) Require Test Administrators and Proctors to remain with the students and be observant and nondisruptive throughout the testing session. 13) Prohibit coaching students in any way during State assessments. 14) Prohibit reading test items and passages by anyone other than those reading to students being tested through read aloud, unless indicated in test instructions or accommodations. Read aloud issues should be addressed with extra day. Assure that everyone getting accommodations they are supposed to get. Sped teachers should understand what they are supposed to read aloud. Do not block classroom door windows. Everyone should be able to look in. Teachers should not read test book.

Test Security 15) Ensure students respond to test without assistance from anyone. 16) Ensure that test items are not reproduced, duplicated, or paraphrased in any way, for any reason, by any person. 17) Maintain confidentiality of student-specific accountability demographic information and test results at all times. Make sure you have identified version numbers. Call the state if this is not done.

Test Security 18) Document test security concerns, including missing materials, on the Breach of Testing Security Report form. 19) Failure to report a breach of security compromises the integrity of the testing process and should be treated as a breach of testing security. If you learn of a breach and do not report it, that is a breach.

Item Review Office of Assessment, Evaluation, & Research Items are scrutinized by teachers and then goes through a bias review, a Q&A review by vendors, and then another review by the State. Office of Assessment, Evaluation, & Research

Item Review Purpose: Obtain Tennessee educator input Support the content validity of the Achievement Test Two sets of committees content reviews bias and sensitivity reviews

Item Review Item Content Review Committees Item Bias Review Committees Grades 3-4 (4 content areas) Grades 5-6 (4 content areas) Grades 7-8 (4 content areas) Item Bias Review Committees Reading and Language Arts Grades 3-8 Mathematics Grades 3-8 Social Studies Grades 3-8 Science Grades 3-8

Item Development Cycle All steps in the item development cycle are completed under the direction of staff at the TDOE, and all items and test forms are approved by TDOE. LD Note to TDOE: Don’t mean to leave TDOE out of this process. Do we want to add TDOE role as separate steps on this slide in approval role? Or do you prefer as is with a speaking note to TDOE’s role.

Item Review ETS brings a range of items developed for all performance indicators. Following a structured process educators read all items individually the committee discusses items Educators mark their individual comment form with input on items

Item Review Each committee reviews all ACH items and all MAAS items for a grade span (3-4; 5-6; 7-8). Each committee is comprised of teachers of regular and special education students, as well as teachers of limited English proficient (LEP/ELL) students. ELL, SpEd, Regular Ed on Committee for review.

Item Review For each item, educators will recommend: accept “as is” accept with edits reject

Item Review Guiding Questions Does the item measure the SPI? Does the item reflect sound instructional practice at the grade level? Is the item clearly and concisely stated? Does the item have one, and only one, clearly correct answer?

Item Review Guiding Questions Does the item measure the SPI? Does the item reflect sound instructional practice at the grade level? Is the item clearly and concisely stated? Does the item have one, and only one, clearly correct answer?

Item Review Guiding Questions Are the other answer options clearly incorrect? Are all stimuli and/or graphics clear and understandable, and important to answering the item? Is the item appropriate for all groups of students?

Questions??? Contact Information Laura Atkins – Achievement (615) 253-5208 laura.atkins@state.tn.us

Questions??? Contacts Laura Atkins - Achievement Uma Natarajan – NAEP Lori Nixon – Special Ed. Assessment (portfolio/MAAS) Steven Nixon – ELDA/ELSA/ELL Assessment Marcy Tidwell – Assessment Literacy Freddie Summers – Achievement Orders Mary Taylor – GWY/EOC/ACT/Plan/Explore Charlotte Woehler – Writing 615/741-0720 Another regional testing meeting in March. Concerns: Upload from EIS