Quality of New Assessments By Mark Teoh, Susan Volbrecht and Michael Savoy, Winter 2015 Funded by The Leona M. and Harry B. Helmsley Charitable Trust

Many Caught off Guard by the Board Public Dialogue about CCSS Standardized Testing Has Provoked Both growing interest and pushback from parents, educators, and policymakers. Teach Plus—an organization aiming to reduce teacher attrition—surveyed more than 1,000 teachers about the Partnership for Assessment of Readiness for College and Careers (PARCC) test after a day of professional development on the assessment. Teach Plus found that most teachers believe the PARCC assessments are better than their states’ previous tests and that they measure skills that students need to be ready for college and careers. However, teacher responses were mixed on whether the PARCC test is appropriately calibrated to each grade.

Testing in states are administering the new Common Core State Standards (CCSS)– aligned consortia tests this spring 15 SBAC 13 PARCC 22 states are using non-consortia tests

SBAC and PARCC States 15 SBAC States 13 PARCC States 1.Arkansas 2.Colorado 3.District of Columbia 4.Illinois 5.Louisiana 6.Maryland 7.Massachusetts 8.Mississippi 9.New Jersey 10.New Mexi9co 11.New York 12.Ohio 13.Rhode Island

Testing the Test Events Events were open to all public school teachers and were held in Boston, Chicago, Nashville, Memphis, and DC from September to November 2015 Of the 1014 survey respondents 50% elementary school 31% middle school 19% high school Respondents taught the following subjects 38% were elementary generalists 23% ELA 18% math 4% science 17% other

Testing the Test - Events The events consisted of three 2-hour sessions that provided training around the common core and the PARCC assessment items. Survey data was collected from the teachers afterwards While 66% of teachers had been PARCC sample questions in the past, just 14% had field tested PARCC items with their classes during the prior Spring semester.

Key Research Findings

1. Quality Compared to Previous Test

2a. Alignment of Test with CCSS

2b. Test Measures What’s Taught More teachers think that PARCC is aligned to CCSS than think that it accurately measures what is taught in their classroom. One possible explanation is that there could be some teachers who are not yet fully teaching a CCSS-aligned curriculum. This could be due to curricular choices made by the district or state, lack of great materials, professional development, or time to implement a new curriculum.

3a. Test Measures College and Career Readiness Does the test require students to demonstrate knowledge? 66% felt that the test did “extremely well” or “very well” 6% responded with “not very well” or “not well at all. Is the test appropriately rigorous for grade level? 48% responded that PARCC was “extremely well” or “very well” appropriately rigorous for the grade level 17 % said it was “not very well” or “not at all well”.

3b. Appropriate Level of Rigor? Did the test assess an appropriate depth of student knowledge 49 % believed that the depth of student knowledge PARCC assessed with regard to Common Core was appropriate Almost as many believed it was assessing too deep a depth of student knowledge

3c. Measuring Appropriate Depth of Knowledge To better understand if this issue of difficulty was consistent across both subject areas, we examined the issue of rigor in teachers’ specific feedback on ELA and math questions

4. ELA Test Aligned with CCSS FINDING #4: Teachers find the English Language Arts (ELA) assessment strongly aligned to the key instructional shifts of the CCSS but may require knowledge students don’t yet have. Nearly all participants found that the PARCC passages were better quality than the passages in state tests there was some concern students did not have “background knowledge, nor the vocabulary to understand” vocabulary within the texts to assess students as accurately as possible, some portions may need to be edited for diverse learners, or those with limited background knowledge of certain content areas. There was concern about the greater focus on higher-order thinking when it comes to using the data to support all learners. They felt that an implication of this focus on higher-order thinking is that it may be harder to acquire actionable data for below level students, those with special needs, or English Language Learners.

Teachers would like to see Some portions of the test edited in order to assess diverse students as accurately as possible After the assessment is completed, provide support for teachers to effectively utilize the data for all students In summary, teachers found: the questions and passages to be high-quality with an appropriate variety of question types though some teachers’ comments suggested that the questions were possibly too difficult

FINDING #5: Teachers find the math questions cognitively demanding and balanced among concepts, procedures and application, though they wanted to see additional questions that increase the assessment rigor. Common Core goals in math most participants felt that the sample questions focused strongly on the “major content” needed for success in later mathematics and that they “strongly connect previous and upcoming grade level skills.” Participants appreciated the increased focus on this content in the assessment Most participants reported that there was an appropriate balance of concepts, procedures, and application, however they wanted to see even more application questions on the assessment. While some questions were adequately assessed procedurally, participants seemed to highly value application of content, which is consistent with the CCSS’ call for more real-world application in mathematics instruction. 5. Math Test Aligned with CCSS Shifts

Teachers would like to see More modeling and mathematical arguments in order to connect practice to content. More concept and application questions to help increase the rigor of the question. Add a “why” justification to help increase the rigor of the question. In summary, teachers were highly positive about the PARCC assessment when reviewed through several different facets of the assessment, most importantly the focus and balance of concepts, procedures, and applications as well as the overall cognitive demand

Moving Forward Teachers would like to see 1.Additional sample questions 2.Professional development on PARCC 3.Resources 4.The ability to monitor student progress through formative assessments 5.Access to better technology 6.Time to collaborate