Independent Study of State End-of-Course Assessments Presentation of Findings House Education Committee February 1, 2008

Slide 2 Introduction The Washington State Board of Education (SBE) contracted with Education First Consulting to conduct an independent study of state end-of-course assessments. Our research team includes: –Jennifer Vranek, President of Education First Consulting –Jessica de Barros, Education Policy Consultant –Richard Brown, Psychometrician and Assistant Professor in the USC Rossier School of Education –Betheny Gross, Senior Research Analyst in the University of Washingtons Center on Reinventing Public Education –Christopher Mazzeo, Education Policy Consultant, Consortium for Chicago School Research

Slide 3 Our Approach Our team researched the following: What lessons can Washington State learn from the literature on end of course tests (EOCs), comprehensive assessments and exit exams? What have been the experiences of other states implementing EOCs? What alternatives measure Washingtons standards? What are the policy implications for Washingtons high school assessment system based on lessons learned from other states with EOCs?

Slide 4 Methodology First, the team conducted a thorough review of the primary and secondary literature on EOCs and comprehensive tests, as well as the use of these formats for exit exams and school accountability. Next, we researched and summarized nine states EOC programs. –CA, IN, MD, NJ, NY, SC, TN, TX and VA We simultaneously conducted 30 interviews of education, government and business leaders in seven states. –CA, IN, NJ, SC, TN, TX and VA Finally, we used our research findings to consider policy implications for Washington States high school assessment system.

Slide 5 What are EOCs? What is the WASL? End-of-course assessments are tests that: –Are given at the end of a particular course and measure the content found only in the standards for that course. –Are taken only by students taking a particular course, so not all students are assessed. The 10 th grade Washington Assessment of Student Learning in reading, writing, mathematics and science is a comprehensive assessment because: –It is given at a common point in time (to all 10 th graders) and –It measures content and skills from standards that are taught over several years, rather than in a particular course.

Slide 6 Purposes Of Assessment There are many ways to describe the purposes of statewide high school assessment. We summarize these as four main purposes: –Supporting student learning; –Holding students and/or schools accountable; –Determining readiness for postsecondary education and training; and, –Ensuring high-quality and efficient operations.

Slide 7 Assessments: Means to an End On balance, both comprehensive and EOC assessments can meet these four purposes equally well. However, each approach has different strengths and meets the purposes in different ways. The advantages and disadvantages of either test format depend on the states purposes for assessment.

Slide 8 Comparing Use Of Assessments Comprehensive assessments are more common today than EOCs… But more states are shifting to EOCs. –16 states currently use EOCs –11 additional states are planning to use EOCs

Slide 9 States with EOCs (In Place or Planned)

Slide 10 States with EOCs as Exit Exams or for School Accountability (In Place or Planned) 26 All states currently or planning to have exit exams for students: AL, AK, AR, AZ, CA, FL, GA, ID, IN, LA, MD, MA, MN, MS, NV, NJ, NM, NY, NC, OH, OK, SC, TN, TX, VA, WA 10 States currently or planning to use EOCs to hold students accountable: AR (2010), IN (2000), MD (2009), MS (2006), NY (2000), NC (2010), OK (2012), TN (2005), TX (2012), VA (2004) 3 States that will use comprehensives in English/math and EOCs in other subjects: MA (English/math 2003, science EOCs 2010, U.S. History EOC 2012) NJ (English/math 2003, Biology EOC 2010) SC (English/math 2006, Biology and U.S. History EOCs 2010) 12Use or plan to use some or all of their EOCs for school accountability under NCLB: AR, MA, MD, MS, NJ, NY, NC, OK, SC, TN, UT, VA

Slide 11 Key Findings: Comprehensive Assessments Overall, we found that comprehensive testing systems: Usually focus on 10 th grade or lower standards; Assess a slice of high school standards, rather than deep knowledge of particular subjects; Can potentially narrow the delivered curriculum to what is tested; Provide a snapshot of system performance; Often take up less testing time overall and cost less; Take a more straightforward approach to exit exams and school accountability; and Rarely provide information on students readiness for postsecondary education coursework and training.

Slide 12 Key Findings: EOC Assessments Overall, we found that EOC testing systems: Vary widely with respect to number and kinds of courses assessed; Will measure a broader and deeper range of standards, but only if there are a sufficient number of EOCs in each subject; Do not assess all students against common standards, unless states require all students to take certain courses and/or require all students to take certain EOCs; Can promote more consistency of teaching and provide more timely information on learning and course quality; Motivate students to learn through exit exams as well as other forms of lesser student stakes (e.g. counting results in course grades; Make it more complicated to hold schools accountable, yet offer the potential to produce more validity and reliability; and Can be better-suited for placing students in postsecondary education courses than comprehensive tests given by states in the 10 th grade.

Slide 13 Policy Implications for Washington Our research revealed that both comprehensive and EOC assessments have strengths and limitations, depending on the states purpose of assessment. We believe Washington policymakers must first determine what their policy priorities are for student assessment in order to choose the most appropriate testing format.

Slide 14 A Decision Framework For Policy Priorities To help policymakers compare how each test format meets various policy priorities, we developed a decision framework analyzing each type of assessment according to the four purposes of assessment: –Supporting Student Learning; –Holding Students and Schools Accountable; –Determining readiness for postsecondary education and training; and –Ensuring High-Quality and Efficient Operations. Within each issue area, we analyzed the relative advantages and disadvantages of comprehensive assessments and EOCs according to multiple criteria.

Slide 15 Supporting Student Learning

Slide 16 Holding Students & Schools Accountable

Slide 17 Determining Readiness for Postsecondary Education

Slide 18 Ensuring Quality & Efficient Operations

Slide 19 Sequencing Policy Decisions Assessments are a means to the end of understanding how well students, schools and districts are meeting state standards. It is important to consider the assessment policy within the larger context of the states K-12 education system and efforts to improve learning and teaching. Over the next several months, the statewide system of standards, curriculum, instruction, assessment and accountability is being reshaped. State policymakers should first address key questions about high school education policy and then determine which format for the high school assessments is most relevant.

Slide 20 Recommended Policy Questions What skills and knowledge do students need to be successful after exiting Washingtons K-12 public education system? How important is measuring readiness for postsecondary education? Is it important for the state to assess academic standards close to the time when students learn content or at a common point in time? When the new graduation requirements are adopted, what course credits will all students be expected to earn in English, math, science and social studies? Should the main high school assessments also be used to measure readiness for postsecondary education? Does the state want additional ways to hold students accountable, such as tying test results into course grades? Are more statewide measures needed for schools or are the four current measures sufficient for the state accountability system?

Slide 21 EOC-Specific Recommendations If, after addressing these questions, Washington policymakers consider transitioning to an EOC-based system, we recommend that policymakers: –Minimize cost and development time by working in collaboration with other states to implement standards-based, criterion-referenced assessments; –Require all students to earn a common set of course credits, and require all students to take the corresponding EOCs in these subjects, to ensure equity. –Maintain the 10 th grade WASL in reading and writing, rather than creating EOCs in these subjects.

Slide 22 Discussion

Slide 23 Back-Up Slides

Slide 24 Detailed Findings 1.Well-designed comprehensive test systems and EOC systems share many characteristics of high-quality testing systems, such as –Measuring state standards with reliability and validity. –Assessing standards with a variety of formats (multiple- choice, constructed-response). –Turning around test results in a timely manner. –Scoring costs may increase with EOCs, but this depends on number of tests/test forms, types of item formats and scoring turnaround needs.

Slide 25 Detailed Findings 2.State EOC systems vary widely on the number and kinds of courses that are assessed. –EOC systems can assess a broader and deeper range of content and skillsif there are multiple assessments in a subject area, assessments for advanced-level courses and/or multiple subject areas. –Comprehensive tests typically focus on 10 th grade standards and assess all students at a common point in time. –Most EOC systems still measure English with a comprehensive test. –Unless states require all students to pass a certain series of courses and/or require all students to take certain EOCsdifferent groups of students will be tested on different content. –E.g.: 700,000 California students took the Algebra I EOC (a course required for graduation) in 2006, but only 300,000 took the Geometry EOC.

Slide 26 Detailed Findings 3.While both types measure standards, EOCs are typically chosen by states to promote more consistency of teaching. –EOC exams are designed to measure the impact of a specific course on student learning. –Comprehensive exams are designed to measure the impact of instruction over several years on student learning. –CA, IN, TN and VA education leaders told us they administer EOCs because the standards, content, instruction, curriculum and assessment are more closely aligned than in comprehensive systems.

Slide 27 Detailed Findings 4. Changing test formats will not necessarily improve student learning of state standards or increase student performance. –Performance on EOCs and comprehensive tests varies widely. Student results on the high-stakes California and South Carolina 10th grade exit exams are much higher than on those states EOCs. –It not possible to conclude that test format type has any influence on the number or proportion of students meeting standards. Factors that impact test scores include: Content measured; Rigor and complexity of test questions; Performance standards (cut scores); Alignment of standards and local curriculum with the assessment; and Students opportunity to learn the material tested. –Most exit exam statesregardless of test formathave high levels of student performance overall (and many have closed performance gaps), especially when results from retakes are added.

Slide 28 Detailed Findings 5.Both EOCs and comprehensive assessments can be used as exit exams. In addition, EOCs can be used as part of student course grades. –A handful of statesincluding CA, IN, LA and UTcurrently attach no stakes for students to their EOCs. –Several statesincluding GA, NC, SC and soon TN and TXrequire districts to count EOC results as a portion of final course grades (usually 15-25%). –13 states currently or will require students to pass at least one EOC in order to graduate. 10 of these states use only EOCs, while MA, NJ and SC use comprehensive and EOCs. This includes IN and TX, which plan to transition away entirely from their comprehensive tests.

Slide 29 Detailed Findings 6.It can be more complicated to hold schools accountable with EOC tests. –It is more straightforward to use comprehensive tests for school accountability. –EOCs can paint a more complexand more reliablepicture of high school performance. More EOC tests will yield more data and have the potential for greater validity and reliability. 7.States can use EOCs to meet No Child Left Behind requirements in reading, math and science. –10 states currently use EOCs to determine AYP. –Two additional states are planning to use EOCs for AYP.

Slide 30 Detailed Findings 8.EOCs are better-suited to determine student readiness for postsecondary education and training than comprehensive tests given in 10 th grade. –Most states planning to use high school tests for readiness and placement purposes are moving towards EOCs, in large part because EOCs given in advanced subjects can be used more easily to assess readiness than comprehensive tests given in the 10 th grade. –E.g. CA, LA, NJ, NY, TX and TN do or plan to use EOCs to signal readiness for postsecondary education. –States also are adapting the SAT, ACT, Compass/Accuplacer to administer to all 11 th- graders.