1. The State of the State TOTOM Conference September 10, 2010 Jim Leigh Office of Assessment and Information Services Oregon Department of Education

2. Test Administration Manual www.ode.state.or.us/go/tam The Test Administration Manual contains important information that must be used for valid administration of the Oregon Statewide Assessments. Parts I – VIII of the Test Administration Manual contain test administration and security policies. Specific procedures and guidelines for administering the mathematics assessment are contained in Appendix D located at the end of the manual. Topics include: Allowable Resources Manipulatives Tools Calculators (Online in 2010-11) Accommodations (Read Aloud) The manual has been further revised for 2010-2011 and is posted on the ODE web site.

3. Timeline for Assessment of New Grade 3-8 Content Standards (www.ode.state.or.us/go/mathematicsassessment ) Extensive feedback to the Spring 2008 survey was received: About 75% of the responding districts indicated that all students would have had an opportunity to learn all of the new grade 3-8 standards by 2009-2010. However, 25% had concerns that some grades (especially 3rd) may not have had the opportunity to learn the entire depth and breadth of the new standards. As a result of this input: For 2010-2011, the 2007 Grades 3-8 and 2009 High School Mathematics Content Standards will be operationally assessed Only the “overlap” standards will be assessed in 2010-2011 The 2009 High School Content Standards were adopted June 9, 2009 and Majority of 2009 HS Standards overlap the 2002 Standards All of the 2007/2009 Mathematics Content Standards will be assessed beginning 2011- 2012 (ODE will field test items on “new” content in 2010-2011).

4. Achievement Level Standards ODE conducted an Achievement Standards Verification Workshop to establish achievement standards (i.e., cut scores) August 9-11. (http://ode.state.or.us/teachlearn/standards/contentperformance/recommended2010- 11mathstandards.pdf) (http://ode.state.or.us/teachlearn/standards/contentperformance/recommended2010- 11mathstandards.pdf) These cut scores will be presented to the State Board of Education for adoption in October, and will be in effect for the 2010-11 testing year. Achievement Standards are based on the Achievement Level Descriptors, which were developed beginning in December 2009. They describe what students are able to do at each of four achievement levels. They are stated in terms of the content standards.

5. Summary of Achievement Standards Verification Necessitated by change to new content standards and required changes in the state assessment Recognized that cut scores need to be modified to help ensure that students who meet the standard at each grade level are on a path to success in meeting the high school cut scores. Based on ODE’s analysis of national data (National Assessment of Educational Progress or NAEP), the current cut scores are below the more rigorous achievement standards of other states in the nation The Common Core State Standards are scheduled for assessment in 2014-15, with higher cut scores expected. Panelist were given cut scores of other states and from the NAEP and PISA tests, as well as projected cut scores based on ODE’s statistical analysis.

6. Summary of Achievement Standards Verification 10-12 panelists for each grade level 4, 6, 8, and HS came to consensus  Used Ordered Item Booklets for bookmarking  4 rounds of bookmarking to consensus  Then proposed cut scores for grades 3, 5, and 7 were determined using a commonly used regression strategy for smoothing data  Changes to cut scores using recommendations from the panel: A survey gathering structured input on these recommendations will be available through a link in the September 8th Assessment and Accountability Update. (http://www.ode.state.or.us/wma/teachlearn/testing/asmtacty_update_9_1_10.pdf)http://www.ode.state.or.us/wma/teachlearn/testing/asmtacty_update_9_1_10.pdf Grade 3Grade 4Grade 5Grade 6Grade 7Grade 8HS Nearly Meets +4 +5+6+7+5+1 Meets +7 +6 +4+1 Exceeds +2 +5 +4 +5

7. Test Specifications and Blueprints http://ode.state.or.us/go/mathematicsassessment http://ode.state.or.us/go/mathematicsassessment ODE has posted drafts of Content Pages, Formula Sheets, Achievement Level Descriptors, and Scoring Guide. The complete Test Specifications and Blueprints documents will be posted in the Fall, and updated when cut scores have been adopted by the State Board.

8. 2009 Item Writing for Assessment of New Math Content Standards About 3000 multiple choice items were developed in July-November 2009 Approximately 1500 items were field tested in January-May 2010 Approximately 1500 more will be field tested in January-May 2011 Additionally, ODE began development of computer scored open response items (GRID Items). Allow assessment in ways not possible with multiple choice alone Able to have students construct, draw, and move objects on screen – only beginning to tap into potential 26 Oregon-developed items were field tested in April-May 2011 (More development planned for the future.)

9. Work Samples and Scoring Guides www.ode.state.or.us/go/worksamples www.ode.state.or.us/go/worksamples More comprehensive guidelines in Appendices L, M, and N of Test Administration Manual As of June 2009, the graduating class of 2014 will be the first students required to demonstrate proficiency in the Essential Skill of “Apply Mathematics in a Variety of Settings” for the purpose of receiving a high school diploma. The State Board of Education cautioned districts to continue assessing student proficiency in this Essential Skill in the 2009-10 school year. Work Samples as Option for Fulfilling the Essential Skills Diploma requirement for class of 20 14 : Must Meet Standard on both work samples (score of 4 on each of the five process dimensions ) Must be scored with the Official State Mathematics Problem Solving Scoring Guide

10. Work Sample as Local Performance Assessments to Demonstrate Opportunity to Learn: Must complete one local performance assessment in mathematics problem solving per year for grades 3 through 8 and once in high school A work sample scored with the Official State Mathematics Problem Solving Scoring Guide may be used to satisfy this requirement. In place of a work sample scored using an official state scoring guide, districts may also satisfy this requirement using a locally developed comparable measure. Problem Solving Resources for teachers were added to the ODE website in August, 2009, and will be updated for Fall 2010. Additional tasks for use as work samples Additional scored anchor papers for each grade Work sample tasks and anchor papers will be aligned to 2007 Grades 3-8 Content Standards and 2009 High School Content Standards for 2010-2011. Contact information for districts needing training in the use of Scoring Guide and Work Sample Tasks

11. Additional Ways for Students to Demonstrate Mathematics Proficiency http://www.ode.state.or.us/news/announcements/announcement.aspx?=4883 Students in the Class of 20 14 and beyond are required to demonstrate proficiency in Essentials Skills as part of attaining an Oregon Diploma. On May 14th, 2009, the State Board of Education adopted achievement levels (“cut scores”) for tests which may be used by students to demonstrate their proficiency in the Essential Skills of Reading and Applying Mathematics. ACT (19) PLAN (19) Work Keys (5) Compass (66 – Intermediate Algebra Test) ASSET (41 – Intermediate Algebra Test) The adopted scores represent a rigorous standard similar to the level of achievement required to earn a score of 236 on the Oregon Assessment of Knowledge and Skills (OAKS) tests. recommended by the Assessment of Essential Skills Review Panel (AESRP), a group appointed by Superintendent Susan Castillo to advise the Department of Education and the State Board on issues related to the Essential Skills. AESRP-approved criteria for other local performance options are pending.

12. New Documents! Use “/go/math” (http://www.ode.state.or.us /go/math) Then select “Assessment” and find: Test Specs and Blueprints Content Pages Formula Sheets (3-5) (6-7) (8-11) Achievement Level Descriptors Score Reporting Categories Scoring Guide Work Sample Resources Pages

13. Questions? Mark R. Freed Mathematics Education Specialist Oregon Department of Education Mark.Freed@ode.state.or.us Jim Leigh Mathematics Assessment Specialist Office of Assessment and Information Services Oregon Department of Education James.Leigh@state.or.us James.Leigh@state.or.us