1. Smarter Balanced Assessment Consortium Building a System to Support Improved Teaching and Learning Joe Willhoft Shelbi Cole Juan d’Brot National Conference on Student Assessment June 29, 2012

2. A National Consortium of States 27 states representing 43% of K-12 students 22 governing, 5 advisory states Washington state is fiscal agent WestEd provides project management services

3. A Next Generation of Assessments Rigorous assessment of progress toward “college and career readiness” Common cut scores across all Consortium states Provide both achievement and growth information Valid, reliable, and fair for all students, except those with “significant cognitive disabilities” Administered online Use multiple measures Operational in 2014-15 school year US Dept. of Ed has funded two consortia of states with development grants for new assessments aligned to Common Core State Standards (Federal Register / Vol. 75, No. 68 / Friday, April 9, 2010 pp. 18171-85)

4. Smarter Balanced Assessments The System Design

5. Theory of Action: Seven Key Principles 1.An integrated system 2.Evidence-based approach 3.Teacher involvement 4.State-led with transparent governance 5.Focus: improving teaching and learning 6.Actionable information – multiple measures 7.Established professional standards

6. A Balanced Assessment System Common Core State Standards specify K-12 expectations for college and career readiness Common Core State Standards specify K-12 expectations for college and career readiness All students leave high school college and career ready Teachers and schools have information and tools they need to improve teaching and learning Interim assessments Flexible, open, used for actionable feedback Summative assessments Benchmarked to college and career readiness Teacher resources for formative assessment practices to improve instruction

7. Summative Assessments for Accountability Built on solid technology Coverage of full breadth/depth of Common Core Precise assessment of all students Computer Adaptive Testing (CAT) Portion Deeper learning with thematic and scenario-based tasks Real-world problems aligned to Common Core PT scores combined with CAT for overall score Performance Task (PT) Portion K-12, higher ed, business, and policymakers part of decision-making Broad review of “Achievement Level Descriptors” Field testing includes PISA, TIMSS, NAEP items Longitudinal data systems used for triangulation Setting Performance Standards (Cut Scores) Page 7 ✔

8. Interim Assessments to Signal Improvement Non-Secure Timing and frequency are locally determined Interim test-builder creates aligned assessments Flexible Teachers can match assessments with scope and sequence Teachers can review student responses Teachers can score student responses Supports Proficiency Based Instruction Includes full range of item types Uses the same scale as the Summative Assessment Includes performance assessments Authentic Measures Page 8 ✔

9. Summative and Interim Use of Computer Adaptive Technology Turnaround time is significantly reduced Faster results Fewer questions compared to fixed form tests Shorter test length Improved measures of student growth over time Increased precision Item difficulty based on student responses Tailored to student ability Large item pool means not all students receive the same questions Greater security GMAT, GRE, COMPASS (ACT), Measures of Academic Progress (MAP) Mature technology Page 9 ✔ ✔

10. Formative Tools for Classroom- Based Assessment Practices Tools/materials for Classroom-based Assessments Fully aligned to Common Core State Standards Available for in-service and pre-service development Improving Instruction Access to the best resources available Collaborate with other states on special projects Professional social networking across the Consortium Tools to evaluate publishers’ tests Pooling Resources Page 10 ✔

11. K-12 Teacher Involvement Support for implementation of the Common Core State Standards (2011-12) Write and review items/tasks for the for the pilot test (2012-13) and field test (2013-14) Development of teacher leader teams in each state (2012-14) Evaluate formative assessment practices and curriculum tools for inclusion in digital library (2013-14) Score portions of the interim and summative assessments (2014-15 and beyond)

12. Smarter Balanced Assessments Item and Task Development

13. Item and Test Design Organize the Content Design the Items Design the Tests Write the Items Content Specifications October 2011 Item Specifications January 2012 Test Specifications February 2012 Item Writing for Pilot Test 2012-2013

14. Item Development and Research June –Cog Lab Recruiting of Locations and Students –Item Writer Recruitment –Stimulus Reviewer Confirmations July –Collect Item and Student Responses for AI Scoring Research –Cog Lab Administration –AI Scoring Engine Evaluations –Item Reviewer Recruitment

15. Item Development Cont. July cont. –Small Scale Trials Sampling Plan –Guidance for States in Procurement Study –Item Reviews August –Item Reviews for Higher Ed input September –Content, Bias/Sensitivity, and Accessibility Reviews –Small Scale Trials Administration

16. Smarter Balanced Assessments Formative Practices and Professional Learning

17. Improve Teaching and Learning Provide comprehensive support for formative assessment, including instructional modules aligned with CCSS Training modules help teachers focus their instruction on the CCSS and develop teaching practices that support more in- depth learning Enhance assessment literacy by training teachers to use formative assessment tools and interim assessment to determine next steps in instruction Provide supports for students to manage their own learning Formative Practices, Professional Learning and Implementation

18. Formative Master Work Plan IDTask Name Work Start Work Finish Professional Learning, Formative Assessment Practices, and Curriculum Resources with Technology Development for the Digital Library 1. Digital Library application, including hosting and moderated collaboration among users. Oct. 2012 Sept. 2014 2. National Advisory Panel, State Leadership Teams, and State Networks of Educators to develop Smarter Balanced quality criteria policies for Digital Library resources. Oct. 2012 Feb. 2013 3. Procedures to implement feedback cycles with State Leadership Teams and State Networks of Educators during the development process for all Digital Library resources, including continuous improvement after resources are uploaded. Oct. 2012 Feb. 2013 4. Professional learning materials on Assessment Literacy with supplemental assessment resources recommended by State Networks of Educators. Jan. 2013 Dec. 2013 5. Smarter Balanced Exemplar Instructional Modules in ELA/literacy and mathematics across grades K–12 with supplemental instructional resources recommended by State Networks of Educators. Jan 2013 May 2014 6. Training materials for all users on interpreting Smarter Balanced interim and summative assessment reports, searching Smarter Balanced resources, and using collaboration tools in the Digital Library June 2014 Sep. 2014

19. Smarter Balanced Timeline

20. Field Test 37,000 Items/Tasks Major Milestones All-Call for Pilot Testing Cognitive Labs & Field Trials IT Readiness (Round 1) IT Readiness (Round 1) Interim & Formative Available for Use Operational Summative Assessment Set Performance Standards (Cut Scores) Verify Performance Standards Pilot of 10,000 Items/Tasks Item/Task Writing Begins IT Readiness (Round 2) IT Readiness (Round 2) All-Call for Field Testing State Educator Teams Begin IT Readiness (Round 3) IT Readiness (Round 3)

21. Find Out More Find us at: SmarterBalanced.org