Exploring SBAC August 2, 2013MCLA

A Balanced Assessment System Common Core State Standards specify K-12 expectations for college and career readiness Common Core State Standards specify K-12 expectations for college and career readiness All students leave high school college and career ready Teachers and schools have information and tools they need to improve teaching and learning Summative: College and career readiness assessments for accountability Interim: Flexible and open assessments, used for actionable feedback Formative resources: Supporting classroom-based assessments to improve instruction

How SBAC is different…  Transparent test development using evidence-centered design  Uses technology to enable students to demonstrate in new ways  Common standards and common achievement levels across multiple states  Flexibility in administration for individual students  Specific knowledge and skills to be measured are identified  Available options seem to support student-centered curriculum and instruction

Summative Assessments  Replace all current Maine summative assessments in grades 3-8 & 11 in ELA and math beginning in the spring of 2014 – last 12 weeks of school. There will be an 18 month gap between next fall’s NECAP and SBAC.  Individual students can do the test in segments, if necessary, and can repeat it during the 12 week window.  Testing time is 8 hours total, which includes 1.5 – 2 hour performance task.  Accommodations are built into test items and keyed by individual student needs.  Items are adaptive within each grade level, but not across grade levels.  Scoring will account for adaptive items, but exactly how this is reported hasn’t been determined yet.  Descriptors at each score point have been identified, but cut scores won’t be determined until after 2014 administration.  Scores available 2 weeks after administration for summative assessment.  Determination of “college-ready” at grade 11 & agreements with higher ed.

College Content-Readiness Definition English Language Arts/Literacy 3 “Students who perform at the College Content-Ready level in English language arts/literacy demonstrate reading, writing, listening, and research skills necessary for introductory courses in a variety of disciplines. They also demonstrate subject-area knowledge and skills associated with readiness for entry-level, transferable, credit-bearing English and composition courses.”

Mathematics “Students who perform at the College Content-Ready level in mathematics demonstrate foundational mathematical knowledge and quantitative reasoning skills necessary for introductory courses in a variety of disciplines. They also demonstrate subject-area knowledge and skills associated with readiness for entry-level, transferable, credit-bearing mathematics and statistics courses.”

Policy Framework Grade 11 College Content-Ready Assessment Results Not Yet Content-Ready - Substantial Support Needed K-12 & higher education may offer interventions Level 1 Not Yet Content-Ready – Support Needed Transition courses or other supports for Grade 12, retesting option Level 2 Conditionally Content-Ready/Exempt from Developmental In each state, K-12 and higher ed jointly develop Grade 12 requirements to earn exemption Level 3 Content-Ready/Exempt from Developmental In each state, K-12 and higher education may jointly set Grade 12 requirements to retain exemption Level 4

Assessment Types  Selected Response Items  Constructed Response and Extended Response Items  Technology- Enhanced Items  Performance Tasks

Selected Response Items Selected Response Items (SR) contain a series of options from which to choose correct responses. Items: Reflect important knowledge and skills consistent with the expectations of the CCSS across the Depths of Knowledge (i.e., Recall/Literal Comprehension, Interpretation/Application, and Analysis/Evaluation). Allow students to demonstrate their use of complex thinking skills, such as formulating comparisons or contrasts; identifying cause and effects; identifying patterns or conflicting points of view; categorizing, summarizing, or interpreting information.

Constructed Response and Extended Response Items Constructed Response (CR) is a general term for items requiring the student to generate a response as opposed to selecting a response.  Short constructed response items may require test-takers to enter a single word, phrase, sentence, number, or set of numbers, whereas extended constructed response items will require more elaborated answers and explanations of reasoning.  comparisons or contrasts; proposing cause and effects; identifying patterns or conflicting points of view; categorizing, summarizing, or interpreting information; and developing generalizations, explanations, justifications, or evidence-based conclusions

Technology- Enhanced Items Technology-Enhanced Items employ technology to:  Elicit a response from the student (e.g., selecting one or more points on a graphic, dragging and dropping a graphic from one location to another, manipulating a graph)…, and/or  Employ technology to assess content, cognitive complexity, and Depth of Knowledge not assessable otherwise. Items include: drag and drop hot spot drawing graphing gridded-response items simulation technologies use of online tools

Performance Tasks Measure the student’s ability to integrate knowledge and skills across multiple standards and to better measure capacities such as depth of understanding, research skills, and complex analysis, which cannot be adequately assessed with selected response or constructed response items. Performance tasks:  Measure capacities such as depth of understanding, research skills, and/or complex analysis with relevant evidence;  Require student-initiated planning, management of information and ideas, and/or interaction with other materials;  Require production of more extended responses (e.g., oral presentations, exhibitions, product development), in addition to more extended written responses that might be revised and edited;  Reflect a real-world task and/or scenario-based problem;  Lend itself to multiple approaches;  Represent content that is relevant and meaningful to students;  Allow for demonstration of important knowledge and skills, including those that address 21st century skills such as critically analyzing and, synthesizing media texts;  Focus on big ideas over facts;  Allow for multiple points of view and interpretations.

Performance Tasks  Integrate knowledge and skills  Measure understanding, research skills, analysis, and the ability to provide relevant evidence  Require student to plan, write, revise, and edit  Reflect a real-world task  Demonstrate knowledge and skills  Allow for multiple points of view  Feasible for classroom environment

General Specifications for Performance Tasks  Allow teacher and peer interactions and group work  Organization of complex task directions  Simulated Internet access  Rubrics

Options  Interim assessments 3-8 and 9, 10, & 12 Can be used for any number of students Reported on the same scale as the summative assessment, Schools will have the flexibility to assess small elements of content or the full breadth of the Common Core State Standards at locally-determined times throughout the year.Common Core State Standards  Digital Library Resources, including formative assessments, based on an accepted formative assessment cycle.

Formative Process

Evidence-Centered Design Modern Approach to Designing Items and Tasks Traditional Item Development versus Evidence-Centered Design

Sampling: Traditional Approach to Item Development 1.2.1Apply reference skills to determine word meanings Understand and apply new vocabulary Know common sight words appropriate to grade-level Apply fluency to enhance comprehension Apply different reading rates to match text Understand how to use questioning when reading Understand how to create mental imagery Apply comprehension monitoring strategies before, during, and after reading: use prior knowledge/schema Apply comprehension monitoring strategies before, during, and after reading: monitor for meaning, create mental images Apply vocabulary strategies in grade level text Understand and apply content/academic vocabulary Understand and determine important or main ideas and important details in text Apply comprehension monitoring strategies before, during, and after reading: predict and infer Apply comprehension monitoring strategies during and after reading: summarize informational/expository text and literary/narrative text

Why evidence-centered design?  Inconsistency between what performance level descriptors said students were able to do and what they were actually able to do based on further investigation.  So, while the content of an item might be aligned with a content standard, the information elicited by the item may not say anything meaningful about whether or not the student has achieved the standard.

Traditional Approach to Item Development Item: Beth says that = 6. Explain why Beth is correct.

Traditional Approach to Item Development Item: Beth says that = 6. Explain why Beth is correct. Content Standard 2.2.3: Perform addition accurately for single and two digit numbers. Do we know if the student can actually add single and two digit numbers?

Applying Evidence-Centered Design to Item and Task Development Beth says that = 6. Explain why Beth is correct = ____ Content Standard : Perform mathematical operations and justify solutions. Content Standard : Perform addition accurately for single and two digit numbers.

Using Evidence-Centered Design to Guide Item Design 1.What evidence is required given the assessment target I am measuring? 2.What are the key features that must be included in the item? 3.Will this item allow for the production of the evidence I am seeking? 4.Is there anything about this item that may make it more difficult to collect evidence from some students?

6 Key Components of Evidence-Centered Design 6. Develop Items or Performance Tasks 1. Define the domain 2. Define claims to be made 3. Define assessment targets 4. Define evidence required 5. Develop Task Models