Presentation on theme: "Sample Items and Performance Tasks"— Presentation transcript:
1 Sample Items and Performance Tasks Smarter BalancedSample Items and Performance TasksOctober 11, 2012OSPI – Assessment and Student Information
2 Today’s Topics Website and teaser item CCSS context for these items and Smarter Balanced testsNavigating Sample Item webpageResources for reviewingCustomer supportFAQSmarter Balanced timelineWelcome to our introduction of sample items and performance tasks that have been developed by the Smarter Balanced Assessment Consortium to assess the ne Common Core State Standards.Smarter Balanced released these items on Tuesday so some of you may have already explored the site, but we wanted to do today’s webinar to assist DACs and CCSS leads in reviewing it so you can share it with your colleagues.We want to begin today with the website and an item or two but then do need to take a step back to set the context of these items with respect to CCSS. I will be passing the mic to several folks here with me today, from OSPI’s assessment division and Teaching an Learning division.
3 Sample Items Smarter Balanced Website Link is toward bottom of page
6 Purpose of Sample Items and Performance Tasks Demonstrate rigor and complexity of ELA/literacy and mathematics itemsShowcase variety of item types:Selected responseConstructed responseTechnology enhancedPerformance tasksHelp teachers continue planning shifts in instruction related to Common Core State Standards (CCSS)The sample items and performance tasks represent a milestone in the development of the assessment system. For the first time, teachers, policymakers, and interested stakeholders can see what the new assessments will look like.The sample items represent the first of many steps to help familiarize teachers, students, and parents with the expectations of the Common Core State Standards and next-generation assessments.The sample items illustrate the knowledge and skills students will be expected to demonstrate on the Smarter Balanced assessments, giving educators clear benchmarks to inform their instruction.In addition, the sample items showcase the variety of item and task types under development by Smarter Balanced.Selected response: Prompt students to select one or more responses for a set of optionsConstructed response: Prompt students to produce a text or numerical response in order to collect evidence about their knowledge or understanding of a given assessment targetTechnology enhanced: Take advantage of computer-based administration to assess a deeper understanding of content and skills; collect evidence through a non-traditional response type, such as editing text or drawing an objectPerformance tasks: Measure capacities such as depth of understanding, research skills, and complex analysis, which cannot be adequately assessed with selected- or constructed-response itemsIt is important to note that these samples represent only a small fraction of the 10,000 items and tasks currently in development to support the Pilot Test in early 2013.While the items are not intended to be used as sample tests, educators can use the items to begin planning the shifts in instruction that will be required to help students meet the demands of the new assessments.To talk about those shifts and set the context of these items with respect to the CCSS, I’d like to pass the mic to Greta Bornemann, OSPI’s Mathematics Director.
7 Career and college ready learning expectations for k-12 All students leave high school college and career readyVisionPurposeCore ValuesEvery WashingtonStudent and EducatorOur Vision: Every student will have access to the CCSS standards through high quality instruction aligned with the standards every day; andevery educator is prepared and supported to implement the standards in their classrooms every day.Our Purpose: To develop a statewide system with resources that supports all school districts in their preparation of educators and students to implement the CCSS.
8 Common Core State Standards (CCSS) Implementation Timeline Phase 1: CCSS ExplorationPhase 2: Build Awareness & Begin Building Statewide CapacityPhase 3: Build State & District Capacity and Classroom TransitionsPhase 4: Statewide Application and AssessmentOngoing: Statewide Coordination and Collaboration to Support Implementation
9 Smarter Balanced Assessment System Components Summative assessmentsBenchmarked to college and career readinessTeachers and schools have information and tools they need to improve teaching and learningCommon Core State Standards specifyK-12 expectations for college and career readinessAll students leave high school college and career readyTeacher resources forformative assessment practicesto improve instructionInterim assessments Flexible, open, used for actionable feedback
11 Key features of Sample Item Tool Mathematics and English Language Arts/Literacy itemsComputer Adaptive Testing items and Performance Tasks:Selected responseConstructed responseTechnology enhancedMeta-data for each itemOn the spot scoring for many itemsItems and tasks will be similar for summative and interim assessments
12 Sample Items and Tasks Navigation View English Language Arts/Literacy or Mathematics itemsAdvance to next item, or go back to previousUsers can select mathematics or ELA/literacy items and cycle through them using the next and back buttons on the top right corner of the screen.
13 Claims for the ELA/Literacy Summative Assessment “Students can demonstrate progress toward college and career readiness in English Language arts and literacy.”Overall Claim for Grades 3-8Overall Claim for Grade 11“Students can demonstrate college and career readiness in English language arts and literacy.”Claim #1 - Reading“Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.”Claim #2 - Writing“Students can produce effective and well-grounded writing for a range of purposes and audiences.”Claim #3 - Speaking and Listening“Students can employ effective speaking and listening skills for a range of purposes and audiences.”Claim #4 - Research/Inquiry“Students can engage in research and inquiry to investigate topics, and to analyze, integrate, and present information.”
14 Sample Items and Tasks Navigation Content ClaimGrade band
15 ELA Comparison Washington State & Smarter Balanced SimilaritiesDifferencesMultiple Choice/Selected ResponseShort Answer/Constructed ResponseOnlineEssay WritingComputer AdaptiveListening ItemsText ComplexityBrief Write, Revise and EditPerformance Tasks
16 Technology Enhanced Vocabulary Grade 4 Item For Smarter Balanced, students may be asked to select more than one phrase to support the meaning of a word as it is used in context. Students are now evaluating multiple sentences within a paragraph as opposed to three to four options presented with the stem. This adds a higher level of rigor than our current vocabulary items.
17 Measurements of Student Progress Reading Vocabulary Item This is an example of a current Washington State vocabulary item. Currently, all Washington State vocabulary items are in a multiple-choice format.
18 Listening TaskThis is an example of a listening item. After listening to the audio, students will answer a set of items.
19 Item Metadata About this item Note the Common Core standards connected to this targetEvidenceEach item also includes metadata:Item nameGrade levelContent claimAssessment targetsCommon Core State StandardsEvidence to show how the student demonstrates understandingScoring rubrics (downloadable PDF)Text complexity analysis for ELA/literacy items (downloadable PDF)View the rubricAccess information on text complexity
20 Item Rubric Item Scoring Rubric This is an example of a Smarter Balanced rubric for a reading constructed response item.Scoring Rubric
21 Text Complexity Analysis The PlacematThe placemat is a tool used to organize quantitative and qualitative data to identify the recommended placement of the reading stimulus. Text complexity placemats are available for the three sample texts. Most teachers are familiar with quantitative measures (Flesch-Kincaid and Lexile are 2 examples). Qualitative measures are determined using a matrix of qualitative criteria. The blank placemat template and the qualitative matrices can be found on the Smarter Balanced website.
22 Brief Write with Text Evidence This is an example of a brief write item. The item has been written with text evidence from both sides of the argument embedded. The item asks students to write a paragraph (considered a brief write) using evidence from the text stimulus to support their position. Different writing sample items ask students to revise or edit text.
23 High School Proficiency Exam Writing Grade 10 Persuasive Argumentative Writing Experience-based persuasive promptCurfewsCommunity officials have proposed that individuals under the age of 18 cannot be out after 9:00 p.m. unless they are with an adult. Take a position on this proposal. Write a multiple-paragraph letter persuading community officials to support your position.
24 Smarter Balanced Grade 11 Performance Task Smarter Balanced – Nuclear PowerSource-based performance task20-minute classroom activity (accessibility)Part I: Research and evaluate sources (take notes and answer questions)Part II: Write argumentative essay citing evidence from sources
32 Claims for the Mathematics Summative Assessment Overall Claim for Grades 3-8“Students can demonstrate progress toward college and career readiness in mathematics.”Overall Claim for Grade 11“Students can demonstrate college and career readiness in mathematics.”Claim #1 - Concepts & Procedures“Students can explain and apply mathematical concepts and interpret and carry out mathematical procedures with precision and fluency.”Claim #2 - Problem Solving“Students can solve a range of complex well-posed problems in pure and applied mathematics, making productive use of knowledge and problem solving strategies.”4 Claims in mathematics. Claim 1 is mathematical content, and Claims 2-4 incorporate the mathematical practices.Claim #3 - Communicating Reasoning“Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.”Claim #4 - Modeling and Data Analysis“Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.”
33 Smarter Balanced Grade 5 Item In the Common Core State Standards, fifth grade students have two standards specifically addressing whole number multiplication and division. They are:5.NBT.5 Fluently multiply multi-digit whole numbers using the standard algorithm.5.NBT.6 Find whole-number quotients of whole numbers with up to four-digit dividends and two-digit divisors, using strategies based on place value, the properties of operations, and/or the relationship between multiplication and division.In addition, students must understand the patterns in the number of zeros when multiplying and dividing by powers of 10 (5.NBT.2). Likewise, they have been working with equations for a number of years.This 1-point item requires that a student have a full understanding of all of these standards and the student would benefit from the good number sense that would bring.In part A, a student may be able to do this work mentally, illustrating that they understand division by 10 and simple multiplication. If not, they could work both sides of the equation, check for equality, and answer True or False.In parts B and C, number sense would be handy. Dividing 2487 by 3 would be about 800. With this knowledge and realizing the left side of the equation is much greater than 800, B is False. Approximately 4000 x 7 is close to 28,000 so C is also False. If students don’t have this number sense, they could work each problem and check for equality and if their work is correct, they would see these are both False.In part D, however, a student would have to do this full multiplication to check for equality. They cannot make a calculation error as they must answer True or False to the item.It takes correct responses of True or False on all parts of this item to be correct. This item adequately explores these standards and the student’s ability to fluidly move between them.As compared to a current Washington item in the next slide.
34 Measurements of Student Progress Grade 5 Mathematics Item Find the quotient. 9,018 ÷ 3 What is the quotient? A. 36 B. 306 C. 3,006In Washington State, in fourth grade students delve deeply into multiplication and then develop division in fifth grade so a division item was chosen for this comparison. The particular PE is 5.1.C: Fluently and accurately divide up to a four-digit number by one- or two-digit divisors using the standard long-division algorithm.We sometimes assess this PE as a completion item but for comparison purposes a multiple-choice item was selected. This item only covers one part of that PE (a four-digit number divided by a one-digit divisor). We also use misconceptions in Washington State as distractors and provide the correct answer as one of the options. Students usually struggle with the place position of zeros in this type of problem, hence the choice of answer options. We do not do true/false questions as in the previous item. Therefore, a student can do the work and, if the answer calculated is not present, can rework the problem to try again. There is a potential self-correction process in these types of standards in WA.Good number sense would help a student with this answer as well because of the focus on misconceptions but, if they get it correct, it still may not tell us whether the student is truly fluent in the PE being assessed. Because this is only part of the PE, one cannot generalize on a larger scale about the student’s fluency with division.
35 Smarter Balanced Grade 8 Item CCSS 8.EE.7a – Solve linear equations in one variable. a. Give examples of linear equations in one variable with one solution, infinitely many solutions, or no solutions. Show which of these possibilities is the case by successively transforming the given equation into simpler forms, until an equivalent equation of the form x = a, a = a, or a = b results (where a and b are different numbers). b. Solve linear equations with rational number coefficients, includingequations whose solutions require expanding expressions using the distributive property and collecting like terms.In the Common Core standards, grade 6 students generate equivalent algebraic expressions, in grade 7 these are expanded to include expressions with rational coefficients, and in grade 8 students use earlier strategies to solve increasingly complex equations.In this 1-point item, the focus is on part a of the standard, though students would have to use skills from part b to “simplify” the equation into a form where a decision about the nature of the solutions can be made. The student must show that they understand the procedures to solve one-variable equations, but also interpret the “simplified” versions of the equations. Rather than just working through a series of procedures to combine terms, this item requires that a student make decisions about the end result of those procedures. For example, in the first equation, when the student comes to 36x = 36x or 24=24 or 0=0, do they know what that means about the number of solutions that exist for the equation? Likewise, though there is likely little “work” that the student would do to “solve” the second equation, they have to consider what the equation 0=1 means if they do subtract x from both sides. If the student guesses, they have a 1 in 27 chance of being right.
36 Smarter Balanced High School Item Similar to the short answer problems in WA state – this is a constructed response (CR).This item demonstrates how students use data to make decisions. The student should find the slope of each segment of the line to determine where the profit per dollar exceeds money spent on advertising.Mathematical Content: Slope of a line, reading a graph, interpreting data, mathematical reasoning
37 Mathematics End of Course Exam Item The cylinder shown has a volume of 36 cubic feet.What is the height of the cylinder?2 ftG.6.C – Apply formulas for surface area and volume of three-dimensional figures to solve problems.In this problem, the student is required to use the formula for volume of a cylinder and the volume given to solve for the height. The answer can be given in terms of pi.
38 Smarter Balanced High School Item Claims 2A & 2B – Apply mathematics to solve well-posed problem arising in everyday life, society, and the workplace & Select and use appropriate tools strategically.The student must use the volume from a rectangular prism and the formula for a cylindrical tank to determine the possible radius.The cognitive complexity is increased for this item as the student needs to transfer the volume from one-shaped container into another. Also, multiple approaches are possible. Different portions of the water can be transferred and the radius determined at many points.
39 Item ScoreSelected response and technology enhanced items are machine scorableSelected response and technology enhanced items can be scored automatically.Some constructed-response items and performance tasks can be scored automatically; many will be hand-scored by professionally trained readers.
40 Sample Items and Tasks Navigation Filter by item type, themesSample items can be filtered by item type (technology enhanced and performance tasks) and by themes (connections across grades and difficulty progressions).Connections across grades: The Common Core State Standards include a sequence of concepts that build through the grades in a logical and coherent fashion. These sample items show how the vertical articulation of content in the standards will be evident in Smarter Balanced assessments.Difficulty Progressions: Computer adaptive tests require items that span a wide range of difficulty levels to provide precise measures of what students know and can do. These sample items show the range in difficulty for items designed to measure the same assessment targets. Capturing the full range of item difficulty across assessment targets is essential for creating high quality adaptive tests, and final difficulty estimates for items will be validated through field testing.
41 Sample Items and Tasks Navigation Filter by item type, themes
42 Feedback and Support available through first week of November Online feedback and phone support availableSmarter Balanced welcomes feedback on the sample items and tasks.Users will be able to submit feedback or questions through an online form or by calling a number.
43 http://www.smarterbalanced.org/sa mple-items-and-performance-tasks/ Additional Resourcesmple-items-and-performance-tasks/FAQWashington State Lead:The Smarter Balanced assessment system will provide valid, fair, and reliable measures of achievement and growth for English language learners and students with disabilities. The sample items are displayed in a simulated test platform that does not include accessibility and accommodations tools that will be available when the assessments are administered to students—such as Braille, translation options, and the ability to change font size, highlight text, or magnify portions of items.The operational system in the school year will include tools to address visual, auditory, and physical access barriers—as well as the unique needs of ELLs
44 Accessibility and Accommodations Sample items do not include accessibility and accommodations featuresFull range of accessibility tools and accommodations options under development guided by:Magda Chia, Ph.D., Director of Support for Under-Represented StudentsAccessibility and Accommodations Work GroupStudents with Disabilities Advisory CommitteeChair: Martha Thurlow (NCEO)English Language Learners Advisory CommitteeAccessibility & Accommodations FrameworkLearn more online:represented-students/The Smarter Balanced assessment system will provide valid, fair, and reliable measures of achievement and growth for English language learners and students with disabilities. The sample items are displayed in a simulated test platform that does not include accessibility and accommodations tools that will be available when the assessments are administered to students—such as Braille, translation options, and the ability to change font size, highlight text, or magnify portions of items.The operational system in the school year will include tools to address visual, auditory, and physical access barriers—as well as the unique needs of ELLs
45 Smarter Balanced Items Developed with Educators and Other Experts Early 2012: Assessment claims for ELA/literacy and mathematics approved by Governing StatesApril 2012: Item/task specifications and review guidelines publishedJune 2012: Training modules available for item writers/reviewersreviewSummer 2012: Educators from Governing States begin writing items and tasks; cognitive labs / small scale trials beginSeptember 2012: Sample items reviewed by Smarter Balanced staff and advisors, Student Achievement PartnersOctober 2012: Sample items and tasks availableFebruary / March 2013: Pilot Test of first 10,000 items and performance tasksSmarter Balanced sample items and performance tasks were developed in collaboration with educators and content experts.This process began with the development of content specifications in ELA/literacy and mathematics. The specifications ensure that the assessment system will cover the full range of college- and career-ready knowledge and skills in the Common Core State Standards.Earlier this year, Governing States adopted assessment claims for ELA/literacy and mathematics, following two rounds of public review and comment. These guide development of assessments, providing descriptions of knowledge and skills (“assessment targets”) that items/tasks will assess.Item/task specifications were finalized in April to provide detailed instructions to writers for developing items. Review guidelines ensure that all items and performance tasks are reviewed consistently for content, accessibility, bias and sensitivity.Smarter Balanced recruited K-12 teachers and higher education faculty to participate in the writing and review of items and tasks.The sample items and tasks were reviewed by content experts, including Student Achievement Partners.The Pilot Test will occur this spring and will be open to all schools in Smarter Balanced states.
46 Our guiding beliefs and approach for CCSS Implementation in WA 2-Prongs:The What: Content Shifts (for students and educators)Belief that past standards implementation efforts have provided a strong foundation on which to build for CCSS; HOWEVER there are shifts that need to be attended to in the content.The How: System “Remodeling”Belief that successful CCSS implementation will not take place top down or bottom up – it must be “both, and…”Belief that districts and communities across the state have the conditions and commitment present to engage wholly in this work.Professional learning systems are critical
47 The “What”: ELA and Math Content Shifts Shifts in ELABuilding knowledge through content-rich nonfiction and informational texts in addition to literatureReading and writing grounded in evidence from the textRegular practice with complex text and its academic vocabularyThese apply to content area (social studies, science, and technical subject) teachers as well as to English teachers.Shifts in MathematicsFocus: 2-3 topics focused on deeply in each gradeCoherence: Concepts logically connected from one grade to the next and linked to other major topics within the gradeRigor: Fluency with arithmetic, application of knowledge to real world situations, and deep understanding of mathematical concepts47
48 Smarter Balanced Timeline (Summative tests) 47 districts invited to conduct Small Scale Trials in October/November 2012Limited pilot inNeed 22% of stateAvailable to allComprehensive field test inOperational use in
49 Testing System Transition Current Testing SystemReading and Math: Grades 3–8 and 10Writing: Grades 4, 7, 10Science: Grades 5, 8, 10SMARTER Balanced (SBAC) / Common Core State Standards (CCSS) Testing SystemEnglish/Language Arts and Math: Grade 3–8 and 11*Science exams are required under ESEA but are not included in SBAC*11th grade to measure college and career readiness. We are working with higher ed to explore the possible use of these measures as an alternative for college placement (or entrance).
50 Current Statewide Summative (Student) Assessments ReadingMathematicsScienceWritingGrade 3MSPGrade 4Grade 5Grade 6Grade 7Grade 8High SchoolHSPEEOCMSP= Measurements of Student Progress;HSPE = High School Proficiency Exams;EOC= End of Course exams
51 Washington’s Context… Likely Summative Assessments in 2014–15 English/LAMathematicsScienceGrade 3SBACGrade 4Grade 5MSPGrade 6Grade 7Grade 8Grade 10E/LAusing SBAC itemsEOCsEOCGrade 11SBAC=SMARTER Balanced Assessment ConsortiumEOCs= End of Course exams* SBAC is vertically scaled; MSP/HSPE are not.*
53 Feedback and Support available through first week of November Online feedback and phone support availableSmarter Balanced welcomes feedback on the sample items and tasks.Users will be able to submit feedback or questions through an online form or by calling a number.