Presentation is loading. Please wait.

Presentation is loading. Please wait.

REGIONAL SMARTER BALANCED ASSESSMENT WORKSHOPS January/February 2014.

Similar presentations


Presentation on theme: "REGIONAL SMARTER BALANCED ASSESSMENT WORKSHOPS January/February 2014."— Presentation transcript:

1 REGIONAL SMARTER BALANCED ASSESSMENT WORKSHOPS January/February 2014

2 Introductions Jennifer Judkins OSPI Mathematics Assessment, EOC Anton Jackson OSPI Mathematics Assessment, Grades 6 – 8 Julie Wagner OSPI Mathematics Assessment, Grades 3 – 5

3 Housekeeping Be sure to sign-in. Be on time when returning from breaks. Use cell phones and personal electronic devices at breaks. Keep side conversations to a minimum. Use the Parking Lot for non-item-writing questions. Provide evaluation and clock-hour forms at end of day.

4 Workshop Program Smarter Balanced Assessment System Overview Shifts in the CCSS Smarter Balanced Summative Assessment Claims and the CCSS Progressions Smarter Balanced Resources Claim 1 Item Writing Claim 2-4 Item Writing

5 Purposes and Goals of Workshop Increase understanding of the Smarter Balanced assessment system Increase understanding of Smarter Balanced claims, item changes, and how the shifts in the CCSS manifest in items Observe how the progression of learning is assessed through the years Bring awareness to the Smarter Balanced resources that are available to educators Provide participants with knowledge of the item-writing process 5

6 SMARTER-BALANCED ASSESSMENT SYSTEM Formative Resources Interim Assessments Summative Assessments

7 A Balanced Assessment System Page 7 Common Core State Standards specify K-12 expectations for college and career readiness All students leave high school college and career ready Teachers and schools have information and tools they need to improve teaching and learning Summative: College and career readiness assessments for accountability Interim: Flexible and open assessments, used for actionable feedback Formative resources: Digital Library with instructional and professional learning resources for educators to improve instruction

8 FORMATIVE RESOURCES Digital Library with formative assessment instructional and professional learning resources for educators

9 Digital Library Leadership Structures National Advisory Panel (NAP) experts Began December 2012 Provided policies and criteria for resources State Leadership Team (SLT) 5- 8 members Recruiting began November 2012 Provides training for State Network of Educators State Network of Educators (SNE ) 70 to 100 members Representation from LEAs, AEAs, content leaders, ELL, IHE Began recruiting in January 2013 Provides reviews and resources 9

10 Resources in the Digital Library 10 Commissioned Professional Learning Modules Resources for students and families Frame the formative assessment process within a Balanced Assessment System Articulate the formative assessment process Highlight formative assessment practices and strategies Assessment Literacy Modules Commissioned Professional Learning Modules Instructional materials for educators Instructional materials for students Demonstrate/support effective implementation of the formative assessment process Focus on key content and practice from the Common Core State Standards for Mathematics and English Language Arts Exemplar Instructional Modules High quality vetted resources and tools for students and families Reflect and support the formative assessment process Reflect and support the Common Core State Standards for English Language Arts/Literacy and Mathematics Create Professional Learning Communities Education Resources * Resources include the following file types: Video, HTML5, Audio, PPT, Excel, Word, and PDF.

11 Digital Library Functionality 11 Enable trained educators (SNEs and SLTs) to submit, review, and publish resources Allow educators to view, download, and rate resources Use state-of-the-art tagging and search to quickly find resources by CCSS and other topics Enable educators from across the Consortium to collaborate and share their knowledge

12 Resource Submission Process SNE 1 Submit Resource with Cover Profile Submit Resource with Cover Profile SNE 1 SNE Choose Resource to Review in SNE’s Queue SNE Initial Review of Resource with Gate- Keeping Criteria SNE Review of Resource with Quality Criteria Rubric Reviewed by 3 SNEs Review of Resource with Quality Criteria Rubric Reviewed by 3 SNEs Resource Passes Gate-Keeping Resource Fails Gate- Keeping—Return to SNE to Revise SNE SNE 1: The SNE who reviews the resource with the Gate-Keeping Criteria will also be one of the 3 SNE reviewers. SNE 1: The SNE who reviews the resource with the Gate-Keeping Criteria will also be one of the 3 SNE reviewers.

13 INTERIM ASSESSMENTS ICAs and IABs

14 Interim Assessment Two Types: Interim Comprehensive Assessments (ICA) Fixed form in fall 2014 for all grades Adaptive (as pool allows) Will be taken online in same form as summative Some level of hand-scoring Interim Assessment Blocks (IAB) Fixed form for most blocks in all grades Adaptive for certain blocks (as pool allows) Adaptive for all blocks – later Will be taken online in same form as summative Some level of hand-scoring

15 SUMMATIVE ASSESSMENT Assessment Claims Computer Adaptive Testing (CAT) Item Types Performance Tasks

16 Smarter Balanced Claims Claims are the broad statements of the assessment system‘s learning outcomes, each of which requires evidence that articulates the types of data/observations that will support interpretations of competence towards achievement of the claims.

17 Overall Assessment Claims Claim for Grades 3-8 : “Students can demonstrate progress toward college and career readiness in mathematics.” Claim for Grade 11: “Students can demonstrate college and career readiness in mathematics.”

18 Assessment Claims 1- 4 Claim 1 – Concepts & Procedures Claim 2 – Problem Solving Claim 3 – Communicating Reasoning Claim 4 – Modeling and Data Analysis The claims are supported by the Standards for Mathematical Practice.

19 Computer Adaptive Testing “ Based on student responses, the computer program adjusts the difficulty of questions throughout the assessment. … By adapting to the student as the assessment is taking place, these assessments present an individually tailored set of questions to each student and can quickly identify which skills students have mastered.” Smarter Balanced

20 Computer Adaptive Testing (CAT) New Item Types Multiple response multiple choice Matching Tables, Drag and Drop, Hot Spot Performance Tasks Automatically Scored Machine Scored Artificial-Intelligence (AI) scored

21 KEY CCSS SHIFTS Focus Coherence Rigor

22 Shift One: Focus strongly where the Standards focus Significantly narrow the scope of content and deepen how time and energy is spent in the math classroom Focus deeply only on what is emphasized in the standards, so that students gain strong foundations

23 Mathematics topics intended at each grade by at least two-thirds of A+ countries Mathematics topics intended at each grade by at least two- thirds of 21 U.S. states The Shape of Math in A+ Countries 1 Schmidt, Houang, & Cogan, “A Coherent Curriculum: The Case of Mathematics.” (2002).

24 K 12 Number and Operations Measurement and Geometry Algebra and Functions Statistics and Probability Traditional U.S. Approach

25 Focusing attention within ‘Number and Operations’ Operations and Algebraic Thinking Expressions and Equations Algebra  Number and Operations— Base Ten  The Number System  Number and Operations— Fractions  K High School

26 Shift Two: Coherence Think across grades, and link to major topics within grades Carefully connect the learning within and across grades so that students can build new understanding onto foundations built in previous years. Begin to count on solid conceptual understanding of core content and build on it. Each standard is not a new event, but an extension of previous learning.

27 Informing Grades 1-6 Mathematics Standards Development: What Can Be Learned from High-Performing Hong Kong, Singapore, and Korea? American Institutes for Research (2009, p. 13) 4.NF.4. Apply and extend previous understandings of multiplication to multiply a fraction by a whole number. 5.NF.4. Apply and extend previous understandings of multiplication to multiply a fraction or whole number by a fraction. 5.NF.7. Apply and extend previous understandings of division to divide unit fractions by whole numbers and whole numbers by unit fractions. 6.NS. Apply and extend previous understandings of multiplication and division to divide fractions by fractions. 6.NS.1. Interpret and compute quotients of fractions, and solve word problems involving division of fractions by fractions, e.g., by using visual fraction models and equations to represent the problem. Grade 4 Grade 5 Grade 6 CCSS

28 Coherence Example: Grade 3 The standards make explicit connections at a single grade Properties of Operations Area Multiplication and Division 3.OA.5 3.MD.7c 3.MD.7a

29 Shift Three: Rigor Equal intensity in conceptual understanding, procedural skill/fluency, and application The CCSSM require a balance of: Solid conceptual understanding Procedural skill and fluency Application of skills in problem solving situations This requires equal intensity in time, activities, and resources in pursuit of all three

30 (a) Conceptual Understanding... or is it?

31 Place Value Concepts 206 = ____ hundreds ____ tens ____ones 206 = ____ tens ____ ones 206 = ____ ones 347 = ____ tens ____ ones 452 = 3 hundreds ____ tens ____ones 521 = ____ hundreds 22 tens ____ ones Is the comparison true or false? 12 tens + 11 ones > 1 hundred + 2 tens

32 (b)Fluency The standards require speed and accuracy in calculation. Teachers structure class time and/or homework time for students to practice core functions such as solving equations so that they are more able to understand and manipulate more complex concepts.

33 Fluency Beyond the Facts

34 (c) Application Students can use appropriate concepts and procedures for application even when not prompted to do so Provide opportunities at all grade levels for students to apply math concepts in “real world” situations, recognizing this means different things in K-5, 6-8, and HS Teachers in content areas outside of math, particularly science, ensure that students are using grade-level-appropriate math to make meaning of and access science content

35 Application

36 Operationalizing the CCSS Shifts Do: Read the standards including critical areas of focus. Know the structure of the standards. Know the major, supporting and additional clusters for your grade. Study the progression documents with colleagues. Think about unit design first, then lesson, then task. It is not about what you are going to ‘do’ but what the students will learn.

37 Activity Turn to your elbow partner: Explain in your own words what the three shifts are in the CCSS. Explain to your partner how these shifts will change your instructional practice.

38 THE SMARTER BALANCED SUMMATIVE ASSESSMENT Considerations Claims Item Types Performance Tasks

39 CONSIDERATIONS

40 Evidence-Centered Design (ECD) Item development approach that begins with claims about what test scores will tell us about students' learning Determine evidence needed to support claims made about students based on their test scores Determine types of items and tasks needed to collect evidence

41 Six Key Concepts of ECD 6. Develop Items or Performance Tasks 1. Define the domain 2. Define claims to be made 3. Define assessment targets 4. Define evidence required 5. Develop Task Models

42 Universally Designed Items and Tasks – Some Guidelines Simplify vocabulary and sentence structure Place problems in familiar contexts with familiar items and names Use graphics only when necessary Simplify graphics and tables Avoid the use of color and maximize contrast Avoid content that spans multi-pages or that contains multiple-elements Use highly interactive manipulations only when necessary Identify terms and visuals that should not be altered

43 Bias Concerns Bias occurs when content contained in an item or task creates an unfair disadvantage for a sub-group of students Unfamiliar contexts or examples Unusual names of people or places References to local events or issues

44 Jimmy is the star on his football team. This season he scored ten touchdowns, kicked twenty extra points, and had eight field goals. What is the total number of points Jimmy has scored this season? Jimmy is the star on his football team. This season he scored ten touchdowns, kicked twenty extra points, and had eight field goals. What is the total number of points Jimmy has scored this season? What is this question asking me? What is a field goal? 104

45 Sensitivity Sensitivity focuses on content that creates unease, provokes negative feelings, or challenges beliefs or values Religions, religious practices, and religious figures Political topics Issues of gender, race, and ethnicity

46 Acceptable or Unacceptable? Two people who were conversing at a street corner parted and moved away from the corner in straight lines that are perpendicular to each other. If one person walked at 3 miles per hour and the second person jogged at 4 miles per hour, how far apart would they be after one hour?

47 Acceptable or Unacceptable? Two people stood next to each other. They started walking in straight lines that are perpendicular to each other. One person walked at 3 miles per hour. The other person walked at 4 miles per hour. How far apart are they after one hour?

48 Acceptable or Unacceptable? It takes Sarah an average of 30 minutes to clean her bedroom. She cleans her bedroom once a week. How many hours would Sarah spend cleaning her bedroom in one year?

49 THE CLAIMS

50 Assessment Claims Claim 1 – Concepts & Procedures “Students can explain and apply mathematical concepts and interpret and carry out mathematical procedures with precision and fluency.” Claim 2 – Problem Solving “Students can solve a range of complex well-posed problems in pure and applied mathematics, making productive use of knowledge and problem solving strategies.” Claim 3 – Communicating Reasoning “Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.” Claim 4 – Modeling and Data Analysis “Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.”

51 Claim 1 Concepts and Procedures Detailed: Rapid Development: The graph shown compares the height of Tree A and the height of Tree B over time (in years). How many years after Tree B was planted did Tree A and Tree B have the same height? Grade 8

52 Claim 2 Problem Solving

53 Claim 3 Communicating Reasoning Grade 6

54 Claim 4 Modeling and Data Analysis Grade 5

55 Contrasting the Claims - Activity Individually review the items on the ‘Contrasting the Claims’ Worksheet. Review with a partner the items by claim number.  What are the differences in the way the problems are written by claim?  How does the focus on four claims change the way you instruct and assess students in the classroom?

56 ITEM TYPES

57 CAT Item Types Multiple Choice Single correct answer Multiple correct answers Equation/Numeric Matching Table Fill-in Table Grid Items Drag and Drop Graphing Hot Spot

58 Multiple Choice – single correct response Grade 3

59 C) D) Multiple Choice – multiple correct responses A) B) Select all the graphs that show a proportional relationship. Grade 8

60 Equation/Numeric A triangular-shaped garden is shown. Enter the area of the garden, in square meters. Grade 5

61 Equation/Numeric

62 Matching Table Grade 11

63 Fill- in Table

64 Drag and Drop Kayla asked 10 students in her class whether they owned a dog or a cat or both. Drag one number into each box to complete the table, given this information: 40% of the students own a dog. 30% of the students own a cat. 10% of the students own both a dog and a cat. Grade 8

65 Drag and Drop

66 Graphing Grade 8

67 Hot Spot Grade 11

68 Item Type Implications Review the ‘Item Type’ worksheet and talk to a partner about these questions. Be prepared to share your discussion: What implications do these item types have, if any, on your instruction and assessment of students? What resources do you have that might allow you to create similar items and imbed them in instruction and assessment? N-rich Virtual manipulatives

69 PERFORMANCE TASKS

70 Performance Tasks 6 questions Distributed among Claim 2, 3, and 4 Item types on CAT may be used in a PT Also included short, constructed response: “Show your work.” “Explain your reasoning.” Classroom activity to precede work on PTs 2 items automatically scored, rest hand- scored

71 Classroom Activity

72 Sample Task Grade 3

73 Culminating Question

74 Performance Task Implications Talk to a partner about these questions and be prepared to share your discussion: How would you prepare students for sustained, connected activities like the performance tasks? What resources do you have for authentic, real- world applications of the mathematics students are learning? Illustrative Mathematics Engage NY

75 PROGRESSION OF LEARNING

76 Progression Documents

77 Progression of Learning and Items through the Grades Individually review the items in the progressions document. Find an example where what is learned in a prior year is fundamental to what is learned later. With a partner, determine the new learning that is added from year to year. Be prepared to share.

78 SMARTER BALANCED RESOURCES

79 Smarter Balanced Website

80 For Educators Field Test Practice and Pilot Tests Sample Items and Performance Tasks Item Writing and Review Content Specifications Item Specifications Considerations, Style Guide, etc. Achievement Level Descriptors and College Content Readiness

81 CLAIM 1 ITEM WRITING Depth of Knowledge Item Specification Documents

82 DEPTH OF KNOWLEDGE

83 Cognitive Rigor and Depth of Knowledge Level of complexity of cognitive demand Level 1: Recall and Reproduction Requires eliciting information such as a fact, definition, term, or a simple procedure, as well as performing a simple algorithm or applying a formula. Level 2: Basic Skills and Concepts Requires the engagement of some mental processing beyond a recall of information. Level 3: Strategic Thinking and Reasoning Requires reasoning, planning, using evidence, and explanations of thinking. Level 4: Extended Thinking Requires complex reasoning, planning, developing, and thinking most likely over an extended period of time.

84 Level 1 – Recall and Reproduction – (–5) 6 (–5) –12 · ·

85 Level 2 – Application of Skills/ Concepts A cylindrical tank has a height of 10 feet and a radius of 4 feet. Jane fills this tank with water at a rate of 8 cubic feet per minute. How many minutes will it take Jane to completely fill the tank without overflowing at this rate? Round your answer to the nearest minute. A cylindrical tank has a height of 10 feet and a radius of 4 feet. Jane fills this tank with water at a rate of 8 cubic feet per minute. How many minutes will it take Jane to completely fill the tank without overflowing at this rate? Round your answer to the nearest minute.

86 Level 3 – Strategic Thinking and Reasoning The total cost for an order of shirts from a company consists of the cost for each shirt plus a one-time design fee. The cost for each shirt is the same no matter how many shirts are ordered. The company provides the following examples to customers to help them estimate the total cost for an order of shirts. 50 shirts cost $ shirts cost $2370 Part A: Using the examples provided, what is the cost for each shirt, not including the one-time design fee? Explain how you found your answer. Part B: What is the cost of the one-time design fee? Explain how you found your answer. The total cost for an order of shirts from a company consists of the cost for each shirt plus a one-time design fee. The cost for each shirt is the same no matter how many shirts are ordered. The company provides the following examples to customers to help them estimate the total cost for an order of shirts. 50 shirts cost $ shirts cost $2370 Part A: Using the examples provided, what is the cost for each shirt, not including the one-time design fee? Explain how you found your answer. Part B: What is the cost of the one-time design fee? Explain how you found your answer.

87 Level 4 – Extended Thinking During the task, the student assumes the role of an architect who is responsible for designing the best plan for a park with area and financial restraints. The student completes tasks in which he/she compares the costs of different bids, determines what facilities should be given priority in the park, and then develops a scale drawing of the best design for the park and an explanation of the choices made. This investigation is done in class using a calculator, an applet to construct the scale drawing, and a spreadsheet.

88 Depth of Knowledge Look at the ‘Comparing the Claims’ and determine the Depth of Knowledge level of each. Be prepared to share.

89 ITEM SPECIFICATIONS How to read these documents

90 Item Specification Overview As each section of an item specification document is reviewed, find the same section in an item specification document at your grade- band level. Grade 3-5 – Grade 6-8 – Grade 11 –

91 Procedures to Support Validity Start with a clear explanation of what students are expected to know and be able to do (CCSS-M). Control irrelevant sources of variability by: Standardizing items using Task Models. Standardizing items using a Style Guide. 91

92 Description of the Target Targeted Standards

93 Relevant above and below grade standards Achievement Level Descriptors Depth of Knowledge

94 Statements of Required Evidence (Derived from Standards) Allowable Response Types Allowable Stimulus Materials and Mathematics Vocabulary

95 Task Models Flow from Evidence Statements 95 Details about stimuli for task model Required evidence is identified

96 Task Models Flow from Evidence Statements 96 Stem asks for the required evidence Preamble orients student to problem

97 Item Specification - Claim 1Targets GradeDetailedRapid Development

98 Rapid Development Specifications Detailed guidelines for stimuli for Task Model 1

99 Rapid Development Specifications TM Statement of required evidence Stimulus Format for Task Model Phrasing of Stem Rules for answer choices based on common errors related to standard

100 Claim 1 Activity Choose a task model from your item specification document and write an item to meet that task model. Share and review an item with a partner. How can you use this process to write items for your classroom or the district?

101 CLAIM 2-4 ITEM WRITING

102 Targets for Claim 2 Target A: Apply mathematics to solve well-posed problems in pure mathematics and arising in everyday life, society, and the workplace. Target B: Select and use appropriate tools strategically. Target C: Interpret results in the context of a situation. Target D: Identify important quantities in a practical situation and map their relationships (e.g., using diagrams, two-way tables, graphs, flowcharts, or formulas).

103 Targets for Claim 3 Target A: Test propositions or conjectures with specific examples. Target B: Construct, autonomously, chains of reasoning that will justify or refute propositions or conjectures. Target C: State logical assumptions being used. Target D: Use the technique of breaking an argument into cases. Target E: Distinguish correct logic or reasoning from that which is flawed and – if there is a flaw in the argument – explain what it is. Target F: Base arguments on concrete referents such as objects, drawings, diagrams, and actions.

104 Targets for Claim 4 Target A: Apply mathematics to solve well-posed problems in pure mathematics and arising in everyday life, society, and the workplace. Target B: Construct, autonomously, chains of reasoning to justify mathematical models used, interpretations made, and solutions proposed for a complex problem. Target C: State logical assumptions being used. Target D: Interpret results in the context of a situation.

105 Targets for Claim 4 Target E: Analyze the adequacy of and make improvements to an existing model or develop a mathematical model of a real phenomenon. Target F: Identify important quantities in a practical situation and map their relationships. Target G: Identify, analyze, and synthesize relevant external resources to pose or solve problems.

106 Claim 2-4 Activity Take the item that you wrote for Claim 1. How might this item be modified to become Claim 2, 3, or 4? What changes would have to be made to move away from just Claim 1 items? What role does rigor play? How might these be included in a Performance Task? Share your thoughts with a partner. How can you use these thoughts to impact classroom instruction in your district?

107 Regional Assessment Workshops

108 Day 1 Wrap-Up What are the next steps you would do to use the information you learned today in your classroom, building and/or district? What role will you play in these next steps? Evaluations Clock hours Sign out

109 THANK YOU

110 DAY 2 - ITEM WRITING Regional Smarter Balanced Assessment Workshops

111 Schedule Start at 9 a.m. Preparation for Claim 1 item writing Claim 1 item writing Preparation for Claim 2-4 item writing Claim 2-4 Item Writing Adjourn by 4 p.m. With morning and afternoon breaks and lunch break as appropriate. 111

112 Day 2 Item Writing Agenda Welcome and Introductions Forms and Paperwork Group Norms Item-Writing for the Common Core Assessment Purposes and Goals Importance of Validity Documents Used to Support Validity Claim 1 Item Writing Considerations Procedure for Writing, Reviewing, Approving, and Entering Items Claim 1 Item Writing Claim 2, 3, 4 Item Writing 112

113 Forms and Paperwork Sign - in Non-Disclosure Agreement The items are secure, but the process is not. Substitute Reimbursement Evaluation Module Reviews Item Writing Clock Hours Module Reviews Item Writing 113

114 Public Disclosure Please be aware that your participation on this committee is considered public record under Washington State Public Record Act RCW Please be assured that your personal and residential information are not released. 114

115 Group Norms Be on time when returning from breaks. Use cell phones and personal electronic devices at breaks. Keep side conversations to a minimum. Respect each other’s work styles (need for quiet). Ask for and give feedback. Use the Parking Lot for non-item-writing questions. 115

116 Purposes and Goals of Workshop Write assessment items that are aligned to the Common Core State Standards for Mathematics Increase understanding of Smarter Balanced item specifications and other item-writing documents Involve Washington educators in the assessment process Provide participants with knowledge to share item- writing process with others and increase student achievement 116

117 Item Writing Considerations As reviewed yesterday: Alignment to CCSS-M Standards Item Specifications and Task Models Plain Language and Vocabulary Bias and Sensitivity Universal Design Cognitive Complexity And now the Style Guide 117

118 Style Guide Highlights Use words for numbers that begin sentences. Consider rewording when the number is needed to solve mathematical problems. Use numerals for: numbers 10 and above numbers that precede abbreviated units of measure numbers used to solve mathematical problems Use a comma: in numbers with five or more digits numbers with four digits only if other numbers in the item have five or more digits. 118

119 Style Guide Highlights Fractions: as words, fractions are hyphenated. as numerals, fractions are stacked vertically Time: use lower case abbreviations “a.m.” and “p.m.” use “noon” and “midnight” to avoid confusion Units: spell out units in item stems abbreviate units in options; do not include periods in abbreviated units page give specific abbreviations

120 Style Guide Highlights Conditional clauses should be avoided if possible.

121 CLAIM 1 ITEM WRITING CONSIDERATIONS 121

122 Mathematical Practices Mathematical Practices that support Claim 1: MP 4: Model with mathematics MP 5: Use appropriate tools strategically. MP 6: Attend to precision. MP 7: Look for and make use of structure. MP 8: Look for and express regularity in repeated reasoning.

123 Metadata Table – Claim 1

124 Achievement Quintiles Five divisions are: 1 - Very easy 2 - Easy 3 - Medium 4 - Hard 5 - Very hard

125 Maximum, Minimum Grade Claim 1 items will often be written to a specific grade level so both maximum and minimum grade will be the same. Some items where there is a direct building of skills from grade to grade may make the maximum or minimum grade one above or below.

126 Score Points Maximum Points Score Points: 0, 1 for a one-point item 0, 1, 2, for a two-point item. Maximum Points: 1 for a one-point item 2 for a two-point item.

127 Depth of Knowledge DOK 1—Recall; perform a rote procedure DOK 2—Application of Skill/Concept DOK 3—Strategic Thinking DOK 4—Synthesizing information and creating new example Most of Claim 1 items will be DOK 1 or 2 but watch what is written in the item specification documents. If multiple DOKs are listed, you must pick one.

128 Presentation Format T – Text TG – Text with Graphics

129 Allowable Response Types DAS Multiple Choice, single correct response (MC) Multiple Choice, multiple correct response (MS) Matching Tables T/F and Y/N variations (MA) ITS Equation/Numeric (EQ) Fill-in Table (TI) Drag and Drop, Hot Spot, and Graphing (GI)

130 Lessons Learned from Certification Set Successes Identifying the Task Model Following the Task Model rpdv, or rapid development, must be followed explicitly det, or detailed, has a bit more flexibility Alternatives for MC and MS items Rubric and Scoring information Foci of Attention Including only what you want the “student” to read in the item Rationales for MC and MS alternatives

131 Item Content - Student

132 Rationales MC and MS items need rationales for each alternative Use “Correct Answer” for correct answer(s) Item spec documents usually give examples of reasonable distractors (incorrect alternatives) You can include other types of distractors Describe, in positive terms, or show work as to why students would choose a distractor: The student added before multiplying by 6 The student multiplied the exponents 4 and 7 instead of adding the exponents Student adds the leg lengths of the triangle and does not square the lengths first

133 Questions?

134 Steps to Write Item Work with a partner. Choose a standard from the folders and take an item-specific sheet. Check item specification for item considerations for the given target. Check task model rules for item type, cognitive complexity, required evidence, tools Write item. Follow Peer Review Process. 134

135 Peer Review Process Ask a pair of colleagues to work your item and make suggestions for edits. Submit for approval. Specialist will review item and provide feedback to writers and peer reviewers, if appropriate. Specialist may ask for revisions/edits. Writers make edits and resubmit item to same specialist. 135

136 Let the writing begin! Write 2 items in Claim 1 136

137 CLAIM 2 ITEM WRITING

138 Claim 2 Students can solve a range of well-posed problems in pure and applied mathematics, making productive use of knowledge and problem-solving strategies.

139 Problem Solving Problem solving, which of course builds on a foundation of knowledge and procedural proficiency, sits at the core of doing mathematics. Proficiency at problem solving requires students to choose to use concepts and procedures from across the content domains and check their work using alternative methods. Smarter Balanced

140 Claim 2 Math Practices Mathematical Practices that support Claim 2: MP 1: Make sense of problems and persevere in solving them. MP 5: Use appropriate tools strategically. MP 7: Look for and make use of structure. MP 8: Look for and express regularity in repeated reasoning.

141 Secondary Claim Items written primarily to assess Claim 2 will necessarily involve some Claim 1 content targets. The content should draw upon the knowledge and skills articulated in the progression of standards leading up to and including the targeted grade Related Claim 1 targets should be listed below the Claim 2 targets in the metadata form.

142 Primary and Secondary Standard/Cluster While tasks developed to assess Claim 2 will have a primary content focus, components of these tasks will likely produce enough evidence for other content domains that a separate listing of these content domains needs to be included where appropriate.

143 Four Targets for Claim 2 Target A (TM1): Apply mathematics to solve well- posed problems in pure mathematics and arising in everyday life, society, and the workplace. Target B (TM2): Select and use appropriate tools strategically. Target C (TM3): Interpret results in the context of a situation. Target 4 (TM4): Identify important quantities in a practical situation and map their relationships (e.g., using diagrams, two-way tables, graphs, flowcharts, or formulas).

144 Assessment Targets Any given task should provide evidence for several of the preceding assessment targets; each of the targets should not lead to a separate task. Multiple targets should be listed in order of prominence as related to the task.

145 Activity Review the Claim 2 Document with your group. Distinguish between the various targets and related task models based on your grade band. Be prepared to speak to the differences between each target and task model.

146 Claim 2 Standards

147 Metadata One Claim 2 One Claim 1 Fill in rest of data

148 Metadata Two Claim 2 One Claim 1 Fill in rest of data

149 Metadata Two Claim 2 Two Claim 1 Fill in rest of data

150 A Review of Metadata Fields Mathematical Practices Achievement Quintiles Maximum, Minimum Grade Score Points Maximum Points Depth of Knowledge Presentation Formats Response Type

151 Scoring Scoring rules and answer choices will focus students’ ability to use the appropriate reasoning. Multiple correct responses are possible. MC and MS will be scored as correct/incorrect (1 point) If MA items require two skills, they will be scored as: All correct choices (2 points); at least ½ but less than all correct choices (1 point) Justification for more than 1 point must be clear in the scoring rules Where possible, include a “disqualifier” option that if selected would result in a score of 0 points, whether or not the student answered ½ correctly. EQ, GI, and TI items will be scored as: Single requirement items: will be scored as correct/incorrect (1 point) Multiple requirement items: All components correct (2 points); at least ½ but less than all correct choices (1 point) Justification for more than 1 point must be clear in the scoring rules. For a CAT item to score multiple points, either distinct skills must be demonstrated that earn separate points or distinct levels of understanding of a complex skill must be tied directly to earning one or more points.

152 Lessons Learned from Certification Set Successes Identifying the Task Model Following the Task Model rpdv, or rapid development, must be followed explicitly det, or detailed, has a bit more flexibility Alternatives for MC and MS items Rubric and Scoring information Foci of Attention Completing the metadata table Including only “student” information in item Rationales for MC and MS alternatives

153 Allowable Response Types DAS Multiple-Choice, single correct response (MC) Multiple Choice, multiple correct response (MS) Matching Tables T/F and Y/N variations (MA) ITS Equation/Numeric (EQ)

154 Problem Solving Contexts/Situations Brainstorm contexts/situations that are appropriate for your grade band Consider specific contexts from the designated standards for this claim Contexts/situations can include content from multiple domains.

155 Claim 2 Items Start with these item types: MC, MS, and EQ Do not copy examples or try to clone them. Items must be original and contain several standards that are in alignment with the chart for that grade level.

156 Item Writing Process Review Claim 2 document carefully. Work with a partner. Decide to which target you will write. Use the Claim 1 documents as a reference for how the problem is posed to assess Claim 1. Identify how the Claim 2 item you are writing differs from Claim 1 items.

157 Questions?

158 Debrief of Claim 2 How was this writing different from Claim 1: The process for Claims 3 and 4 is similar. How might you prepare to write these types of items? What are you next steps in your classroom, building, or district?

159 Day 2 Item Writing Wrap-up Parking Lot Questions/Comments Feedback Suggestions to improve process. Other questions? Evaluations Clock-hours Substitute reimbursement form 159

160 THANK YOU!

161 CLAIM 3 ITEM WRITING

162 Claim 3 Students clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.

163 Communicating Reasoning Communicating reasoning is the ability to construct and present a clear, logical, convincing argument. This is not just a requirement of the Standards for Mathematical Practice, it is also a recurrent them in the Standards for Mathematical Content. Many content standards call for students to explain, justify, or illustrate.

164 Claim 3 Mathematical Practices Mathematical Practices that support Claim 3: MP 3 – Construct viable arguments and critique the reasoning of others. MP 6 – Attend to precision.

165 Secondary Claim Items written primarily to assess Claim 3 will necessarily involve some Claim 1 content targets. The content should draw upon the knowledge and skills articulated in the progression of standards leading up to and including the targeted grade Related Claim 1 targets should be listed below the Claim targets in the metadata form.

166 Primary and Secondary Standard/Cluster While tasks developed to assess Claim 3 will have a primary content focus, components of these tasks will likely produce enough evidence for other content domains that a separate listing of these content domains needs to be included where appropriate.

167 Targets for Claim 3 Target A: Test propositions or conjectures with specific examples. Target B: Construct, autonomously, chains of reasoning that will justify or refute propositions or conjectures. Target C: State logical assumptions being used. Target D: Use the technique of breaking an argument into cases. Target E: Distinguish correct logic or reasoning from that which is flawed and – if there is a flaw in the argument – explain what it is. Target F: Base arguments on concrete referents such as objects, drawings, diagrams, and actions.

168 Assessment Targets Any given task should provide evidence for several of the preceding assessment targets; each of the targets should not lead to a separate task. Multiple targets should be listed in order of prominence as related to the task.

169 Task Models for Claim 3 TM1: Test propositions or conjectures with specific examples. TM2:Construct, autonomously, chains of reasoning that will justify or refute conjectures. TM3: Use the techniques of breaking an argument into cases. TM4: Determine conditions under which an argument does and does not apply. TM5: Distinguish correct reasoning from flawed reasoning.

170 Activity Review the Claim 3 document with your group. Distinguish between the various targets and related task models based on your grade band. Be prepared to speak to the differences between each target and task model.

171 Claim 3 CCSS Standards

172

173 Metadata One Claim 3 One Claim 1 Fill in rest of data

174 Metadata Two Claim 3 One Claim 1 Fill in rest of data

175 Metadata Two Claim 3 Two Claim 1 Fill in rest of data

176 A Review of Metadata Fields Mathematical Practices Achievement Quintiles Maximum, Minimum Grade Score Points Maximum Points Depth of Knowledge Presentation Formats Response Type

177 Scoring Scoring rules and answer choices will focus students’ ability to use the appropriate reasoning. Multiple correct responses are possible. MC and MS will be scored as correct/incorrect (1 point) If MA items require two skills, they will be scored as: All correct choices (2 points); at least ½ but less than all correct choices (1 point) Justification for more than 1 point must be clear in the scoring rules Where possible, include a “disqualifier” option that if selected would result in a score of 0 points, whether or not the student answered ½ correctly. EQ, GI, and TI items will be scored as: Single requirement items: will be scored as correct/incorrect (1 point) Multiple requirement items: All components correct (2 points); at least ½ but less than all correct choices (1 point) Justification for more than 1 point must be clear in the scoring rules. For a CAT item to score multiple points, either distinct skills must be demonstrated that earn separate points or distinct levels of understanding of a complex skill must be tied directly to earning one or more points.

178 Lessons Learned from Certification Set Successes Identifying the Task Model Following the Task Model Alternatives for MC and MS items Rubric and Scoring information Foci of Attention Completing the metadata table Including only “student” information in item Rationales for MC and MS alternatives

179 Communicating Reasoning in Contexts/Situations Brainstorm contexts/situations that are appropriate for your grade band Consider specific contexts from the designated standards for this claim Contexts/situations can include content from multiple domains.

180 Claim 3 Items Start with these item types: MC, MS, MA, and EQ Write items that are not copies or clones of the examples. Items must be original and contain several standards that are in alignment with the chart for that grade level. Check that the item doesn’t follow a Claim 1 task model.

181 Item Writing Process Review Claim 3 document carefully. Work with a partner. Decide to which target you will write. Use the Claim 1 documents as a reference for how the problem is posed to assess Claim 1. Claim 3 will NOT be assessed in that manner. Follow the peer review process.

182 Questions?

183 Debrief of Claim 3 How was this writing different from Claim 1? The process for Claims 2 and 4 is similar. How might you prepare to write these types of items? What are your next steps in your classroom, building, or district?

184 Day 2 Item Writing Wrap-up Parking Lot Questions/Comments Feedback Suggestions to improve process. Other questions? Evaluations Clock-hours Substitute reimbursement form 184

185 THANK YOU!

186 CLAIM 4 ITEM WRITING

187 Claim 4 Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.

188 Modeling and Data Analysis Modeling is the process of choosing and using appropriate mathematics and statistics to analyze empirical situations, to understand them better, and to improve decision-making. Smarter Balanced

189 Claim 4 Mathematical Practices Mathematical Practices that support Claim 4: MP 2 – Reason abstractly and quantitatively. MP 4 – Model with mathematics. MP 5 – Use appropriate tools strategically.

190 Secondary Claim Items written primarily to assess Claim 4 will necessarily involve some Claim 1 content targets. The content should draw upon the knowledge and skills articulated in the progression of standards leading up to and including the targeted grade Related Claim 1 targets should be listed below the Claim 4 targets in the metadata form.

191 Primary and Secondary Standard/Cluster While tasks developed to assess Claim 4 will have a primary content focus, components of these tasks will likely produce enough evidence for other content domains that a separate listing of these content domains needs to be included where appropriate.

192 Targets for Claim 4 Target A (TM1): Apply mathematics to solve well-posed problems in pure mathematics and arising in everyday life, society, and the workplace. Target B (TM 2): Construct, autonomously, chains of reasoning to justify mathematical models used, interpretations made, and solutions proposed for a complex problem. Target C (TM3): State logical assumptions being used. Target D (TM 4): Interpret results in the context of a situation.

193 Targets for Claim 4 Target E (TM 5): Analyze the adequacy of and make improvements to an existing model or develop a mathematical model of a real phenomenon. Target F (TM 6): Identify important quantities in a practical situation and map their relationships. Target G: Identify, analyze, and synthesize relevant external resources to pose or solve problems.

194 Assessment Targets Any given task should provide evidence for several of the preceding assessment targets; each of the targets should not lead to a separate task. Multiple targets should be listed in order of prominence as related to the task.

195 Distinguishing Claim 4 Distinguishing between Claim 4 and Claims 1 and 2: In early grades when equations are still new to students, an important distinction between Claim 2 and Claim 4 is requiring a model that would lead to a problem's solution. In Claim 2 problems are well posed, while in Claim 4 they may have extraneous or missing information. In Claims 1 and 2, measurements of objects or figures can be accurately determined. In Claim 4, modeling is used to make approximations. In Claim 1, data analysis is straightforward procedural. In Claim 4, the analysis should be tied to some useful purpose in the real world.

196 Activity Review the Claim 4 Document with your group. Distinguish between the various targets and related task models based on your grade band. Be prepared to speak to the differences between each target and task model.

197 Claim 4 CCSS Standards

198 Metadata One Claim 4 One Claim 1 Fill in rest of data

199 Metadata Two Claim 4 One Claim 1 Fill in rest of data

200 Metadata Two Claim 4 Two Claim 1 Fill in rest of data

201 A Review of Metadata Fields Mathematical Practices Achievement Quintiles Maximum, Minimum Grade Score Points Maximum Points Depth of Knowledge Presentation Formats Response Type

202 Scoring Scoring rules and answer choices will focus students’ ability to use the appropriate reasoning. Multiple correct responses are possible. MC and MS will be scored as correct/incorrect (1 point) If MA items require two skills, they will be scored as: All correct choices (2 points); at least ½ but less than all correct choices (1 point) Justification for more than 1 point must be clear in the scoring rules Where possible, include a “disqualifier” option that if selected would result in a score of 0 points, whether or not the student answered ½ correctly. EQ, GI, and TI items will be scored as: Single requirement items: will be scored as correct/incorrect (1 point) Multiple requirement items: All components correct (2 points); at least ½ but less than all correct choices (1 point) Justification for more than 1 point must be clear in the scoring rules. For a CAT item to score multiple points, either distinct skills must be demonstrated that earn separate points or distinct levels of understanding of a complex skill must be tied directly to earning one or more points.

203 Lessons Learned from Certification Set Successes Identifying the Task Model Following the Task Model rpdv, or rapid development, must be followed explicitly det, or detailed, has a bit more flexibility Alternatives for MC and MS items Rubric and Scoring information Foci of Attention Completing the metadata table Including only “student” information in item Rationales for MC and MS alternatives

204 Modeling Contexts/Situations Do we Brainstorm contexts/situations that are appropriate for your grade band Consider specific contexts from the designated standards for this claim Contexts/situations can include content from multiple domains.

205 Claim 4 Items Start with these item types: MC, MS, and EQ Do not copy examples or try to clone them. Items must be original and contain several standards that are in alignment with the chart for that grade level.

206 Item Writing Process Review Claim 4 document carefully. Work with a partner. Decide to which target you will write. Use the Claim 1 documents as a reference for how the problem is posed to assess Claim 1. Identify how the Claim 4 item you are writing differs from Claim 1 items.

207 Questions?

208 Debriefing Claim 4 How was this writing different from Claim 1? The process for Claims 2 and 3 is similar. How might you prepare to write these types of items? What are your next steps in your classroom, building, or district?

209 Day 2 Item Writing Wrap-up Parking Lot Questions/Comments Feedback Suggestions to improve process. Other questions? Evaluations Clock hours Substitute reimbursement form 209

210 THANK YOU


Download ppt "REGIONAL SMARTER BALANCED ASSESSMENT WORKSHOPS January/February 2014."

Similar presentations


Ads by Google