Access to all materials for today:

Access to all materials for today: https://tinyurl.com/LA5DAssess

Developing 5D Assessments of Student Science Learning
Access to all materials for today: Developing 5D Assessments of Student Science Learning William R. Penuel University of Colorado Boulder LA Leadership Summit

Introductions and Reflections
In tables: What are some big ideas about NGSS that you’ve learned as a group and from your own practice that you think are relevant to NGSS assessment? One person share out: One idea you heard in common at your table, names and roles of people in your group Think pair share

Workshop Agenda Day 1 9:00a-9:15a Introductions/getting started
Where you’ve been, and how it connects with today 9:45-10:30a What makes for good phenomena and problems for assessment? 10:30-10:45a Break 10:45-11:45a Analyzing sample 3-D assessments 11:45a-12:40p Lunch 12:40-2:05p Practice with building a scenario related to curriculum we use 2:05-3:05p Integrating SEPs into Assessments 3:05-3:20p 3:20-4:05p Integrating CCCs into Assessments 4:05-4:45p Making assessments more equitable 4:45--5:00p Reflection and Exit Ticket

Workshop Agenda - Day 2 Day 2 8:15-9:55a
Developing Scoring Guides for Assessments 9:55-10:45a Peer Review and Revision of Assessments 10:45-11:00a Break 11:00-11:15a Question and Answer 11:15a-12p Why two extra dimensions? 12-12:40p Lunch 12:40-1:40p How do we “assess” interest and identity? 1:40-2:40p Developing an exit ticket related to student experience 2:40p-3:00p 3:00-3:30p 3:30-4:30 Planning and Revision of Assessments 4:30-5:00 Reflection, evaluation, and close

Your Questions What are questions about “5D” assessment that you hope to have answered in this workshop? Driving Question Board

Workshop Aim To prepare participants to use resources and tools for helping educators create assessments of students’ integrated understanding of core ideas, practices, and crosscutting concepts and that connect to students’ interest and identity.

Some Norms for Today Yes, Yoda, there is such a thing as try.
Be willing to turn and face the uncomfortable and unfamiliar. Look around often at others appreciatively who are willing to be uncomfortable with you. Bring your wisdom and expertise to the table, and offer it when it can help us move forward. Get up and move around as you need.

What makes for a good scenario?
Our Criteria

Some Terminology The Task Scenario Prompts or Questions
In 1906, a massive earthquake shook the San Francisco Bay Area along the San Andreas fault, producing a break in a fence in one area that looked like this picture. Using what you know about the causes of earthquakes, construct an explanation for why we see what we see here. Write a claim about what happened to the fence during the earthquake. What pattern of evidence in the photo appears to support this claim. What additional evidence might you need to support your claim? The San Andreas fault lies along a transverse plate boundary. How does that information support or contradict your claim? Scenario Prompts or Questions

Gallery Walk Use stickies to indicate when a scenario does meet one of our criteria. Write the name of the criterion on the sticky. Use one criterion per sticky. Link to electronic version

Achieve Science Task Screener
Presents real-world observations Based on at least one specific instance, not a topic or generally observed occurrence (e.g., observations related to a specific hurricane rather than “hurricanes” in general) Presented as puzzling/intriguing Creates a “need to know” Is Explainable using grade appropriate SEPs, DCIs, and CCCs

Effectively use at least two modalities (e.g., images, diagrams, video, simulations, textual descriptions) If data are used, scenarios present real/well- crafted data The local, global, or universal relevance of the scenario is made clear to students Are comprehensible to a wide range of students at grade-level

Use as many words as needed, no more Are sufficiently rich to drive the task

Analyzing a scenario Take one of the assessments and jot down what you notice about each of these. Review the annotations: What stands out to you? Swallows (HS - Annotated) Natural Hazards (MS - Annotated) Oil Spills (ES - Annotated) Create a new G-doc in our folder to complete this task.

Beginning with Analysis of a PE
Choose a single PE Locate the full description of the PE on the NGSS website. Template for Analysis

Begin with Analysis of the Standards
5-ESS1-1. Support an argument that differences in the apparent brightness of the sun compared to other stars is due to their relative distances from Earth. [Assessment Boundary: Assessment is limited to the relative distances, not sizes, of stars. Assessment does not include other factors that affect apparent brightness (such as stellar masses, age, stage).] Insert link to blank Google doc for analysis 5-ESS1-2. Represent data in graphical displays to reveal patterns of daily changes in the length and direction of shadows, day and night, and the seasonal appearance of some stars in the night sky. [Clarification Statement: Examples of patterns could include the position and motion of Earth with respect to the sun and select stars that are visible only in particular months.] [Assessment Boundary: Assessment does not include causes of seasons.] Insert link to blank Google doc for analysis We’re going to bring you through a short version of a process we use to select anchoring phenomena for a unit in the Inquiry Hub partnership among CU Boulder, UCAR, and Denver Public Schools. We’ve used this process to develop three units of high school biology, all of which have both an anchoring phenomenon and a design challenge associated with them. We always begin with analyzing the bundle of standards or performance expectations that we’ve pre-selected as the focus for our unit. Today, we’ll work with two performance expectations. We’ll use a process to analyze each of the three dimensions in these standards that we use, which involves tracing the standards backward to the Framework, to try and identify the key facets of student understanding that need to be developed, as well as the prerequisites from early grades that students need to bring to the task of explaining a phenomenon at the appropriate grade level. There are lots of ways to do unpacking, and this is but one. Key is that you focus on all three dimensions and that you provide the design team selecting phenomena with some time to make sense of what the standards really are. Assign groups to one of three groups, and have them download the view-only templates for unpacking to complete (or create a separate Google Drive space for them). Standard 1 ( Standard 2 ( Some guidance: This is an occasion for considering how long to engage participants in unpacking. Going deep for a short period of time is always more beneficial than going broad; if you like, you can pre-unpack some standards to show people at the conclusion of this section. But giving some time for sensemaking is critical to helping people generate initial ideas for phenomenon: It gets them engaged with the content at a level needed to think about potential phenomena. [4 minutes]

Guidelines for Analysis
Use the Framework text as your guide, not your own ideas about “facts” you should teach that aren’t in the text. Formulate as claims in complete sentences. Consider facets or different aspects of the ideas represented. Consider knowledge students would need that is foundational for understanding the ideas. When tempted to write a definition, instead of writing “X is,” start a sentence with, “X explains…” or “X helps explain…” Here are a few guidelines to help you get started with Analysis. First, use the Framework text as your guide, and not your own ideas about what kids “should” know. Sometimes those are in the Framework, but many times when we bring our own ideas we bring the “facts” and not the “big ideas behind the facts.” The Framework emphasizes the latter, as well as relationships among those big ideas. Pay attention to those when unpacking. The Framework was trying to narrow content that is taught around specific topics—and to create cohesive progressions of ideas. Second, don’t just use single words or ideas when you “unpack.” Write full sentences, and think of them as claims that students should be able to elaborate upon and defend with evidence. Third, each sentence in the Framework packs in a lot of ideas. Think about the different facets or aspects of the idea that go into understanding what the sentence means. Fourth, some of the knowledge is assumed students will bring from elementary school. Use Framework text about grade band expectations to develop claims that go in the right hand column of the unpacking. Fifth, unpacking can take us backward from the vision of the Framework if we’re not careful. We could wind up with a lot of definitions we think we are supposed to teach. Discipline yourself: Every time you are tempted to unpack by writing a definition, say what the concept or idea helps you explain. [3 minutes]

Some Example Claims 4-LS1-1. Construct an argument that plants and animals have internal and external structures that function to support survival, growth, behavior, and reproduction. Facets Prior Knowledge All living things have structures that serve important functions for them. Some of these structures are things we can see (external) like hands and arms, or eyes. Plants and animals are living parts of the environment. Plants and animals have some parts we can see and some that are inside them we can’t easily see. Here are a few guidelines to help you get started with unpacking. First, use the Framework text as your guide, and not your own ideas about what kids “should” know. Sometimes those are in the Framework, but many times when we bring our own ideas we bring the “facts” and not the “big ideas behind the facts.” The Framework emphasizes the latter, as well as relationships among those big ideas. Pay attention to those when unpacking. Second, don’t just use single words or ideas when you “unpack.” Write full sentences, and think of them as claims that students should be able to elaborate upon and defend with evidence. Third, each sentence in the Framework packs in a lot of ideas. Think about the different facets or aspects of the idea that go into understanding what the sentence means. Fourth, some of the knowledge is assumed students will bring from elementary school. Use Framework text about grade band expectations to develop claims that go in the right hand column of the unpacking. Fifth, unpacking can take us backward from the vision of the Framework if we’re not careful. We could wind up with a lot of definitions we think we are supposed to teach. Discipline yourself: Every time you are tempted to unpack by writing a definition, say what the concept or idea helps you explain. [2 minutes]

Building Understanding
What are some core ideas and crosscutting concepts that students will students need to use, to explain a phenomenon we choose? What practices will they need to engage in, that help us identify the kinds of evidence students will need to make sense of? Join with another group. As a small group discuss and write down answers to the questions here in your notes. The purpose of this discussion is to build knowledge of the bundle as a whole, and to prepare us to brainstorm phenomena, by helping us refine a set of criteria for use in selecting an anchoring phenomenon for a unit or sequence focused on these two standards. Ask groups to share their discussion and notes, and add to the list of criteria the group has developed for what makes something a good anchoring phenomenon. [ANALYSIS 20 MINUTES, SHARING 15 MINUTES]

Resources for Finding Phenomena and Problems
Datasets Science Nuggets, National Climatic Data Archive Digital Media 99% Invisible, How To Science, Nature, RadioLab, ScienceNews for Students, Science Friday, SciShow Tangents, Quirks and Quarks Youth Voice SciJourner, Youth Radio Sci Show Tangents Partly crowdsourced thanks to Philip Bell

Using the SEP Task Formats Tool

Writing Prompts Using the Tool
In 1906, a massive earthquake shook the San Francisco Bay Area along the San Andreas fault, producing a break in a fence in one area that looked like this picture. Using what you know about the causes of earthquakes, construct an explanation for why we see what we see here. MS-ESS2-2: Construct an explanation based on evidence for how geoscience processes have changed Earth's surface at varying time and spatial scales. Engaging in Argument from Evidence Task Format #3

Writing Prompts Using the Tool
In 1906, a massive earthquake shook the San Francisco Bay Area along the San Andreas fault, producing a break in a fence in one area that looked like this picture. Using what you know about the causes of earthquakes, construct an explanation for why we see what we see here. Write a claim about what happened to the fence during the earthquake. What evidence in the photo appears to support this claim. What additional evidence might you need to support your claim? The San Andreas fault lies along a transverse plate boundary. How does that information support or contradict your claim?

Now you try! Start by writing an unprompted explanation of the phenomenon that reflects what you want students to write. You may need to modify the questions so that they would make sense to students. You can combine formats from different practices. Remember to take the student point of view when writing prompts: Students should know why they are answering the question.

Gallery Walk of Assessments

Integrating the Crosscutting Concepts
The structure of this tool is different: It focuses on potential prompts you can add to your assessments at particular places.

Integrating CCC prompts into tasks
In 1906, a massive earthquake shook the San Francisco Bay Area along the San Andreas fault, producing a break in a fence in one area that looked like this picture. Using what you know about the causes of earthquakes, construct an explanation for why we see what we see here. MS-ESS2-2: Construct an explanation based on evidence for how geoscience processes have changed Earth's surface at varying time and spatial scales.

Integrating CCC prompts into tasks
In 1906, a massive earthquake shook the San Francisco Bay Area along the San Andreas fault, producing a break in a fence in one area that looked like this picture. Using what you know about the causes of earthquakes, construct an explanation for why we see what we see here. How could we test whether the two parts of the fence are continuing to move apart, when they don’t appear to be moving after the earthquake?

Gallery Walk of Assessments

Equity in Assessment Tasks
The task provides ways for students to make connections of local, global, or universal relevance. The task includes multiple modes for students to respond to the task. The task is accessible, appropriate, and cognitively demanding for all learners, including students who are English learners or are working below or above grade level. The task cultivates students’ interest in and confidence with science and engineering. The task focuses on performances for which students’ learning experiences have prepared them.

Generating Ideas In small groups:
What are some strategies for making our tasks more equitable? Norms Go for Quantity Suspend Judgment Listen to Everyone Share First-Draft Thinking When finished, post your ideas to this document.

Introduction to Universal Design for Learning
[1.5 minutes] Read: Universal Design for Instruction (UDL) is about designing materials to be accessible to a wide range of users up front. Thinking about the diversity of users before, during, and after the design process reduces the need for later accommodations. A common example of UDL is a curb-cut. The divot in the curb makes it possible for people who use wheelchairs to move about freely. However, the curb cut makes life better for everyone. Bikers, people pushing strollers, and people with limited ability can all get around better because of the curb cut. Resources and tools designed with UDL in mind are not just better for people who need accommodations, they can be better for everyone. Facilitators: Discuss- Are there other examples of UDL that you can think of. What makes the resource, product, or tool work well for a wide variety of ability levels? A curb cut in that leads to the sidewalk Material developed by Gina Tesoriero and Robbin Riedy for the ACESSE Project

Universal Design for Learning (UDL)
[estimated time: 3 mins] Designing materials and resources for a wide range of people means attending to the Why, What, and How those resources will be used. When it comes to instruction, think about the why, what, and how of learning. The learning should matter to students so that they have an reason for being engaged. Information should be represented in multiple ways so, and likeways, students should be able to present what they know in different modes of expression. Thinking about the assessment you are designing: why are students interested in this topic? What are they learning, what is the content? How are students demonstrating their learning? Material developed by Gina Tesoriero and Robbin Riedy for the ACESSE Project

UDL Instructional Supports
Optimize individual choice Foster collaboration Provide mastery feedback Promote self assessment and reflection Add graphics and animation, and tactile representations Highlight critical features Activate prior knowledge Eliminate irrelevant barriers Vary methods of response Support planning and organization Support progress monitoring Enhance access to technology [2 min] There are many ways to support instruction. These are just a few. Examples could include allowing students to present results in an infographic, narrative, PowerPoint presentation, or poster. Eliminating irrelevant barriers could include helping students with vocabulary along the way when needed. Facilitators notes: The group can provide an example of each of the bullet points and come up with additional ideas. Material developed by Gina Tesoriero and Robbin Riedy for the ACESSE Project

Tools and Tips for UDL General tips
Ask students how they prefer to consume and generate information Ensure that lab safety information is accessible to add participants Technology tips Avoid PDFs when possible. If using PDFs, they should be accessible to screen readers. Use captions in videos Consider font choices--color, brightness, and size for participants with low vision The University of Washington’s DO-IT website has numerous technology and pedagogical tips and resources Additional resources (Dis)ability awareness, privilege and interactions Example of UDL in a STEM learning environment at CU [1 min] PDFs need to be OCRd, or read for Optical Character Recognition. This can be done in Adobe Acrobat Pro or in other tools. If you cannot highlight an individual line of text in a PDF, it has not accessible. Even with OCR, complicated graphs, tables, and mathematical notation might not render correctly. You may need to have alternative formats available. View the handout Material developed by Gina Tesoriero and Robbin Riedy for the ACESSE Project

Prompt for UDL Revision Task
Revise the task to be more accessible. [10 min] Think about your assessment task and modify according to UDL ideas. Are materials, tools, and resources presented in multiple ways? Are students about to demonstrate what they know in different ways? UDL Center Material developed by Gina Tesoriero and Robbin Riedy for the ACESSE Project

Put your assessments here
Folder for Assessments

Exit Ticket Please complete the exit ticket here:
What did you take away from today? What questions do you hope we’ll answer together tomorrow?

Developing 5D Assessments of Student Science Learning
Access to all materials for today: Developing 5D Assessments of Student Science Learning William R. Penuel University of Colorado Boulder Genesee Valley Educational Partnership BOCES February 6-7, 2019

Exit Ticket Review Themes I saw from what you were concerned about from yesterday:

Exit Ticket Review Link to data What do you notice in the data?
What questions should we prioritize today?

Developing scoring guides: An analogy
When test developers engage in principled assessment design, we seek to: Develop evidence about what students know and can do To support claims or inferences about what they know and can do Supported by reasoning about the task’s appropriateness and how scores turn students’ answers into evidence.

Purposes of this activity
Model a way to review assessments that are already designed to ensure alignment to a given performance expectation Build scoring guides from anticipated and actual student responses that award points for what’s most important (i.e., focus on elements of the performance expectation)

Part 1 Instructions Look at the assessment you are assigned.
Answer the questions as if you were a student or write an explanation of the phenomenon if you can’t find the questions. :-) Write some claims down of what you think you would be able to make about what students would be able to say and do, if they completed this assessment and answered the questions well. Review claims feedback on your assessment.

Sample Claim: Swallows
“From the student responses I anticipate across all these prompts, students can use their understanding of natural selection and adaptation to interpret analyze data on how proportions of traits in a population of organisms are changing.”

Paired Assignments List pairs here
Swap assessment with colleague to take as a student. Write a claim what you think is for the activity. Students don’t know the PE.

Part 2 Instructions Create a scoring guide, assigning points for each answer. Review the scoring guide against your list of facets and the evidence statements for the PE: Annotate where you see points linked to the NGSS evidence statements.

Tips for Developing Scoring Guides
Focus on the entire body of work to draw conclusions about what students know and can do. You don’t have to award points for all the questions, especially early ones just to get kids going! Award points based on what’s called out in the evidence statements and analyzed facets. Sum up the points for the task as a whole. Make sure that you are giving the most points for prompts that get at the heart of the matter.

What are evidence statements? Link
They are an analysis or “unpacking” of a single performance expectation of the NGSS. They were developed to support assessment design. We are using them here as a way to make sure our scoring guides award points that are important parts of the PE. (e.g., award more points for understanding natural selection than for correct labeling of graphs and tables)

Exploring Interest and Identity
Complete this survey here: It will ask you several questions about a pursuit you had for at least three years during adolescence that you: did with others, really enjoyed, got better at or learned a lot from, sought every opportunity to pursue, and had something to do with your decision to become a teacher

Analyzing Our Data Open up to the results of the survey here:
At your tables, discuss: What patterns do you notice in the data? What do these patterns tell us about interest, identity, and how they are related to each other?

The importance of interest
Interest is a key catalyst for learning in science (Hulleman & Harackiewicz, 2009; Maltese & Tai, 2010). The diversity of students’ experiences can be a resource for learning, if and when the anchor connects to everyday experiences and the teacher elicits and makes use of those experiences (Rosebery et al., 2010). References: Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. science, 326(5958), Maltese, A. V., & Tai, R. H. (2010). Eyeballs in the fridge: Sources of early interest in science. International Journal of Science Education, 32(5), Rosebery, A. S., Ogonowski, M., DiSchino, M., & Warren, B. (2010). “The coat traps all your body heat”: Heterogeneity as fundamental to learning. The Journal of the Learning Sciences, 19(3), Calabrese Barton, A., & Tan, E. (2018). A longitudinal study of equity-oriented STEM-rich making among youth from historically marginalized communities. American Educational Research Journal, Research points to a number of ways that interest matters for students’ learning.

The importance of interest
Students need opportunities to see ways that science and engineering practices are relevant to them personally and to their communities (Calabrese Barton & Tan, 2018). References: Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. science, 326(5958), Maltese, A. V., & Tai, R. H. (2010). Eyeballs in the fridge: Sources of early interest in science. International Journal of Science Education, 32(5), Rosebery, A. S., Ogonowski, M., DiSchino, M., & Warren, B. (2010). “The coat traps all your body heat”: Heterogeneity as fundamental to learning. The Journal of the Learning Sciences, 19(3), Calabrese Barton, A., & Tan, E. (2018). A longitudinal study of equity-oriented STEM-rich making among youth from historically marginalized communities. American Educational Research Journal, Research points to a number of ways that interest matters for students’ learning.

When students are interested, they say…
“We’re analyzing data from the CDC about the number of cases of antibiotic resistance so that we can see how big of a problem this is.” “We’re conducting investigations to grow bacteria so we can see where Addie could have gotten her infection.” “We’re using this NetLogo model, to see why some kinds of bacteria might be surviving more than others, even when there’s antibiotics in Addie’s bloodstream.”

Identity in science and engineering
“People define themselves through other people and through the artifacts and resources—visible and invisible—of their social and cultural worlds. In that sense, social relationships, significant others, particular activities and practices, political ideologies, religious beliefs or any other artifact, such as a flag or a song, become resources for making and expressing identity.” Esteban-Guitart & Moll (2014)

Funds of identity There are different kinds of funds of identity that can be used in teaching and elicited through assessment. Geographical Cultural Geographical Funds of Identity (any area or territory, such as a river, a country, a village or a mountain which become a source of self-identification. For example, the Grand Canyon as a symbol of Arizona state in the United States of America); Practical Funds of Identity (any activity such as work, sports, music. For instance, I can say that psychology is very important to me. It is part of my identity.); Cultural Funds of Identity (any artefacts such as religious symbols, national flags, national anthems or any social category such as introversion/extroversion, age, ethnic group or gender); Social Funds of Identity (significant others such as relatives, friends or colleagues); Institutional Funds of Identity (any social institution, such as family, marriage, or the Catholic Church). Practical Social Institutional Moisès Esteban-Guitart

When students identify with science, they say…
“I felt like a real engineer/scientist today.” “What we are learning in class is important and could benefit the city.” “I could imagine myself studying something like this in the future.” “I never knew you could study this as a scientist.”

Consider this PE What are some strategies you could imagine pursuing to identify students’ interests and identities in relation to this performance expectations?

Review these resources and discuss
Group A: Phenomenon Survey Group B: Photo-elicitation Group C: Funds of Knowledge Survey Group D: Funds of Identity Drawing Task In small groups, discuss: When might educators use such a tool? What might be benefits and challenges of using this kind of tool?

A first point about practices and assessment
Assessment activities should follow the contours of the science and engineering practices. Driving Question Board: Asking Questions Assessment itself is engineering design and engaging in argument from evidence

What makes for a good exit ticket? (Criteria)
Complete

Review these Exit Ticket examples and discuss
Example 1: iHub SEET Example 2: Isopod Habitat Challenge Example 3: Ambitious Science Teaching In small groups, discuss: What might you learn from how students answer the questions in this example? How could you imagine using the data? Then, design and post an exit ticket for a lesson.

How can we apply what we’ve learned?
Planning Template

Where do I find other stuff?
Assessment Design and Analysis Tools Achieve STEM Teaching Tools (ACESSE) Curriculum: iHub biology - Assessments (Evolution) Research on NGSS Assessment: LearnDBIR

Feedback on this workshop

Access to all materials for today:

Similar presentations

Presentation on theme: "Access to all materials for today:"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Access to all materials for today:

Similar presentations

Presentation on theme: "Access to all materials for today:"— Presentation transcript:

Similar presentations

About project

Feedback