1. Using CTS to Design Formative Assessment Probes NSTA Web Seminar Page Keeley Maine Mathematics and Science Alliance

2. In this Session You Will… Learn how to use the CTS process to design research-based formative assessment probes. Examine a variety of probes and their connection to CTS. Share your own ideas for developing an assessment probe.

3. Formative Assessment Probes Specifically designed tasks used to elicit student conceptions around a specific topic. Probes are diagnostic in nature.

4. Background on CTS Probe Development Let’s take a “tour” to see how a probe is developed using CTS. The steps are described in detail on page 81 of the CTS book.

5. Step 1 Identify the CTS guide(s) for the unit topic you will be teaching Use Section III, IV, V, and VI

6. Scenario: You are teaching a middle school unit on sound. Your students had some experience with sound in the elementary grades. You are interested in finding out what they already know about key ideas related to sound. Which topic study would you choose?

7. Sound Concepts and Specific Ideas What concepts and ideas are covered in elementary grades? What concepts and ideas are important to cover in middle school?

8. Step 2 Use CTS to examine and record key concepts and ideas

9. Example (p 82): Vibration 1.Sound is produced by vibrating objects (NSES K-4 p 127). 2.The pitch of the sound can be varied by changing the rate of the vibration. (NSES K-4 p 127) 3.Explain ways different forms of energy [such as sound] are produced. (Maine’s Learning Results H2 grades 3-4)

10. Step 3 Identifying possible misconceptions or student difficulties

11. Identify the concepts and associated misconceptions or difficulties addressed in the research. Make notes on any contextual issues, relevant phenomena, developmental considerations, strategies, etc. that could be used to inform the design of your assessment probe.

12. Many children explain how sound is produced in terms of the physical properties of the material that produces the sound. The force needed to produce sound, like a human beating a drum, is another proposed mechanism for sound. Misconceptions Associated with Sound Production

13. Explanations involving vibration or vibrations being transferred to air increase with age. Children appear to be context bound when using vibration ideas, particularly musical instruments. Children may not have a generalized theory of sound production across contexts particularly in situations where vibration is not obvious, such as hitting two stones together. Misconceptions Associated with Sound Production

14. Select Precursor Concept and Ideas Sound is produced by vibrating objects (NSES K-4 p 127). Explain ways different forms energy [such as sound] are produced. (Maine’s Learning Results H2 grades 3-4)

15. Concept and Noted Difficulty from the Research Summary Children appear to be context bound when using vibration ideas, particularly musical instruments. Children may not have a generalized theory of sound production across contexts particularly in situations where vibration is not obvious, such as hitting two stones together.

16. Step 4 Focus on a concept, specific idea, and the research finding. Choose a format for the type of probe that would give you information about students’ thinking related to the concept, specific idea, and research finding.

17. Step 5 Develop the stem and distracters to specifically target the research findings and match the developmental level of your students. Use familiar objects, phenomena, and examples.

18. List some familiar objects or materials you might include in a justified list.

19. Mark the objects on this list you think your students would most likely choose as involving the production of sound by vibration: __ Guitar strings __ Radio speaker __ Car engine __ Singer __ Dripping faucet __ Barking dog __ 2 stones rubbed together __ Drum __ Crumpled paper __ Wood Saw __ Wind __ Piano

20. Explain your thinking. What rule or reasoning did you use to decide which objects produce sound by vibration?

21. CTS Connection How did CTS inform the development of this probe? How would you expect middle school students to respond? Why is it helpful to develop a probe that connects specific concepts and ideas to research findings?

22. Next Steps Share your developed probe with colleagues for feedback Pilot with students and modify as needed Give probe and use assessment data to inform teaching and learning

23. Another Example - Ice Cubes in a Bag Explain an Idea (e.g. Ice Cubes in a Bag) Committing to an Outcome (e.g. The Mitten Problem) Justified Lists (e.g. Is It an Animal?) Conservation of Matter CTS How would you use CTS to understand the intent of each of the following probes? Which topic study guides would you choose to help you analyze your students’ responses?

24. Ice Cubes in a Bag You are having an argument with your friend about what happens to the mass when matter changes from one form to another. To prove your idea, you put three ice cubes in a sealed bag and recorded the mass of the ice in the bag. You let the ice cubes melt completely. Ten minutes later you recorded the mass of the water in the bag.

25. ___The mass of the water in the bag will be less than the mass of the ice in the bag. ___The mass of the water in the bag will be more than the mass of the ice in the bag. ___The mass of the water in the bag would be the same as the mass of the ice cubes in the bag. Which response do you think students at your grade level are most likely to select?

26. Your Turn What is an idea you might like to explore and try to “uncover” students’ conceptions? What CTS guide will help you explore and uncover ideas from the learning goals and research on student learning? What sections of the CTS will you examine to help you develop a probe?

27. Next Steps: Develop Your Own Probe Using CTS and Share Your Work Try to develop a probe using the steps in the CTS guide and results from a CTS. Share your example on the discussion board or provide feedback to others who share examples. Be sure to include how you used CTS.

28. Thoughts or Reflections? Additional Questions? How might you use probes in your work?

29. More Info on Probes Probing Students’ Ideas in Science- Volume 1- by P Keeley, F Eberle, and L Farrin (in Press) Probing for Student Understanding by P Keeley, F Eberle, and L Farrin- in Jan 2005 Science Scope Assessment issue.