1. Using Teaching Routines with Classroom Network Technology to Support Improved Classroom Assessment Bill Penuel Patti Schank SRI International NSF DRK-12 PI Meeting November 9, 2009 This work has been supported by National Science Foundation grant DRL- 0822314. All opinions expressed herein are the sole responsibility of the authors.

2. Challenges to Improving Classroom Assessment Validation: Models of student thinking; usefulness and efficacy in guiding instructional decision making Speed of aggregation and interpretation, especially for on the fly instructional decision making The “what’s next problem”: How to use inferences from assessment to adjust classroom instruction Gap between different stakeholders’ visions for formative assessment

3. Classroom Network Technologies One-to-one access enables all students to contribute a response to a teacher prompt –New technologies enable teachers (or students) to pose different kinds of prompts –Schools are purchasing new low-cost devices that have functionality comparable to laptop computers Shared display facilitates classroom discussion of ideas –Displays are anonymous, helping make it safe for students to be public about their difficulties –Displays can encourage shared accountability for learning

4. Classroom Technologies: Clickers –Possibility for easy aggregation and interpretation of student responses –Trade-off: You don’t know what knowledge students can construct on their own, since multiple choice responses provide clues/enable guessing –Can support in class at low cost GroupScribbles –Possibility for more open-ended tasks and collaborative learning opportunities –Trade-off: Must devise strategies to help teachers aggregate results –Can support in lab at low cost, in class at a higher cost (short-term); may not be able to support long-term 5/20/20154

5. Teaching Routines Our definition: –Recurring patterned sequences of interaction teachers and students jointly enact to organize opportunities for student learning in classrooms. Sources –Ball’s notion of pedagogical routines (Schoenfeld, 2002) –Minstrell’s benchmark lessons (see especially van Zee & Minstrell, 1997) –Mazur’s (1997) Peer Instruction –U Mass-PERG’s Assessing-to-Learn (Dufresne & Gerace, 2004)

6. Teaching Routines Serve multiple functions (Lampert, 2005): –making it safe for students to express what they know and do not know –listening to the thinking their students express –constructing effective instruction in response to what they find out students need to learn Are especially important when using classroom network technology

7. Observations from Implementations of Classroom Network Technology Clicker use is only sometimes accompanied by rich classroom discussions –Some teachers limit time for students to think about responses –Teachers may develop beliefs that technologies are less useful, to the extent that they do not engage students in discussions of their ideas (Penuel et al., 2005) Teachers with access to technologies with more advanced capabilities do not readily use them (Penuel et al., 2008)

8. Beyond Routines: Facets Facets of student thinking (Minstrell) –Facets are used to describe students' thinking as it is seen or heard in the classroom or other learning situation. –Facets of students' thinking are individual pieces, or constructions of a few pieces, of knowledge and/or strategies of reasoning. Facets can guide questions and prompts for teachers to use with students –Teachers can use and identify facets to guide discussion of student ideas, identify different entry points for learning concepts

9. Contingent Pedagogies Project We are developing a set of assessment resources aimed at improving students’ conceptual understanding of Earth science that –Builds on the solid foundation of the Investigating Earth Systems curriculum –Includes questions designed to elicit facets of student thinking –Uses advanced classroom network technology to enable better feedback –Provides teachers with additional curriculum activities for “what to do next” based on assessments 5/20/20159

10. Investigating Earth Systems (IES) Curriculum organized around student investigations All development supports assessing concepts and skills from IES units 2 focal units for our development –Rocks and Landforms –Dynamic Planet 5/20/201510

11. Where We’ve Been (’08-’09) Formed a design team to create and provide feedback on interactive assessment activities Developed teaching routines to serve as a resource for the design team Created 28 draft interactive assessment activities using the routines and begun testing of lessons Identified technology needs and requirements for teachers on the design team and field test in a large urban district Begun to create measures of teaching and learning to evaluate the project

12. Where We’re Headed (’09-’10) Designing additional interactive assessments Testing activities that have been designed and that teachers invent, to study their potential and limitations Revising activities Enhancing activities by: –Identifying contingencies –Linking to facets of student thinking –Providing tips for orchestrating class discussions

13. Developmental Model Guiding Sequences of Steps Comfort with technology Teaching Routines Facets Contingenci es

14. Images provide a common background for individual or group expression. Teacher posts an image to individual/group boards. Groups mark images to indicate where processes occur. Diverse student markings across a common background facilitates comparing and contrasting understandings. Discuss similarities and commonalities in groups’ drawings. Similarities and differences invite reflection and revision of ideas. Students use discussion ideas to write an explanation of how processes are related. 1 2 3 4 WHERE ON THIS IMAGE? ROUTINE

15. Ring of Fire Activity Goal –Provide you with an introduction to Group Scribbles technology –Give you an experience of a routine designed specifically for this technology Invitation –Consider ways to strengthen the routine or adapt it for other content areas and skills for inquiry

16. Ring of Fire as Assessment Activity Model of Student Cognition: Using peers and resources to develop an explanation related to mantle convection and plate movements as implicated in both volcanoes and earthquakes Key Task Features: What task features present argument that would provide evidence of student understanding in the activity? How could the tasks be strengthened to elicit these arguments and map better onto ? Evidence from Student Work: What do students say or do that could reveal the quality of their thinking? Outcome Space: What are the range of responses, and how might we order them (in terms of sophistication, in terms of recommended next steps]?

17. Approach to Validation Student responses to tasks are analyzed with respect to facets of student thinking being identified by the study team now Salient features of the use context –Do teachers’ inferences about student thinking help them adjust both pacing and strategies? –Are the strategies they adopt (including ones we suggest) associated with better class-wide learning?

18. For More Information Contact –Bill Penuel (william.penuel@sri.com)william.penuel@sri.com –Patti Schank (patricia.schank@sri.com) Web –http://ctl.sri.comhttp://ctl.sri.com –Projects > Contingent Pedagogies