1. Video Cases Online: Cognitive Studies of Pre- Service Teacher Learning Sharon J. Derry University of Wisconsin-Madison Cindy Hmelo-Silver Rutgers University Funding Provided by The National Science Foundation

3. Unsolved Problem: Spontaneous Transfer to Practice Can conceptual systems we “market” in university professional programs be truly useful in students’ future professional lives? A search for practical instructional approaches that can be implemented on a large scale.

4. Context of Work: Learning Sciences Courses for Preservice Teachers at Rutgers & UW-Madison Goal: Help teachers acquire useful knowledge of learning science concepts.

5. Ideas are powerful things, requiring not a studious contemplation but an action, even if it is only an inner action. Midge Decter, American writer Knowledge in Action

6. Needed for Transfer: Meshed Cognitive Representations Produce representations that mesh: Perceptual visions of (teaching) practice Conceptual understanding learning science Plans for (teaching) practice Mindsets for reflective practice Courses that systematically intertwine: Video case study Conceptual processing (e.g. readings) Lesson design activities

7. STELLAR (Socio-technical Environment for Learning & Learning Activity Research) System for Building High-Mesh Online Courses Configurable Tools: Learning Activity Constructor Interactive Group Whiteboard Individual Student Notebooks Asynchronous and Synchronous Discussions Research Library Multimedia Hypertext Upload video and digital resources Integrate with course text Interface for Viewing Video

8. An Example STELLAR Activity: A Lesson Design Problem for Math Educators Designer Constructs Steps in Activity Designer Configures Tools for Learning Designer Creates Instructions & Activities for Each Step: This is Problem at Step 1

9. Group Whiteboard Tool Voting Utility Feedback Configurable Spaces and Prompts

10. Video Case Interface Links to Hypertext (differs for each mini- case) Active Mini- case

11. A Hypertext Page on Transfer Access to mini- cases on Transfer CFT-Based Navigation Tool

12. Ratings of STELLAR Activity Fall 2002 (5 = high) Activity ComponentsUWRutgers Overall Activity4.354.43 Individual prep3.713.93 Sharing design ideas4.363.70 Group design online4.394.26 Individual reflection3.863.93 Overall group interaction 4.394.32

13. Ratings of Online Tools and Features (5 = high) Tool FeatureUWRutgers Video cases4.034.00 CFT hypertext4.474.50 Links from cases to CFT hypertext 4.194.52 Individual notebook4.074.43 Group whiteboard4.164.36

14. Types of Data Group instructional plans developed on line Online discourse related to group work Individual reflections & analyses of group work Pre and post-course analyses of teaching/learning video cases Self reports of beliefs and attitudes related to teaching & learning Evaluative ratings of system activities and tools

15. Psychometrically Validated Assessments: Examples Concepts-in-Use Rubrics Used to score pre- and post video analyses and other student products to determine the level of sophistication in students use of target concepts. Example target concepts: understanding, transfer, metacognition, Bridging Instruction Rubric A complex rubric used to evaluate instructional designs and design discourse

16. About Rubrics Features to guide coding Calibrated to the STEP scoring scale Can be used across multiple types of student products and documents Are psychometrically sound (validity and reliability studies)

17. 1. Generally considers, mentions, or refers to understanding but does not explicitly or implicitly refer to any of the specific features listed below 2. Understanding is actively constructed knowledge 3. Understanding builds on prior knowledge 4. Understanding in context is an active process of comprehension that involves constructing a situation model. 5. Understanding supports the making of inferences and/or application in new contexts 6. There are different depths or forms of understanding 7. Understanding involves grasping the underlying principle, theme or big idea. 8. Understanding is socially negotiated and distributed in “communities of practice” (broadly defined to include classrooms and groups) Features Coders Consider in Judging Student Ability to Use the Concept Understanding to Inform Teaching

18. Results from Fall 2002 (0 = no knowledge; 3 = expert) RutgersUW-Madison N3360 Level & Course Ed Psy prerequisite for entering TE Learning Sci required in final year of TE Pre-course concept “use score” Mean = 0.42 SD = 0.35 Mean = 0.65 SD = 0.46 Post-course concept “use score” Mean = 1.56 SD =.63 Mean = 2.09 SD =.63

19. STELLAR vs. Traditional Course: Understanding Score Spring 2004 (Rutgers) NMeanStd. Dev PreSTEP32.97.55 Comparison37.93.50 PostSTEP331.92.77 Comparison37.78.48 ANCOVA F(1, 67)= 69.62, p<.001

20. Delayed Performance (%) Under Different Activity Designs Task Contrast Video Cases then Text a Text then Contrast Video Cases b Text-Notes (no Video) c Term Recall12.3 (2.3)39.2 (2.3)24.0 (2.4) Gist Recall7.4 (1.2)15.4 (1.2)14.0 (1.2) Transfer24.3 (1.6)32.4 (1.6)32.5 (1.7) Recognition73.9 (4.2)86.6 (4.2)84.3 (4.4) a n = 52 b n = 51 c n = 50

21. What Predicted Understanding Score? 1. Pretest 2. Site (Rutgers vs. UW) 3. Success with collaborative tool (whiteboard) 4. Success with video-text links 5. Website “hits” R Square =.38

22. What Predicted Perceptions of Learning? 1. Success with individual work activity steps 2. Success with Knowledge Web exploration R Square =.42 Comment: Course components promoting individual study and exploration in accordance with personal interests enhanced perceptions of learning but did not improve performance on targeted learning goal.

23. Regression Analysis For MSLQ Variables Predicting Academic Success in a STEP Course

24. Conclusions STELLAR designs produce significant increases in teacher-learners’ abilities to teach for understanding and are more effective than traditional approaches. STELLAR collaborative tools are effective. There are order effects in contrasting cases instructional activities that inform design and support a schema-elaboration view of learning over several other theories. Targeted course outcomes are strongly related to effort, perceived task value, and successful use of tools that scaffold collaboration and site exploration.

25. A Former Student, A Reflective Practitioner

27. The Concepts of Professional Vision