1. The Physics Education Technology Project http://phet.colorado.edu Kathy Perkins & the rest of the PhET Team University of Colorado at Boulder

2. PhET Project Physics faculty: Michael Dubson Noah Finkelstein Kathy Perkins Carl Wieman (Director) Postdocs: Sam McKagan Linda Koch Teacher collaborators: Trish Loeblein Staff: Krista Beck Linda Wellmann Ph. D. students: Wendy Adams Chris Keller Noah Podolefsky Pat Kohl Danielle Harlow Programmers: Ron LeMaster Sam Reid Chris Malley Michael Dubson Website: Chris Maytag ~5-6 full time equivalents

3. PhET Funding NSF Kavli Foundation Hewlett Foundation University of Colorado Alfred Nobel

4. Outline Introduction to PhET Design Process and Role of Research/Assessment Research and Assessment of  Design: User interface: interpretation / effectiveness For learning & engagement in exploration/discovery  Use: Where and how: lecture, lab, recitation, homework Assessment: When these work (and don’t)

5. Physics Education Technology Project Suite of interactive simulations (~60) Covering intro physics, modern physics, bit of chemistry Research-based and user-tested Free! Online or downloadable. (~70 Mbytes) Easy to use and incorporate in class http://phet.colorado.edu Show website

6. Accessibility  Universal, web-based languages Java or Flash  3 ways to run sims: directly from web download whole website to local computer for offline use (new) download selected sims for offline use In last year: 1 Million sims launched; 50,000 site downloads  Easy to translate into other languages Mirror site in Spanish

7. Ease of use  Flexible use Pick and choose which sims to use Customize use to your environment  Facilitate effective use with Activities Database Community of users share ideas/activities Over 50 PhET and user-contributed activities

8. Research Base Learning Goals Initial Design ~Final Design Redesign Interviews Classroom Use Design Process 

9. Initial Design & General Approach Research Base Learning Goals Initial Design Research base: Ed. Psych / Cog. Sci: How people learn Educational Software Design Student Conceptions in Physics PhET research findings

10. Fun and engaging w/ open-style play area Highly interactive Dynamic feedback. Interaction links to animation. Explore and discover (constructivist) Connection to real world Explicit visual & conceptual models (that experts use) Productive constraints K.K. Perkins, et al, “PhET: Interactive Simulations for Teaching and Learning Physics”, Physics Teacher (Jan 2006) Initial Design & General Approach

11. Visual Models

12. Research Base Redesign Interviews Assessment of Design: Usability – easy/intuitive Interpretation – correct/productive Engaged exploration Can students construct understanding of main ideas? Achieve learning goals? GeneralDesignGuidelines Paper to be submitted: Wendy Adams et al.

13. Research Base Redesign Interviews Think-aloud styleThink-aloud style Either no guidance or limited to one or two conceptual questions.Either no guidance or limited to one or two conceptual questions. 30 to 60 minutes (1-3 simulations)30 to 60 minutes (1-3 simulations) 4-6 interviews per version of simulation4-6 interviews per version of simulation Assessment of Design: Interviews

14. Intuitive ControlsIntuitive Controls –Click and Drag –Grab-able Objects RepresentationsRepresentations –Cartoon-like features Limited Text - Students do not readLimited Text - Students do not read Start-up for engagementStart-up for engagement –Little or no animation –“Wiggle me’s” Assessment of Design: Findings from Interviews Research Base Redesign Interviews Full listing of guidelines will be in Adams et al.

15. Research Base Classroom Use Use of Sims: Course (Level & Goals) Setting (Lecture/Lab) Task / Activity Frames of Context Student Well honed tool for learning  Doesn’t guarantee its effectiveness: Depends on context

16. Research Base Classroom Use Use of Sims: Course (Level & Goals) Setting (Lecture/Lab) Task / Activity Frames of Context Student Pre-class or pre-lab Activity Lecture/classroom Visual Aids, Interactive Lecture Demos, & Concept tests Labs/Recitations Group activities Homework

17. Study the comparative use and performance of students in: Labs Lectures Recitation Homework Assessment of Use (Chris Keller) Circuit Construction Kit Assess: student learning student enjoyment student valuation

18. vs. N D. Finkelstein, et al, “When learning about the real world is better done virtually: a study of substituting computer simulations for laboratory equipment,” PhysRev: ST PER 010103 (Sept 2005) p < 0.001 DC Circuit Exam Questions Traditional Laboratory (Alg-based Physics): Simulation vs. Real Equipment

19. Challenge: Build real circuit Make break Explain what happens and why? Laboratory (Alg-based Physics): Build a real circuit and describe it

20. 0 5 10 15 20 25 Time (min) CCK TRAD Timing 0% 20% 40% 60% 80% 100% TRADCCK Percent of Students Explanation Score 3 2 1 0 Laboratory (Alg-based Physics): Build a real circuit and describe it

21. Interactive Recitation Study Reformed large-scale introductory calculus-based physics course with Tutorials “CCK” (N=180)“Real” (N=185) vs. Keller, C.J. et al. Assessing the effectiveness of a computer simulation in conjunction with Tutorials in Introductory Physics in undergraduate physics recitations,", PERC Proceedings 2005

22. Conceptual Understanding (near term) Interactive Recitation Study p=0.01

23. DC Circuit Q's on Post-BEMA Avg 3433 3217 1211 10 9 8 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Questions Fraction Correct CCK Real * and added 3 DC questions (32-34; McDermott), BEMA- Ding, L. et al. Phys Rev: ST PER (2006). Conceptual Understanding (longer) Interactive Recitation Study

24. Context dependence? Different populations Alg-based vs. Calc-based Different activities Traditional vs PER-based Different courses Instruction: More traditional vs More interactive HW/Exams: More traditional vs More conceptual Our Research Goal: Assess use across these sorts of dimensions Characterize and understand. Identify where use is effective at improving learning

25. vs. Influence on Environment & Learning Lecture (Calc-based w/ peer instruction) Sims vs. Pictures/Words/Demo

26. 0 20 40 60 80 100 Percent Correct c Lect 1 CCK Lect 2 No CCK CCK vs. pictures/word/demo Influence on Environment & Learning Lecture (Calc-based w/ peer instruction) Pictures/word/demo 0 20 40 60 80 100 Percent Correct Lect 1 No sim Lect 2 No sim Silent Post Discussion Sims vs. Pictures/Words/Demo

27. When the string is in position B, instantaneously flat, the velocity of points of the string is... A: zero everywhere. B: positive everywhere. C: negative everywhere. D: depends on the position. A B C snapshots at different times. Wave-on-string sim vs Tygon tube demo Follow up question: At position C, the velocity of points of the string is... A: zero everywhere. B: positive everywhere. C: negative everywhere. D: depends on the position. Correct : 2002 demo: 27% 2003 sim: 71% Correct : 2002 demo: 23 % 2003 sim: 84% Follow-up Concept Test: Lecture (Non-science Majors Course) Sims vs. Pictures/Words/Demo Influence on Environment & Learning

28. Integrating a sim on a topic (Photoelectric Effect in Modern Physics) CourseQ1Q2Q3N UW w/o PT65402026 UW w/ PT75854036 % Correct University of Washington: Studied student learning of photoelectric effect Developed & used Photoelectric Tutor (PT) Exam Q: What would happen to current reading if you: Q1: Changed metal. Why? Q2: Double intensity of light. Why? Q3: Increased  V across electrodes. Why? CU: Incorporated sim CU Fa05888584189 CU Sp06788479182

29. LabsDateTopic 1Aug. 22Electric Charge 2Aug. 29Potential & Fields 3Sept. 17Circuits 1, Voltage 4Sept. 26Circuits 2, Current 5Oct. 3Magnets 6Oct. 17Alternating Current 7Oct. 25EM Waves 8Oct. 31Lenses & Optics 9Nov. 14Diff, Interf, Lasers 10Nov. 28Spectroscopy Intervention Assessing Student Views Redesigned Laboratory (Alg-based Physics):

30. Pre-Lab Questions 1. I thought this lab was _____ USEFUL for learning about... (very, pretty, somewhat, barely, not at all) 2. I thought this lab was ____ ENJOYABLE for a physics lab. (very, pretty, somewhat, barely, not at all) Assessing Student Views

31. 1 1.5 2 2.5 3 3.5 4 4.5 5 123456789 Average (1=Not-at-all to 5=Very) Perceived Utility Assessing Student Views Usefulness of Labs Labs

32. 1 1.5 2 2.5 3 3.5 4 4.5 5 123456789 Enjoyment Assessing Student Views Average (1=Not-at-all to 5=Very) Enjoyment of Labs Labs

33. Assessing Student Views Physics of Everyday Life: 3.7 Whole course integration Modern Physics for Engineers: 4.0 Perceived Utility of sims for learning (1=Not useful; 5=A great deal…) “Great sims, I can't imagine QM without them.”

34. Conclusions PhET: suite of interactive sims, free, easy to use. Assessment is critical for evaluating effectiveness of design and use any tools. Findings from the design of PhET sims may be useful for others designing online tools. While sims are designed for productive/effective learning, critical to attend to how we use them. Research needs to identify, characterize, and understand what makes use of PhET sims effective at improving learning and how this depends on context. Get PhET: http://phet.colorado.edu

35. Extra slides below

36. So what’s in activity design? PhET Team Approach to Curriculum Design: Guided Inquiry Approach Does the activity … Address all of your learning goals? Require active thinking? Require sense making / reasoning? Build on prior knowledge? Connect to real world? Help students monitor their understanding?

37. Electrostatics – Traditional balloon demos - Charge transfer, Coulomb attraction, Polarization Lecture – Demo complement Show balloons Simple, but effective

38. Lecture – Interactive Lecture Demos Demo 4: Sketch position vs time and velocity vs time graphs for when Moving Man: walks steadily towards the tree for 6 seconds, then stands still for 6 seconds, and then towards the house twice as fast as before for 6 seconds. + 0 - Position time Velocity + 0 - time 5 s10 s20 s15 s Thorton and Sokoloff, 1997

39. Velocity + 0 - time + 0 - Position time 5 s10 s20 s15 s Velocity + 0 - time + 0 - Position time 5 s10 s20 s15 s Velocity + 0 - time + 0 - Position time 5 s10 s20 s15 s AB C Velocity + 0 - time + 0 - Position time 5 s10 s20 s15 s D

40. Electromagnetic waves: Radio Waves sim Concept Tests and Peer Instruction Lecture – Concept tests The speed of the wave (signal) is measured as… a.how fast this peak moves to the right. b.how fast this peak moves up and down. c.could be a or b