1. March 26, 2007Microsoft Research India1 Tutored Video Instruction and Course Export Richard Anderson University of Washington

2. March 26, 2007Microsoft Research India2 Course Export Offer established courses at external institutions Remote institution is not in a position to offer the course

3. March 26, 2007Microsoft Research India3 Tutored Video Instruction Recorded video basis of instruction Video watched in a group setting with a facilitator Model – alternate between video and discussion Technology has radically changed costs –Capture –Distribution –Replay

4. March 26, 2007Microsoft Research India4 TVI Projects Gibbons, 1974-1980 –Stanford University and HP –Master’s Level Engineering Courses UW, 1997-2000 –University of Washington and Community Colleges –Introductory Computer Programming Digital Study Hall, 2005- –India, Urban and Rural Schools –Primary education UW, 2006 –University of Washington and Beihang University, Beijing –Senior level Computer Science Course (Algorithms)

5. March 26, 2007Microsoft Research India5 What factors influence success or failure of TVI? Initial measures of success: –Impact on students –Sustainability Additional measures of success: –Impact on facilitators –Institutional relationships

6. March 26, 2007Microsoft Research India6 Gibbons, Stanford Master’s level courses offered between Stanford University and HP Engineering sites –Offer Stanford classes to remote degree candidates –Students received Stanford Credits –Centralized grading Courses recorded live –Single camera –Distributed by videotape –Video quality poor – e.g., blackboard writing illegible –Quality of instructor considered important

7. March 26, 2007Microsoft Research India7 Stanford Small sections led by facilitators –3-10 students –Model – stop video frequently for discussion Once every five minutes or when question Goal – student initiated discussion –Peer facilitation Theory that discussion would be better with peer facilitation Student who took the course the previous year

8. March 26, 2007Microsoft Research India8 Positive results reported from Stanford Experiments

9. March 26, 2007Microsoft Research India9 UW-Community Colleges Introductory Computer Science courses from UW offered at Community Colleges State of Washington Higher Education Model UW Branch Campuses WSU CC

10. March 26, 2007Microsoft Research India10 UW - CC Problems being addressed –Lack of qualified CS Instructor’s at CC’s –Courses not available at CC’s –Transfer students not having an equivalent background Political sensitivities between UW and CC’s

11. March 26, 2007Microsoft Research India11 UW-CC TVI Materials recorded from Live UW lectures –Talking head + slides –Shown in CC courses with CC instructors as facilitators –Wanted to have instructors be “peers” Grading done by UW

12. March 26, 2007Microsoft Research India12 UW-CC Two phases of project Phase I –Live materials recorded at UW –Grading Centralized –Substantial management from UW Phase II –Studio created materials –Grading handled by CC’s

13. March 26, 2007Microsoft Research India13 Digital StudyHall Facilitated video instructor for primary education in rural India –Severe resource constraints –Limited teachers Hub and spoke model –Multiple sites (Lucknow, Bangalore, Pune) Capture of live-staged lectures

14. March 26, 2007Microsoft Research India14 UW - Beihang Pilot course offering Algorithms course taught at UW and Beihang University Beihang students received Beihang credit for the course Goal was to offer course using US pedagogy and content

15. March 26, 2007Microsoft Research India15 UW-Beihang Materials captured from live classes –Slides, talking, head, digital ink Classroom Technology –Students used Tablet PCs to participate in classroom activities –Tablets PCs used both at Beihang and UW

16. March 26, 2007Microsoft Research India16 Facilitation Three sections of 25 students each in Beihang Teaching Assistants led each section Instructional materials in English, but much of the discussion was in Chinese

17. March 26, 2007Microsoft Research India17 High degree of UW involvement Initial visit –Set up –Training –Introduction to the students Second visit –Observation –Deliver class Support materials –Lecture summaries

18. March 26, 2007Microsoft Research India18

19. March 26, 2007Microsoft Research India19 General Lessons Were these courses successful? Stanford –Considered successful in terms of outcomes UW-CC –Initial offerings had mixed results –Some of the sections could be considered successful –Program was not sustainable UW-Beihang –Pilot offering, student outcomes and evaluations were positive

20. March 26, 2007Microsoft Research India20 Institutional Relationship Student View –Stanford – favorable –UW CC – negative, fear of competition –UW Beihang – favorable Facilitator View –UW CC – mixed Resentful Supportive of program Personal advancement Positive because course couldn’t be offered without support –UW Beihang Favorable

21. March 26, 2007Microsoft Research India21 Facilitation I Peer vs. non-peer –Stanford – peer –UW CC – non-peer –Beihang - peer Often extensive involvement of source site –Negative: Limits scaling –Positive: Mentorship

22. March 26, 2007Microsoft Research India22 Facilitation II Relationship between facilitator and external teacher –UW CC: Mixed Competitive or supportive Reflected general attidudes –Beihang Teacher / TA

23. March 26, 2007Microsoft Research India23 Facilitation III Gibbons –Encourage students to ask questions to figure out the material UW –Co-teaching with recorded materials –Facilitators initiated interaction (not students) Broad range of techniques Preparation by facilitators important

24. March 26, 2007Microsoft Research India24 Institutional Course Adoption Transfer of curriculum –Institution adopts curriculum or teaching style Transfer of expertise –Facilitators gaining experience to teach the course

25. March 26, 2007Microsoft Research India25 Delivery Mechanics Course materials –Gibbons – capture of live materials –UW CC Version 1 – Live Version 2 – Studio –UW Beihang Live – designed for TVI Supporting materials –UW CC – classroom activities –UW Beihang – lecture summaries

26. March 26, 2007Microsoft Research India26 Evaluation Learning outcome evaluation –Established deployments Robust technology, methodology, programs –Autonomous –Multiple instances –Control –Education experts

27. March 26, 2007Microsoft Research India27 Additional approaches to evaluation Learning outcomes are not the only metric –Broader range of impact –Richer understanding

28. March 26, 2007Microsoft Research India28 Evaluation Student outcomes –How do students perform –Attendance Student reactions –Survey Student satisfaction Evaluation of different components –Short answer Facilitator reaction –Interviews

29. March 26, 2007Microsoft Research India29 Understanding classroom behavior The TVI methodology has certain classroom goals –Are these achieved? Observation Data logging –Observational (e.g., rounds of communication) –Technology logs Digital artifacts

30. March 26, 2007Microsoft Research India30 What I have learned about TVI Relationship between institutions/partners is absolutely critical –Inherently asymmetric –Motivations open to question Role of “source” site –Develop materials that are suitable for target population and facilitated playback No general guidelines –Important to support facilitators Training Support materials

31. March 26, 2007Microsoft Research India31 What I learned about TVI Potential long term impacts on facilitators and institutional relationships Facilitation –Co-instruction by facilitator and video (not Peer instruction as proposed by Gibbons) –Wide range of practices adopted by facilitators –Facilitator training and attitude important

32. March 26, 2007Microsoft Research India32 For more information, contact Richard Anderson (anderson@cs.washington.edu)anderson@cs.washington.edu UW-Beihang course website: Papers This work has been supported by Microsoft External Research and Programs, University of Washington, and Microsoft Research Asia Questions? http://www.cs.washington.edu/education/courses/cse421/06au/ http://www.cs.washington.edu/education/dl/presenter/papers.html