Lessons Learned using the Classroom Experience Project How to fail at video conferenced teaching Jay Beavers, Microsoft Research Richard Anderson, Univ of Washington

UW Professional Master's Program Five year old program with established distance courses. Five year old program with established distance courses. One course per quarter video conferenced between UW and Microsoft. One course per quarter video conferenced between UW and Microsoft. One evening per week One evening per week Commuting distance Commuting distance

Before ClassroomXP technology ISDN Based video conferencing ISDN Based video conferencing Netmeeting for PowerPoint distribution Netmeeting for PowerPoint distribution Electronic white board Electronic white board Lecture archive Lecture archive Continual improvement in quality Continual improvement in quality  Lighting, microphones, audio processing, camera management, etc.

How to fail at video conferencing, Jim Gemmel  Voice quality must be competitive with telephony  Video should increase, not decrease ‘presence’  Starting up a video conference should be as easy as making a telephone call  Video conferencing must be ubiquitous

Potential advantages for using ConferenceXP Higher quality video and audio Higher quality video and audio Ease of setup and installation Ease of setup and installation Address some known problems Address some known problems  Whiteboard

Friendly Environment for ClassroomXP trials Substantial resources on both sides Substantial resources on both sides Established program Established program Near-located Near-located Existing relationship Existing relationship Experiment Experiment  Is higher quality video better  Better interactivity through ink on PPT

What we hoped to achieve Increased interaction between sites Increased interaction between sites Ability of remote students to interact with the instructor Ability of remote students to interact with the instructor Ability of instructor to engage remote students Ability of instructor to engage remote students Student interaction across sites Student interaction across sites No degradation of experience of local students No degradation of experience of local students No System Administrator No System Administrator

What happened Week 1 Week 1  Used old system (for comparison) Weeks 2-4 Weeks 2-4  Bad patch cable caused 30% packet loss, maiming Conferencing app Weeks 5-9 Weeks 5-9  Router incompatibilities caused intermittent multicast failure (~15 minutes between failure) Weeks Weeks  Worked as expected

How to fail at a distance course Transmission failures Transmission failures Production quality Production quality  Lighting, room noise, camera management Structured presentations limit activity Structured presentations limit activity Classroom issues Classroom issues  Table layout, overhead positionning Attitudes Attitudes

Transmission failures No connection – if the bits don’t get through, there is no class No connection – if the bits don’t get through, there is no class Interruptions – short failures Interruptions – short failures Degradation of performance (audio, video, signal and latency) Degradation of performance (audio, video, signal and latency)

Production quality Microphones Microphones  Not picking up sound  Picking up too much Lighting Lighting Camera quality Camera quality Field of view Field of view Distance Distance

Presentation Restrictions imposed by technology Restrictions imposed by technology Limitations on materials Limitations on materials Loss of fidelity of transmitted materials Loss of fidelity of transmitted materials

Classroom issues Placement of projectors Placement of projectors Dispersal of audience Dispersal of audience Student seating Student seating  Conference rooms versus lecture hall

Attitudes Resentment of technology Resentment of technology Not attempting to engage other site Not attempting to engage other site  Instructor – not encouraging questions  Students – not asking questions

Multicast Lack of reliability Lack of reliability Vender incompatibility issues Vender incompatibility issues Substantial time to localize problems Substantial time to localize problems Intermittent failures Intermittent failures

Multicast test tools Details on what is needed – it would be great to create interest in this Details on what is needed – it would be great to create interest in this

Seating Room Configuration Screen 1Screen 2 Seating

Content Issues Sound issues – difficulty of room(s). Sound issues – difficulty of room(s). Value of picking up background noise (sense of presence). Value of picking up background noise (sense of presence). Camera issues in remote room Camera issues in remote room  Configuration required students to sit at distance  Some students avoided sitting in field of view

Student Reactions I Repeated failures reduced students attempt to interact across sites Repeated failures reduced students attempt to interact across sites Even intermittent failures had significant psychological impact Even intermittent failures had significant psychological impact Students recognized that we were attempting to do a good job (we were present in the remote classroom, and shared their pain) Students recognized that we were attempting to do a good job (we were present in the remote classroom, and shared their pain)

Student Reactions II With our workarounds (high latency video, conference phone) students did not participate. With our workarounds (high latency video, conference phone) students did not participate. In spite of technical difficulties many students chose to attend the remote site (commuting 15 miles in traffic being an alternative) In spite of technical difficulties many students chose to attend the remote site (commuting 15 miles in traffic being an alternative) Local students complained of intrusiveness of interruptions. Had little sympathy for remote students. Local students complained of intrusiveness of interruptions. Had little sympathy for remote students.

Faculty Reaction With technological difficulties, taught primarily to local class With technological difficulties, taught primarily to local class Presentation tools readily accepted Presentation tools readily accepted

Take 2 Network diagnostics & testing Network diagnostics & testing  7x24 test agents  Long term statistics gathering & web reporting Concept of “Administrative Console” Concept of “Administrative Console”  Diagnostics to quickly determine fault  Over 20 points of potential failure in “I can’t hear the professor”  Non-invasive reset, diagnostics, configuration change Rooms dedicated to distance classes Rooms dedicated to distance classes  Lighting, wall color, ceiling mikes, projector & A/C noise, projector positioning, table layout Ease of startup & shutdown Ease of startup & shutdown