Concertation meeting, September 2002Oliver Schreer1 VIRTUE – VIRtual Team User Environments Oliver Schreer Heinrich-Hertz-Institute, Berlin Germany

Concertation meeting, September 2002Oliver Schreer2 Outline  The challenge of VIRTUE  The VIRTUE station  The VIRTUE architecture  The developed hardware  Conclusion

Concertation meeting, September 2002Oliver Schreer3 Local Conferees sitting in front of large immersive mono- or stereoscopic displays such that they feel immersed in shared virtual environment Remote Conferees represented by full-body sized 3D video images as natural as possible (no avatars please !!) Conference Situation exact reproduction of eye contact, gaze, gestures and body language realistic viewing conditions by support of head motion parallax The challenge of VIRTUE

Concertation meeting, September 2002Oliver Schreer4 The VIRTUE terminal one year ago Current State of the VIRTUE Station (e.g.: 50’’ plasma display) Terminal Design (e.g.: 80’’ rear projection)

Concertation meeting, September 2002Oliver Schreer5 The current state 4 Pulnix Colour Progressive Scan Cameras Metal framework for mounting cameras and speakers Audio Speakers Access to 3 compartments at the rear of the terminal for PCs and other hardware Semi-circular table 61” Plasma display

Concertation meeting, September 2002Oliver Schreer6 VIRTUE – 2D Demonstrator at IST Event 2001 See video clips at

Concertation meeting, September 2002Oliver Schreer7 VIRTUE Architecture

Concertation meeting, September 2002Oliver Schreer8 VPS Board (Virtue Processor Station) TriMedia TM1300, 143 MHz4 x 32 Mbyte SDRAM Internal PCI bus connecting all TriMedia’s Connection to PCI of host PC via PCI bridge 4 x VideoIn/VideoOut CCIR601/656 compliant connecting the interface board

Concertation meeting, September 2002Oliver Schreer9 SDI/LVDS Interface (SMPTE 259M) 4 x LVDS In/Out 4 x VideoIn/VideoOut CCIR601/656 compliant connecting the VPS board Router FPGA for flexible VideoIn/VideoOut connections 4 x SDI In 4 x SDI Out

Concertation meeting, September 2002Oliver Schreer10 Complete VPS package  Using Multimedia and SIMD capabilities of the TriMedia  Arbitrary VideoIn/VideoOut with interface board  Flexible: C++ code can be used, modified and upgraded directly  Modular: Multiple boards can be plugged in 32 bit/64 bit slots  Based on standard PCI technology

Concertation meeting, September 2002Oliver Schreer11 The MPEG-2 Low Delay Coder MPEG-2 Encoder: 4:2:0 transport stream Encoding latency < 80ms Image size 640x480/25 fps Bitrate 2 Mbit/s MPEG-2 Decoder: Decoding latency < 80ms VIRTUE camera link output MPEG-2 PCI Base Board: 66 MHz/64 Bit universal PCI card 4 MPEG-2 Encoder modules 4 MPEG-2 Decoder modules VIRTUE camera link output

Concertation meeting, September 2002Oliver Schreer12 Conclusions and Outlook  Real-time hardware fullfilling the demands  Video streaming without loading the host PCI bus  Completely PCI based  Flexible in terms of software update, modification and extension  Successful 2D demonstrator presented at IST event 2001  Complete 3D video conferencing system in September 2002  MPEG2 encoding, transmission, 3D analysis, view synthesis, rendering, audio, document sharing  Final human factors test in October 2002

Concertation meeting, September 2002Oliver Schreer13 Announcement Workshop on „Immersive Communication and Broadcast Systems“ January 2003 Heinrich-Hertz-Institut (HHI), Berlin, Germany Kontakt: Oliver Schreer,

Concertation meeting, September 2002Oliver Schreer thank you for your attention END