1. Media Objects in Time A Multimedia Streaming System Björn Feustel, Thomas Schmidt {feustel,schmidt}@fhtw-berlin.de FH für Technik und Wirtschaft Berlin Computer Centre

2. Agenda  Introductory Ideas  Media Object Model  MIR – A Media Information Repository  Media Objects in Time  Synchronized Media Streams  Compound Flow Model  Wavelet Web Video  Conclusions and Outlook

3. Ideas Networked multimedia revolutionizes teaching as did the invention of blackboard and chalk: Online aid in the lecture hall – Teachers load multimedia presentation objects to supplement lectures Telelearning at home – Students recall from and work with full web applications We face an urgent demand for a heterogeneous multitude of multimedia applications

4. Educational Task Provide an Environment for easy implemen- tation of specific educational applications supporting:  Free definition of information layers  General media handling incl. Streaming  Non-linear information structuring  Time-dependent presentation  Event-based interactions

5. Time in Teaching Perception is a time-dependent process:  The Notion of Time is vital to Teaching –A timeline provides speed and (some) linear orderings (as trad. teachers do) –Time control processes determine performance  Multimedia introduces Time-based Material: – Audio, video, online data processing – Synchronisation tasks arise  Process Synchronous Scalable Media Streams

6. Modeling Media Objects in Time Temporal and spatial alignment rises a new level of complexity: Temporal media placement needs playscripts Authoring challenging in geometry+time state space Individual object preservation : pure video-type streaming ansatz ? Fascinating interaction potentials within analytically preserved object model allows for scene hopping  Underlying Logic of an Object Model needed!

7. Media Object Model Media Object (MOB) –Neutral container of MOBs or data –Arbitrary annotations from OO classes –Reusable, self-consistent component –Mob composition open to semantics Active Document Structures – Flexible object reference model – Event-type actions imposable along referential structures

8. Media Information Repository Fundamenting Multimedia Applications www.rz.fhtw-berlin.de/MIR

9. Media Information Repository Media Object Database –General storage of multimedia data –Provides flexible information structuring –Supports arbitrary annotations Mir Authoring Environment – Database editing in the Web – Application specific interface Mir Architecture – General/adaptable access for Clients – Pluggable application subservers

10. Media Object Database

11. MIR Authoring

12. MIR Architecture

13. Media Objects in Time

14. Synchronized Media Streams Accurate scheme for temporal and spatial placement of presentation component: Timeless objects s.a. text, images, etc. Time-streamed media s.a. video, audio, etc. Any composition of the above are combined in a synchronous Web stream for Browser Display

15. C ompound F low M odel  Intuitively structured object logic: Structuring implements temporal & spatial inclusion relation  Reusable complex presentation components  Media Objects as central composition entities include Playscripts  Time based flow generator  Suitable for light weight client implementation

16. MobIT Media Objects  Universal container for MOBs or media elements  Object reference list + Playlist  Local co-ordinate system in time and space  Congruence of structure and state space:  Referential self-consistency  Object structure with temporal and spatial meaning (inclusion)  Scalability and Parametrizability

17. Media Object Structure

18. Flow Generator Object resolution and time linearization Linearization suitable for dynamic change

19. Wavelet Web Video  Joint Work with H. Cycon and Group  JAVA implementation of real-time Wavelet PACC Codec  Scalable Video Streamer at low bandwidth (QCIF at 12 fps < 64 kbit/s)  CIF (352 x 288 pixels) at 5 fps performs in real-time within an applet on a 500 MHz Pentium Machine

20. Architecture and Implementation 3-tiered Open Multimedia Architecture Intelligent Multimedia Database System Additional lightweight XML-Datastore Subserver Instances for Streaming Media or Data Processing Wavelet compressed Web Video Interfaces for additional Media Types or Pluggable Subservers Pure Java Implementation for Universal Use

21. Conclusions & Outlook  Universal Media Database.  Application defined Annotations.  Active Document Structuring.  Reusable Presentation Components.  Web Authoring.  Time Synchronization for General Types of Media.  Scalable Video streams in Wavelet compression.  Interaction Model under way.

22. Virtual Design

23. Knowledge Market Place