1 Multimedia Systems Course Instructor Samana Zehra

2 What will be covered today? Course Introduction  Course Objective  Contents  Resources  Grading Policy Lecture no.1

3 Course Objective To learn what are …  Different forms of multimedia  Principles of multimedia processing and presentation in computers  Multimedia applications  Multimedia networking

4 Course Contents Multimedia Basics Multimedia Formats Image Fundamentals Color Science Sound/Audio Video Compression Fundamentals Multimedia Applications Multimedia Networking

5 Resources Ze-Nian Li & Mark S. Drew, "Fundamentals of Multimedia", Pearson Education, 2004 Steinmetz R & Nahrstedt K, “Multimedia: Computing, Communication & Applications”, Pearson Education, 2001 F. Halsall, "Multimedia Communications: Applications, Networks, Protocols, and Standards", first edition, Addison-Wesley, 2000 James F. Kurose & Keith W. Ross, “Computer Networking – A top-down approach featuring the internet”, Third edition, Pearson Education, 2005

6 Grading Policy Theory Exam (100 marks) Sessionals (50 marks)  Internal (25 marks) Quizzes (4 to 5) Assignments (2 to 3)  can include presentations  External (25 marks) Viva May include some tasks/presentations

7 Lecture -1 The introduction begins…

8 Introduction Multimedia definitions:-  Multi means many; much; multiple  Medium means: a substance regarded as the means of transmission of a force or effect; a channel or system of communication, information or entertainment ( Merriam-Webster Dictionary )  Medium means: an intervening substance through which something is transmitted or carried on ( American Heritage Electronic Dictionary )  So Multimedia is medium having multiple content forms.

9 Introduction (Contd.) Multimedia includes a combination of text, audio, still images, animation, video, and interactivity content forms. Multimedia is usually recorded & played, displayed or accessed by computerized and electronic devices, but can also be part of a live performance. Multimedia also describes electronic media devices used to store and experience multimedia content. ( source: Wikipedia )

10 VVO Multimedia-terminal in Germany A PowerPoint presentation used in a corporate office Virtual reality uses multimedia content

11 Introduction (Contd.) Based on time-dimension in the representation space, media can be  Time-independent (Discrete) Text, Graphics  Time dependent (Continuous) Audio, Video Video, sequence of frames (images) presented to the user periodically. Time dependent aperiodic media is not continuous!! Note: Discrete & Continuous have no connection with internal representation.

12 History of Multimedia Newspaper - perhaps the first mass communication medium to employ Multimedia - they used mostly text, graphics, and images Marconi sent his first wireless radio transmission at Pontecchio, Italy Maroni detected radio waves beamed across the Atlantic. Initially invented for telegraph, radio is now a major medium for audio broadcasting. Television - new media for the 20th century. It brings the video and has changed the world of mass communications. Some of the important events in relation to Multimedia in Computing include:  Bush wrote about Memex  Negroponte formed the Architecture Machine Group at MIT  Nelson & Van Dam hypertext editor at Brown  Birth of The Internet

13 History of Multimedia (Contd.) Architecture Machine Group proposal to DARPA: Multiple Media Lippman & Mohl: Aspen Movie Map Backer: Electronic Book Negroponte, Wiesner: opened MIT Media Lab Tim Berners-Lee proposed the World Wide Web to CERN (European Council for Nuclear Research) K. Hooper Woolsey, Apple Multimedia Lab, 100 people, educ Apple Multimedia Lab: Visual Almanac, Classroom MM Kiosk the first M-bone audio multicast on the Net U. Illinois National Center for Supercomputing Applications: NCSA Mosaic Jim Clark and Marc Andreesen: Netscape JAVA for platform-independent application development. Duke is the first applet Microsoft, Internet Explorer.

14 Multimedia Authoring Tools Multimedia elements are composed into a project using authoring tools. Multimedia Authoring tools are those programs that provide the capability for creating a complete multimedia presentations by linking together objects such as a paragraph of text (song), an illustration, an audio, with appropriate interactive user control

15 Multimedia Authoring Tools (Contd.) By defining the objects' relationships to each other, and by sequencing them in an appropriate order, authors (who use authoring tools) can produce attractive and useful graphical applications. To name a few authoring tools  Macromedia Flash  Macromedia Director  Authorware The hardware and the software that govern the limits of what can happen are multimedia platform or environment

16 Multimedia and Interactivity Multimedia is interactive when the end-user is allowed to control what and when the elements are delivered. Interactive Multimedia is Hypermedia, when the end-user is provided with the structure of linked elements through which he/she can navigate.

17 Multimedia and Interactivity Multimedia is linear, when it is not interactive and the users just sit and watch as if it is a movie, e.g., cinema presentation Multimedia is nonlinear, when the users are given the navigational control and can browse the contents at will, e.g., computer game, computer-based training.

18 Multimedia System A Multimedia System is a system capable of processing multimedia data and applications. A Multimedia System is characterised by the processing, storage, generation, manipulation and rendition of Multimedia information.

19 Characteristics of Multimedia System A Multimedia system has four basic characteristics:  Multimedia systems must be computer controlled.  Multimedia systems are integrated.  The information they handle must be represented digitally.  The interface to the final presentation of media is usually interactive.

20 Challenges for Multimedia System Supporting multimedia applications over a computer network renders the application distributed. Multimedia systems may have to render a variety of media at the same instant; a distinction from normal applications. There is a temporal relationship between many forms of media (e.g. Video and Audio). In this case problems are: Sequencing within the media - playing frames in correct order/time frame in video Synchronisation - inter-media scheduling (e.g. Video and Audio). Lip synchronisation is clearly important for humans to watch playback of video and audio and even animation and audio.

21 Challenges for Multimedia Systems The key issues that multimedia systems need to deal with are:  How to represent and store temporal information.  How to strictly maintain the temporal relationships on play back/retrieval  What process are involved in the above.  Data has to be represented digitally so many initial source of data needs to be digitised - translated from analog source to digital representation. This will involve scanning (graphics, still images), sampling (audio/video) although digital cameras now exist for direct scene to digital capture of images and video.  The data is large (several Mb) for audio and video - therefore storage, transfer (bandwidth) and processing overheads are high. Data compression techniques very common.

22 Desirable Features Given the (mentioned) challenges the following features are desirable (if not a prerequisite) for a Multimedia System:  Very High Processing Power needed to deal with large data processing and real time delivery of media. Special hardware commonplace.  Multimedia Capable File System needed to deliver real-time media e.g. Video/Audio Streaming. Special Hardware/Software needed e.g RAID technology.  Data Representations/File Formats that support multimedia Data representations/file formats should be easy to handle yet allow for compression/decompression in real-time.  Efficient and High I/O input and output to the file subsystem needs to be efficient and fast. Needs to allow for real-time recording as well as playback of data. e.g. Direct to Disk recording systems.

23 Desirable Features  Special Operating System to allow access to file system and process data efficiently and quickly. Needs to support direct transfers to disk, real-time scheduling, fast interrupt processing, I/O streaming etc.  Storage and Memory large storage units (of the order of Gb or more) and large memory ( Mb or more). Large Caches also required and frequently of Level 2 and 3 hierarchy for efficient management.  Network Support Client-server systems common as distributed systems common.  Software Tools user friendly tools needed to handle media, design and develop applications, deliver media.

24 Multimedia Applications Multimedia finds its application in various areas, a few are:  Education Computer-based trainings, reference e-books, documents imaging, distant learning.  Entertainment Interactive games  Medicine Training to doctors through virtual surgery  Business Employee training, advertisements, customer support, selling products.  Arts Art exhibitions, literature presentations  Scientific Research Modeling and simulations

25 Multimedia Applications (Contd.) In Medicine Source: Cardiac Imaging, YALE centre for advanced cardiac imaging

26 Multimedia Applications (Contd.) In training

27 Multimedia Applications (Contd.) Public awareness campaign Source Interactive Multimedia Project Department of food science& nutrition, Colorado State Univ

28 Multimedia Applications (Contd.) Other Applications include:  World Wide Web  Video conferencing  Video-on-demand  Interactive TV  Home shopping  Games  Virtual reality  Digital video editing and production systems  Multimedia Database systems

29