2. Multimedia Systems What is a multimedia system? A multimedia system supports the integrated storage, transmission and representation of the discrete media types text, graphics and image and the continuous media types audio and video on a digital computer.

3. Multimedia Systems Early multimedia system, around 1985 Computers control the analog media streams, e.g., via cross-connect switches.

4. Multimedia Systems Current Digital Multimedia System The media streams are digital. They can be processed (e.g., compressed/decompressed, analyzed) in the computer.

5. Multimedia Systems Scope

6. Multimedia What is Multimedia? When different people mention the term multimedia, they often have quite different, or even opposing, viewpoints. – A PC vendor: a PC that has sound capability, a DVD-ROM drive, and perhaps the superiority of multimedia-enabled microprocessors that understand additional multimedia instructions. – A consumer entertainment vendor: interactive cable TV with hundreds of digital channels available, or a cable TV-like service delivered over a high- speed Internet connection. – A Computer Science (CS) student: applications that use multiple modalities, including text, images, drawings (graphics), animation, video, sound including speech, and interactivity. Multimedia and Computer Science: – Graphics, HCI, visualization, computer vision, data compression, graph theory, networking, database systems. Multimedia and Hypermedia

7. Multimedia Components of Multimedia Multimedia involves multiple modalities of text, audio, images, drawings, animation, and video. Examples of how these modalities are put to use: 1. Video teleconferencing. 2. Distributed lectures for higher education. 3. Tele-medicine. 4. Co-operative work environments. 5. Searching in (very) large video and image databases for target visual objects. 6. “Augmented” reality: placing real-appearing computer graphics and video objects into scenes.

8. Multimedia Components of Multimedia (contd from last slide…) 7. Including audio cues for where video-conference participants are located. 8. Building searchable features into new video, and enabling very high- to very low-bit-rate use of new, scalable multimedia products. 9. Making multimedia components editable. 10. Building “inverse-Hollywood” applications that can recreate the process by which a video was made. 11. Using voice-recognition to build an interactive environment, say a kitchen-wall web browser.

9. Multimedia Current Multimedia Projects which are ongoing: Many exciting research projects are currently underway. Here are a few of them: 1.Camera-based object tracking technology: tracking of the control objects provides user control of the process. 2. 3D motion capture: used for multiple actor capture so that multiple real actors in a virtual studio can be used to automatically produce realistic animated models with natural movement. 3. Multiple views: allowing photo-realistic (video-quality) synthesis of virtual actors from several cameras or from a single camera under differing lighting. 4. 3D capture technology: allow synthesis of highly realistic facial animation from speech.

10. History Of Multimedia What can we say about the evolution of media that has taken place for thousands of years? Since the dawn of time, people have had the need to communicate with one another. This created what we called as communication media. http://www.december.com/present/mediaev.html

11. History Of Multimedia Newspapers  (perhaps) the first mass communication medium, which used mostly text, graphics, and images. 1895  Gugliemo Marconi sent hisfirst wireless radio transmission at Pontecchio, Italy. 1901  he detected radio waves beamed across the Atlantic. Initially invented for telegraph, radio is now a major medium for audio broadcasting.

12. History Of Multimedia Television  new media for the 20th century. It brought video (+audio) and has since changed the world of mass comm.

13. Some Important Events In Computer History 1945 - Vannevar Bush (1890-1974) wrote about Memex –a device in which an individual stores all his books, records, and communications, and which it is mechanized so that it may be consulted with exceeding speed and flexibility Vannevar Bush

14. Engelbart Some Important Events In Computer History 1960s - Ted Nelson started Xanadu project (The Original Hypertext Project) 1968 - Douglas Engelbart demonstrated NLS (oN Line System) - The Debut of The Mouse 1969 - Nelson & Van Dam hypertext editor at Brown University.

15. Some Important Events In Computer History 1976 - Architecture Machine Group proposal to DARPA: Multiple Media 1985 - Negroponte, Wiesner: opened MIT Media Lab in Boston 1989 - Tim Berners-Lee proposed the World Wide Web to CERN (European Organization For Nuclear Research) 1990 – Kristina Hooper Woolsey headed Apple Multimedia Lab, 100 staff, for education

16. Some Important Events In Computer History 1993 - U. Illinois National Center for Supercomputing Applications: NCSA Mosaic 1994 – Jim Clark and Marc Andreessen: Netscape 1995 - JAVA language for platform-independent application development. 1998 – XML 1.0 announced as a W3C recommendation 1998 – Handheld MP3 devices (32MB) flash memory 2000 – WWW size estimated over 1 billion pages.

17. Overlapping Technologies Different branches of multimedia grow together because of new, upcoming multimedia technology and applications. Two challenges lie ahead: –Timing requirements (synchronization etc.) –Integration requirements (of different media types)

18. Multimedia The notion of Multimedia Consists of two words: Multi (Latin)= many; much; Medium (Latin) = An intervening substance through which something is transmitted or carried on.

19. Multimedia What is Multimedia? Multimedia means a (usually) interactive combination of two or more media elements (multimedia building block), such as text, graphics, audio, video and animation integrated using a computer A multimedia system is a system that supports more than a single type of media.

20. Multimedia Building Block Digital environment USER Elements of Multimedia

21. Multimedia Multimedia has a number of distinct and unique features, including: –Based on Edgar Dale (Cone Of Learning), on average, people remember: 10% of what they read, 20% of what they hear, 30% of what they see, 50% of what they hear and see,  multimedia approach –multimedia rich elements, multi-sensory delivery system can facilitate greater retention of new knowledge

22. Multimedia Modalities Modalities are the sensory systems through which a multimedia activity occurs This includes tactile (touch), gustatory (taste), visual (sight), auditory (hearing), olfactory (smell) Based on the multimedia elements we have today, only two modalities are regularly used.

23. Multimedia Channels Channels can be understood as existing within a modality. For example, with the auditory modality, we have different channels for noises, speech and music. With the visual modality, we have different channels for words, pictures and movies.

24. Multimedia Channels Bandwidth is a concept of how much information can be carried by a certain channel within a certain modality. For example, we can read at the rate of 150 words per minute which is the ‘printed text’ channel within the ‘visual’ modality. Much like your modem, you are unlikely to reach the theoretical maximum bandwidth of your channel within any modality, in practice.

25. Multimedia Channels The reason for this discrepancy in practice is because in theory, we assume a perfect encoder and decoder. For example, English text ‘encoded’ on a page and the English language ‘decoder’ in someone’s head is assumed to be perfectly compatible. In reality, however, it is highly dependent on the education level of the reader, the nature of the text information presented and many other factors.

26. Multimedia ‘Medium’ A medium can be understood as a set of co- ordinated channels, spanning one or more modalities, which have come to be referred to as a unitary whole, and which possess a cross- channel language of interpretation. Examples include a television show, which typically uses the auditory and visual modalities; and picture, written text, speech and music channels.

27. Multimedia System Characteristics Multimedia systems must be computer controlled. All multimedia components are integrated. The interface to the final user may permit interactivity. Information must be represented digitally.