1. Chapter3 Multi media Application, system, Authoring, Design

2. Introduction to Multimedia What is Multimedia? Multimedia means that computer information can be represented through audio, video, and animation in addition to traditional media (i.e., text, graphics/drawings, images).

3. Multimedia Application Definition A Multimedia Application is an Application which uses a collection of multiple media sources e.g. text, graphics, images, sound/audio, animation and/or video.

4. What is HyperText and HyperMedia? Hypertext is a text which contains links to other texts.

5. Hypertext is therefore usually non-linear (as indicated below).

6. Hypermedia HyperMedia is not constrained to be text-based. It can include other media, e.g., graphics, images, and especially the continuous media – sound and video.

7. Example Hypermedia Applications? The World Wide Web (WWW) is the best example of a hypermedia application. PowerPoint Adobe Acrobat Macromedia Director

8. Multimedia Systems A Multimedia System is a system capable of processing multimedia data and applications. A Multimedia System is characterised by: processing, Storage Generation manipulation rendition of Multimedia Information

9. Characteristics of a Multimedia System A Multimedia system has four basic characteristics: Multimedia systems must be computer controlled. Multimedia systems are integrated. The information they handle must be represented digitally. The interface to the final presentation of media is usually interactive.

10. Challenges for Multimedia Systems Distributed Networks Temporal relationship between data – Render different data at same time — continuously. – Sequencing within the media playing frames in correct order/time frame in video – Synchronisation — inter-media scheduling E.g. Video and Audio — Lip synchronisation is clearly important for humans to watch playback of video and audio and even animation and audio.

11. Key Issues for Multimedia Systems The key issues multimedia systems need to deal with here are: How to represent and store temporal information. How to strictly maintain the temporal relationships on play back/retrieval What process are involved in the above. Data has to represented digitally — Analog–Digital Conversion, Sampling etc. Large Data Requirements—bandwidth, storage, compression

12. Desirable Features for a Multimedia System The following feature a desirable for a Multimedia System: Very High Processing Power — needed to deal with large data processing and real time delivery of media. Special hardware commonplace. Multimedia Capable File System —needed to deliver real-time media — e.g. Video/Audio Streaming. Special Hardware/Software needed – e.g RAID technology. RAID technology : is a data storage virtualization technology that combines multiple disk drive components into a logical unit for the purposes of data redundancy or performance improvement. [1]storage virtualizationdisk drive [1] Data Representations — File Formats that support multimedia should be easy to handle yet allow for compression/decompression in real-time.

13. Efficient and High I/O —input and output to the file subsystem needs to be efficient and fast. Needs to allow for real-time recording as well as playback of data. e.g. Direct to Disk recording systems. Special Operating System —to allow access to file system and process data efficiently and quickly. Needs to support direct transfers to disk, real-time scheduling, fast interrupt processing, I/O streaming etc. Storage and Memory — large storage units (of the order of 50 -100 Gb or more) and large memory (50 -100 Mb or more). Large Caches also required and frequently of Level 2 and 3 hierarchy for efficient management. Network Support — Client-server systems common as distributed systems common. Software Tools — user friendly tools needed to handle media, design and develop applications, deliver media.

14. Components of a Multimedia System Now let us consider the Components (Hardware and Software) required for a multimedia system: 1-Capture devices — Video Camera, Video Recorder, Audio Microphone, Keyboards, mice, graphics tablets, 3D input devices, tactile sensors, VR devices. Digitising/Sampling Hardware 2-Storage Devices —Hard disks, CD-ROMs, Zip drives, DVD, etc 3-Communication Networks —Ethernet, ATM, Intranets, Internets. 4-Computer Systems — Multimedia Desktop machines,Workstations, MPEG/VIDEO/DSP Hardware 5-Display Devices — CD-quality speakers, HDTV,SVGA, Hi-Res monitors, Colour printers etc.

15. Applications Examples of Multimedia Applications include: World Wide Web Hypermedia courseware Video conferencing Interactive TV Groupware Home shopping Games Virtual reality Digital video editing and production systems

16. Multimedia Authoring: Systems and Applications What is an Authoring System? An Authoring System is a program which has pre-programmed elements for the development of interactive multimedia software titles. Authoring systems vary widely on orientation, capabilities, and learning curve.

17. Why should you use an authoring system? can speed up programming possibly content development and delivery However, the content creation (graphics, text, video, audio, animation, etc.) not affected by choice of authoring system; time gains – accelerated prototyping

18. Authoring Vs Programming Big distinction between Programming and Authoring. Authoring — – assembly of Multimedia – possibly high level graphical interface design – some high level scripting. Programming — – involves low level assembly of Multimedia – construction and control of Multimedia – involves real languages like C and Java.

19. Multimedia Authoring Paradigms The authoring paradigm, or authoring metaphor, is the methodology by which the authoring system accomplishes its task. There are various paradigms Scripting Language Iconic/Flow Control Frame Card/Scripting Cast/Score/Scripting — Macromedia Director Hypermedia Linkage Tagging — SMIL

20. Scripting Language closest in form to traditional programming. The paradigm is that of a programming language, which specifies (by filename) – multimedia elements, – sequencing, – hotspots, – synchronization, etc. Usually a powerful, object-oriented scripting language in-program editing of elements (still graphics, video, audio, etc.) tends to be minimal or non-existent. media handling can vary widely

21. Example Lingo scripting language of Macromedia Director Here is an example lingo script to jump to a frame global gNavSprite on exitFrame go the frame play sprite gNavSprite end

22. Iconic/Flow Control tends to be the speediest in development time best suited for rapid prototyping and short-development time projects. The core of the paradigm is the Icon Palette, contains: – possible functions/interactions of a program, and – the Flow Line — shows the actual links between the icons. slowest runtimes programs, high interaction overheads Examples: Authorware IconAuthor

24. Frame similar to the Iconic/Flow Control paradigm usually incorporates an icon palette the links drawn between icons are conceptual do not always represent the actual flow of the program. Examples Quest (whose scripting language is C) Apple Media Kit.

25. Cast/Score/Scripting uses a music score as its primary authoring metaphor synchronous elements are shown in various horizontal tracks simultaneity shown via the vertical columns. power of this metaphor lies in the ability to script the behavior of each of the cast members. easily extensible to handle other functions (such as hypertext) best suited for animation-intensive or synchronized media applications; Examples Macromedia Director Macromedia Flash — cut Down director Interface

29. Hypermedia Linkage similar to the Frame paradigm shows conceptual links between elements lacks the Frame paradigm’s visual linkage metaphor.

30. Tagging tags in text files to link pages, provide interactivity and integrate multimedia elements. Examples: HTML, SMIL (Synchronised Media Integration Language),

31. Issues in Multimedia Applications Design There are various issues in Multimedia authoring. Issues involved: Content Design Technical Design

32. Content Design Content design deals with: What to say, what vehicle to use. ”In multimedia, there are five ways to format and deliver your message.You can write it, illustrate it, wiggle it, hear it, and interact with it.”

33. Scripting (writing) Rules for good writing: 1. Understand your audience and correctly address them. 2. Keep your writing as simple as possible. (e.g., write out the full message(s) first, then shorten it.) 3. Make sure technologies used complement each other.

34. Graphics (illustrating) Make use of pictures to effectively deliver your messages. Create your own (draw, (color) scanner, PhotoCD,...),. Graphics Styles fonts colors – pastels – earth-colors – metallic – primary color – neon color

35. Animation (wiggling) Types of Animation Character Animation – humanise an object Highlights and Sparkles Moving Text Video – live video or digitized video

36. When to Animate Enhance emotional impact Make a point (instructional) Improve information delivery Indicate passage of time Provide a transition to next subsection

37. Audio (hearing) Types of Audio in Multimedia Applications: 1. Music – set the mood of the presentation, enhance the emotion, illustrate points 2. Sound effects – to make specific points, e.g., squeaky doors, explosions, wind,... 3. Narration – most direct message, often effective

38. Interactivity (interacting) interactive multimedia systems! people remember 70% of what they interact with.

39. Types of Interactive Multimedia Applications: 1. Menu driven programs/presentations – often a hierarchical structure (main menu, sub-menus,...) 2. Hypermedia +: less structured, cross-links between subsections of the same subject –> non-linear, quick access to information +: easier for introducing more multimedia features, e.g., more interesting ”buttons” -: could sometimes get lost in navigating the hypermedia 3. Simulations / Performance-dependent Simulations – e.g., Games, Flight Simulators

40. Technical Design Technological factors may limit the ambition of your multimedia presentation will discuss later

41. Overview of Multimedia Software Tools Digital Audio 1-Macromedia Soundedit Edits a variety of different format audio files, apply a variety of effects (Fig 8)

42. 2- CoolEdit/Adobe Audtion_ Edits a variety of different format audio files

43. Music Sequencing and Notation Cubase Intuitive Interface to arrange and play Music (Figs 9 and 10) Wide Variety of editing tools including Audio (Figs 11 and 12)

48. Image/Graphics Editing Adobe Photoshop Allows layers of images, graphics and text Includes many graphics drawing and painting tools Sophisticate lighting effects filter A good graphics, image processing and manipulation tool Adobe Premiere Provides large number (up to 99) of video and audio tracks, superimpositions and virtual clips Supports various transitions, filters and motions for clips A reasonable desktop video editing tool Macromedia Freehand Graphics drawing editing package Many other editors in public domain and commercially

49. Image/Video Editing Many commercial packages available Adobe Premier Videoshop Avid Cinema SGI MovieMaker

50. Animation Many packages available including: Avid SoftImage Animated Gif building packages e.g. GifBuilder

51. Multimedia Authoring Tools for making a complete multimedia presentation where users usually have a lot of interactive controls. Macromedia Director Movie metaphor (the cast includes bitmapped sprites, scripts, music, sounds, and palettes, etc.) Can accept almost any bitmapped file formats Lingo script language with own debugger allows more control including external devices, e.g., VCRs and video disk players Ready for building more interactivities (buttons, etc.) follows the cast/score/scripting paradigm, tool of choice for animation content (Well FLASH for Web).

52. Authorware Professional multimedia authoring tool Supports interactive applications with hyperlinks, drag-and- drop controls, and integrated animation Compatibility between files produced from PC version and MAC version

53. End of chapter