Mixed Information types Multimedia Internet Communication Lecture 6

Multimedia ? When various media types combined into single coherent object Examples - text, database, stills, animation, graphic, video, audio Communication links use channels which can be shared by different media types Recent advances in computer/communications has significantly expanded possibilities for multimedia use Multimedia not unique to computers - e.g. children’s story tapes/books

Multimedia examples: Sound + Video advs: familiar concept, effective disadvs: familiarisation = contempt, …TV/cinema, passive etc. digitisation: various transfer + synchronisation methods Sound + Image e.g. slide show + audio commentary/music advs: portable, flexible, simple disadvs: needs preplanning, lacks movement digitisation: straightforward transfer + synchronisation not critical

Multimedia - more examples Hardcopy Image + Text e.g. picture album, brochure advs: familiar, simple, portable disadvs: lacks movement, content inflexible, copying (?) digitisation: design important Sound + Text e.g. language learning pack advs: easy to use, familiar disadvs: selected replays difficult, lacks visuals digitisation: more effective links+flexibility, easily added visuals

Selection of media The examples given indicate the diversity available Long history of traditional uses of mixed media types Some mixtures and styles of mixing are more recent Digitisation tends to present more options, but opens up more complex design issues (HCI) Availability of information in a particular form often means inclusion but…… Selection and design should reflect delivery purpose and user preferences/needs

Linkage between Media Types Traditional links between media types used in an application tend to be retained in digitised formats Use of single device (computer) often simplifies linking and synchronisation Two forms of links different types present for ‘automatic’ viewing e.g. text with an inserted picture (in-line link) explicit optional link from text to a musical soundtrack external hyperlink

Synchronisation Not particularly problematic - except for some cases of audio/video transmission Even ‘off-the-shelf’ broadcast packages have difficulties with attaining ‘lip-synch’ quality Audio can by ‘synched’ to the beginning of frames but constraints relating to channel capacity and hardware/ software affect performance Audio and image/text synchronisation is much easier image change(s) at specific point(s) in audio track

Multimedia Issues Vary depending on application Core issues: access how achieved how regulated cost restrictions equipment constraints awareness ? Bandwidth conflicts ? Timeliness Confidentiality Socialising

Another core issue - content Content control: regulatory control (law and professional code of conduct) code of practice (self-imposed/conditions of access) IRC/Chat Forums can reach over international boundaries Local rules (e.g. Video conferencing/Closed Group forum, discussion board) Parental control packages

Interface requirements Computer/Internet based Multimedia is ‘open’ to many platforms and environments via standards Standards can be open, proprietary, or from other sources Some inconsistency e.g Media player, RealPlayer, QuickTime Min requirements can specify Screen size Colour depth – no. of bits Sound sample processing - sampling rate – sample size Video resolution, frame rate Video capture/audio standard etc., Internet connection/network

Usage implications (computer based) Technical capability to achieve planned service needs to be considered E.g video links need to have appropriate bandwidth and machine/peripheral availability Cater for average and ‘peak’ activities Specialist technical support Testing and training Costing

Standards Many available Streaming Audio Video International standards JPEG, MPEG, SMIL

SMIL Based on XML Mark-up elements for Media types Audio, Video, Text, Image Synchronisation and sequencing Parallel and sequential playback Layout Specified screen area use for individual media types

Example SMIL file (slideshow)

Example file details File created by Real Slideshow using.rt file - various text elements.rp - a sequence of pictures.rm - encoded audio file Other tools create different styles Basic elements stay the same

Summary Mixed media is not new Educational use for decades Computers make it more useable Easier to construct and maintain Streaming allows most mixed media to be delivered easily over the Internet Broadband allows greater use of video

Multimedia on the web Many sites available to demonstrate streamed multimedia See assessment Some academic literature available via IEEE digital library at