Multimedia Systems As Presented by: Craig Tomastik

What is Multimedia? Multimedia consists of continuous-media, usually audio and video files. Examples of multimedia include: – MP3 audio files – DVD movies – Video clips, webcasts, and webcams sent over the internet Today, multimedia is not only sent to desktop computers, but also PDA’s and cell phones.

Delivery of the media To view the media correctly, the files must be accessed at a specific rate. For example, frames are a series of images displayed rapidly used to represent video. The faster the frames are displayed, the smoother the video appears. Video files must be accessed at a rate that matches the rate at which it is being displayed. This type of data is known as continuous-media data.

Delivery (Continued) Clients receive the multimedia data one of two ways: – Local file systems (Local Playback) – Remote servers (Streaming) Local playback includes watching DVD movies on the computer or listening to an MP3 player. These multimedia files are stored as a file on the local system and are played back from the system.

Delivery (Continued) Multimedia files stored on servers are delivered to clients via streaming. Two streaming techniques exist: – Progressive download – Real-time streaming (Which has two variations): Live Streaming On-demand streaming Random access is not always achievable with real-time streaming.

Characteristics of Multimedia Systems 1.Multimedia files can be quite large. 2.Continuous media may require high data rates. 3.Multimedia applications are sensitive to timing delays during playback.

Operating-System Issues Systems must guarantee specific rate and timing requirements, also known as Quality of Service (QoS) Requirements. These guarantees affect many components in the computer system and influence operating- system issues.

Operating System Issues (Continued) 1.Compression and decoding may require significant CPU processing. 2.Multimedia tasks must be scheduled with certain priorities to ensure meeting the deadline requirements of continuous media. 3.File systems must be efficient to meet the rate requirements of continuous media. 4.Network protocols must support bandwidth requirements while minimizing delay and jitter.

Compression Multimedia files are usually compressed from their original form to a smaller form. Because of the smaller size, less space is required for storage and the file can be delivered to the client more quickly. Compression Ratio- ratio of the original file size to the compressed file size.

Compression (Continued) Compressed (encoded) files must be decompressed (decoded) before they can be accessed. There are two types of compression algorithms: – Lossless- Compressed files can be restored to its original form. – Lossy- some of the original data is lost when the file is decoded.

Motion Picture Experts Group (MPEG) A set of file formats and compression standards for digital video. Each standard is divided into three layers – Layers 3 and 2 apply to the audio and video portions of the media file, respectively. – Layer 1 (Systems Layer) contains timing information that allows the media player to multiplex the audio and video portions for synchronized playback. Three major MPEG Standards: – MPEG-1 – MPEG-2 – MPEG-4

MPEG-1 Resolution quality (352x240, 30 frames per second, bit rate of Mbps) slightly lower than a VCR. MP3 files use Layer 3 of MPEG-1. Can achieve a compression ratio of 200:1 with video files. Typically used to download short video clips from the internet.

MPEG-2 Better quality than MPEG-1. Used to compress DVD movies & digital television, including high-definition television. Idenifies levels (resolution of the video) & profiles (video quality). The higher level of resolution & better the quality, the higher the required data rate. Typical bit rates range from 1.5 to 15 Mbps. Generally used for local playback.

MPEG-4 Most recent standard. Used to transmit audio, video, & graphics (including 2-D & 3-D animation layers). Provides scalable level of quality. MPEG-4 audio & video files can be delivered to wireless devices (including handheld computers, PDAs, and cell phones).

Requirements of Multimedia Kernels Periodic Processes- tasks that request data at constant intervals (periods). If a part of the data file is not delivered within a specified deadline, this part of data is omitted from the stream. (Possible example).

Quality of Service Levels 1.Best-effort service- system makes best effort to meet requirements, but no guarantees are made. 2.Soft QoS- Traffic streams are treated differently, with some streams given higher priority, but no guarantees are made. 3.Hard QoS- Quality-of-service requirements are guaranteed.

Parameters of QoS Throughput- total amount of work done during a certain interval (for multimedia applications, throughput is the required data rate). Delay- elapsed time from when a request is first submitted to when the desired result is produced. Jitter- Delays that occur during playback of the stream. Reliability- refers to error handling during transmission & processing of continuous media.

QoS (Continued) QoS may be negotiated between client and server. Admission control admits a request for service only if there are enough resources to satisfy the request. Simple admission control accomplished by implementing Semaphores and Resource reservations. Each resource, when implementing resource reservations, is assigned a Resource manager.

CPU Scheduling Continuous media requires that data be delivered to a client by a certain deadline. Data that doesn’t arrive by the deadline is unusable. Hard real-time scheduling guarantees a critical task will be serviced in a guaranteed period of time. Static priority- priority remains unchanged. Dynamic priority- priorities can change over time.

Disk Scheduling Disk scheduling must be optimized to satisfy the two main constraints of continuous-media files: 1.Timing deadlines 2.Rate requirements

Earliest-Deadline-First Scheduling Orders requests in a queue based on their deadlines. Requests with the closest deadlines are serviced first. A consequence of this approach is that higher seek times are incurred.

SCAN-EDF Scheduling A combination of SCAN & EDF scheduling. Uses EDF ordering, but services requests using SCAN order. Batching may allow requests with relatively close deadlines to be serviced together. Reordering requests in a batch must not prevent the request from being serviced by its deadline.

Network Management Currently, computer networks do not provide network protocols that ensure data delivery with timing requirements. Network traffic can affect the transmission of data across the network. Any timing issues must be synchronized between the server and client. Real-time transport protocol (RTP)- an Internet standard for delivering audio & video. RTP does not provide QoS guarantees, but provides features that remove jitter caused by delays and network traffic.

Deliveries from a Server to the Client Unicasting- Server delivers content to a single client. If multiple clients are present, a separate unicast is established for each client. Broadcasting- Servers delivers content to all clients. Only used in Local Area Networks and is not possible over the public Internet. Multicasting- Server delivers content to a group of clients who indicate they wish to receive the content.

Real-time Streaming Protocol Two approaches to delivering media from a server: 1.HTTP Protocol (Stateless) The web browser requests a metafile which is then opened in a media player. The media player will then request the media stream from the server. HTTP Protocol delivers the stream back to the media player. 2.RTSP Protocol Same as above, but is not stateless, which allows the client to pause and seek to random positions in the stream during playback.

RTSP Protocol Commands Setup- Server allocates resources for a client session. Play- Server delivers a stream to a client session established from a SETUP command. Pause- Server suspends delivery of a stream but maintains the resources for the session. Teardown- Server breaks down the connection & frees up resources allocated for the session. The server can switch between the Init, Ready, & Playing states when these commands are sent by the client.