1. Motivation Application driven -- VoD, Information on Demand (WWW), education, telemedicine, videoconference, videophone Storage capacity Large capacity disks (several gigabytes) Increase in storage bandwidth,e.g. disk array technology Network bandwidth high speed fiber optic networks - gigabit networks fast packet switching technology

2. Challenges Data intensive -- compression, high speed communication networks, high performance work stations Time dimension -- strict time constraints for retrieval, transmission and presentation Synchronization -- temporal and spatial relationships among different media Techniques for indexing, retrieval, and recognition of multimedia information

3. Essentials Data compression to reduce demands on storage space and transmission bandwidth A multimedia operating system, transport protocol, schedulers to provide appropriate delay and jitter guarantees High performance work stations and high speed networks to handle high bit rates under time constraints Media synchronization to maintain both spatial and temporal relationships among related media High performance retrieval engines QoS specification and performance guarantees

4. Multimedia Information Representation All media types are stored and processed in digital form –text, images (discrete, measured in units like bits) –audio, video (continuous, measured in bits per sec) analog signals which must be converted into digital form

5. Analog Signals A signal whose amplitude varies continuously with time (for example, electrical signals produced by microphones) –The amplitude of the signal indicates the magnitude of the sound-wave/image intensity at that time instant Signal encoder to convert analog to digital (to store and process within a computer) Signal decoder to convert digital to analog (to present and display on loudspeakers or computer monitors) Advantage of all digital form -- different media types can be readily integrated and transmitted over a single all digital communication network

6. Encoder Bandlimiting filter used to remove selected higher frequency components from the source signal analog-to-digital converter (ADC) consists of a sample and hold circuit and a quantizer –sample and hold is used to sample the amplitude of the filtered signal at regular time intervals and to hold the sample amplitude constant between samples –quantizer circuit converts each sample amplitude into a binary value

7. Sampling Rate How do we choose the sampling rate? –Nyquist sampling theorem which states that to obtain an accurate representation of a time-varying analog signal, its amplitude must be sampled at a minimum rate that is equal to or greater than twice the highest sinusoidal frequency component that is present in the signal.

8. Undersampling Alias Signals –Frequency components are generated which were not present in the original signal –Sampling rate is determined by the transmission channel bandwidth which is lower than that of the source signal –A bandlimiting filter will remover higher frequencies before the signal is sampled

9. Quantization Quantization Interval 2S/2 n Number of quantization levels 2 n Number of bits required to represent a samplen Quantization error or Quantization noise (Quantization step) -- maximum difference between the quantized sample value and the corresponding analog signal values –larger the number of quantization levels, the smaller the quantization step and the smaller the quantization noise.

10. SNR Signal-to-Noise ratio (SNR) : the digital signal quality relative to the original analog signal measured in dB SNR = 20log 10 (S/N) where S is the max. signal amplitude and N is the quantization noise. High SNR means a good quality signal.

11. Summary Sampling rate and quantization are high, then the digitized signal will be a close representation of the original analog signal.

12. Digital Audio Speech signal bandwidth 50Hz to 10kHz Music signal bandwidth 15 Hz to 20 kHz Bit rate for speech : sampling rate 20 kHz or 20 ksps –at 12 bits per sample, 240 kbps bit rate Bit rate for music at 16 bits per sample : 640 kbps (mono) or 2 X 640k = 1280 kbps (stereo) Storage or memory required for a 10 minute Bach at stereo sound: –(1280 X 1000 X 600) / 8 = 96 MBytes

13. CD-DA Standard Sampling rate is 44.1 ksps and 16 bits per sample and the bit rate is 1.411 Mbps 60 minute multimedia title requires 634.95 Mbytes Time to transmit a 30 sec portion of the title using a transmission channel of bit rate: at 64 kbps = (42.33 X 10 6 )/(64 X 10 3 ) = 661.4 s or about 11 minutes at 1.5 Mbps = 28.22s

14. Digital Video Video information units FILM -- NUMBER OF CLIPS CLIP -- NUMBER OF FRAMES FRAME -- NUMBER OF BLOCKS BLOCK -- 8x8 OR 16 x16 PIXELS PIXEL -- 6, 16 OR 24 BITS (pixel depth and determines the range of different colors that can be produced.

15. Digital Video A collection horizontal scanning lines forms a frame Horizontal and vertical blanking intervals Movement of the scene is captured by a sequence of sampled frames Frame rate, at least 25 frames per second for smooth motion Aspect ratio -- the ratio of an image’s width to its height Horizontal resolution - max number of black and white alternating vertical lines for given horizontal distance Vertical resolution - number of horizontal scan lines in a frame

16. Color Specification Primary colors, R, G, B -- Magnitude of the three signals in proportion 0.299R + 0.587G + 0.114B produces white color on the screen Luminance (brightness) component -- amount of white light a source contains -- Y = 0.299R + 0.587G + 0.114B Chrominance (hue and saturation) components -- represents the actual color of the source and strength or vividness of the color, C b = B - Y and C r = R - Y

17. 4:2:2 Format In principle, for digital TV, RGB signals could be transmitted separately. The disadvantage is that they need to be sampled at the same rate and bits per sample -- 4:4:4 In 4:2:2 format, for every 4Y samples, there are 2 C b and C r samples each.