1 Multimedia Information Representation

2 Analog Signals  Fourier transform and analysis Analog signal and frequency components Signal bandwidth and effect of bandlimiting channel

3 Signal encoder design Signal encoder Associated wave form  Nyquist sampling theorem  In order to obtain an accurate representation of a time-varying analog signal, sampling rate >= 2 x highest sinusoidal frequency component  Nyquist rate

4 Bandlimiting Filter  Undersampling (< Nyquist rate) results in additional FC being generated that are not present in the original signal  Cause the original signal to become distorted  Need bandlimiting (antialiasing) filter

5 Quantization Intervals Quantization Interval q = 2V max / 2 n Dynamic range of signal D = 20log10(V max /V min )dB Source of errors Noise polarity

6 Decoder Design

7 Text  Unformatted text: plain text  ASCII character set  Mosaic character set used in Videotex and Teletext  Formatted text: richtext  Hypertext  Pages and hyperlinks  HTML, SGML

8 Images: Graphics  Graphics: computer-generated images  Representation of a computer graphics  High-level version: e.g) PostScript  Bit-map format: GIF, TIFF

9 Images: Digitized Documents

10 Images: Digitized Pictures  Color principles Additive color mixing Subtractive color mixing

11 Raster-scan principles

12 Raster-scan Display Architecture  Pixel depth: # of bits per pixel  CLUT (color look-up table)

13 Screen Resolution Visible lines per frame Digitization spatial resolution

14 Digital Cameras and Scanners Color image capture RGB signal generation alternatives

15 Audio  Speech signals: 50Hz ~ 10kHz  Nyquist rate = 20ksps  Music-quality audio: 15Hz ~ 20kHz  Nyquist rate = 40ksps

16 PCM Speech Signal encoding and decoding Compressor Expander U-law : North America, Japan A-law: Europe and others Obtain 12-bit performance!!

17 CD-quality Audio  CD-DA (Digital Audio) standard  Sampling rate: 44.1 ksps  16 bits/sample  Bit rate for stereo music  2 channel x 44.1 ksps x 16 bits = Mbps

18 Synthesized audio  MIDI (Music Instruments Digital Interface)  A MIDI message = (status byte, data bytes, … )  Music: sequece of message  Play out by the sequencer program or saved in a file  Need sound card

19 Video Applications  Entertainment  Broadcast TV, VCR/DVD recording  Interpersonal  Video telephony, videoconferencing  Interactive  Windows containing short video clips

20 Interlaced Scanning

21 TV broadcast: Color Signals  Requirements  Color TV broadcast should be used by an existing monochrome (back-and-white) TV set  Color TV should be able to produce black-and-white pictures from monochrome broadcast  Properties of a color source  Brightness: amount of energy that stimulates the eye  Varies on a grayscale from back through to white (highest)  Independent of the color of the source  Hue( 색조 ): actual color of the source  Each color has a different frequency/wavelength  Saturation( 채도 ): strength or vividness of the color

22 Luminance and Chrominance  Luminance: related to the brightness  Amount of white light  Same as the signal used by monochrome TV  Y s = 0.299R s G s B s  Chrominance: related to the hue and saturation  Color difference signals i.e no brightness info.  C b = B s – Y s,  C r = R s – Y s

23 Composite video signal  Magnitude of the two color differences are both scaled down  If the two color difference signals are transmitted, amplitude of luminance signal can become greater than that of the equivalent monochrome signal  PAL  Y = 0.299R G B  U = 0.493(B-Y)  V = 0.877(R-Y)  NTSC  Y = 0.299R G B  I = 0.74(R-Y) – 0.27(B-Y)  Q = 0.48 (R-Y) (B-Y)

24 Signal Bandwidth NTSC PAL

25 TV Broadcast

26 TV  Analog TV  Frame rate: # of frames per second produced by a video signal  common rate: Hz  Number of scan lines per frame  scan rate: # of lines scanned per second = frame rate x # of scan lines  Interlacing: to reduce the flickering  odd field, even field  Aspect ratio: width : height  Digital TV  convert analog signal to digital signal  sampling, quantization, coding

27 Standard Analog TV Systems StandardFrame ratesScan linesAspect ratioInterlacing NTSC (480)4:32:1 PAL25625(576)4:32:1 SECAM25625(576)4:32:1

28 Digital Video (I)  4:2:2 format  Studio-Quality TV: ITU-R 601  Bandwidth 6MHz  sampling rate: 13.5MHz  720 samples / line  Bit rate = 216Mbps (PAL, NTSC) Subsampling 4:2:2

29 Digital Video (II)  4:2:0 format  Bit rate = 162Mpbs  HDTV format  16/9 aspect ratio: 1920 x 1152 pixel  Refresh rate: 25Hz / 30Hz  Studio application: 4:2:2 format  Broadcast application: 4:2:0 format

30 Digital Video (III)  SIF (source intermediate format)  VCR quality  Used in MPEG-1  ½ resolution in both horizontal and vertical directions (by subsampling): 4:1:1 format  ½ refresh rate (temporal resolution)  Bit rate: 81Mbps  CIF (common intermediate format)  Videoconferencing quality  Used in H.261  4:1:1 format  Y = 360 x 288, Cb = Cr = 180 x 144  30Hz  QCIF (quarter CIF)  Video Phone Quality  Used in H.261

31 PC Video Digitization Format System Spatial resolution Temporal resolution YCb = Cr 4:2:0525-line640 x x 24060Hz 625-line768 x x 28850Hz SIF525-line320 x x 24030Hz 625-line384 x x 14425Hz CIF384 x x 14430Hz QCIF192 x x 7215/7.5 Hz