1. Multimedia Processing
Chapter 3.
Media Representation and Media Formats

2. Introduction
Introduction
Representation of digital media, and commonly used media formats
Evolution of analog media formats and relation to the current digital formats

3. 3.1 Digital Images
3.1 Digital Images
B/W, color, mosaic

4. Digital Representation of Images
3.1 Digital Images
Digital Representation of Images
Image defined by width, height, and pixel depth
Number of channels typically ranges from 1-4
RGB, YUV, YCrCb, CMYK, α channel
Color image = 640 x 480 x 3 x 8 = 7.37 Mbits ( MB )
Gray image, 640 x 480 x 8 = Mbits (.3072 MB)

5. α channel 3.1 Digital Images + = Final Background Foreground R G B
Final[i][j] = Foreground[i][j]* α [i][j] + Background[i][j]*(1-α[i][j]) α

6. Aspect Ratio Image Aspect ratio: width/height ratio
3.1 Digital Images
Aspect Ratio
Image Aspect ratio: width/height ratio
Commonly used aspect ratios for images are
6:4 (developing and printing photographs),
4:3 (television images),
16:9 (high definition images),
47:20 (anamorphic formats used in cinema)

7. 3.1 Digital Images
Examples

8. Aspect ratio conversion
3.1 Digital Images
Aspect ratio conversion
4:3 image (left) converted to a 16:9 format
anamorphic resized to fit the 16:9 format (left) and a 4:3 format (right).

9. Digital image formats 3.1 Digital Images File Suffix File Name
File Type
Features
.bmp
Windows bitmap
Uncompressed raster
Represents from 1-24 bits per pixel. Normally uncompressed but can use lossless run length encoding (RLE)
.pcx
Windows Paintbrush
Uncompressed/compressed raster
Like .bmp
Used only on Microsoft windows platforms
.gif
Graphics Interchange Format
Compressed raster
Predominantly used on the web.
Allows 256 indexed colors and simple animations
Alpha channel supported
Uses LZW compression
Proprietary to CompuServe
.jpg, .jpeg
Joint Photographic Experts Group
Compressed Raster
For continuous tone pictures (photographs)
Lossy compression
No alpha channel supported
Level of compression can be specified
Commonly used on the WEB
.png
Portable Network Graphics
Allows 1-48 bits of color.
Supports alpha channel
Designed to replace proprietary .gif files
File format approved by W3C
.psd
Adobe Photoshop
Uncompressed layered raster
Used for image editing
Supports a variety of color models
Supports varying pixel bit depths
Image can be organized into layers
Commonly used processing file format
.psp
Paint Shop Pro
Similar to .psd

10. Digital image formats 3.1 Digital Images .tif, .tiff
Tagged Image File Format
Uncompressed raster, also compressed raster
Used in traditional print graphics
Can be compressed using lossless and lossy methods of compression including RLE, JPEG, LZW
Tiff comes in many flavors
.fh
Macromedia Freehand
Compressed Vector Format
Proprietary to Macromedia, used by Flash Players
Supports animation
.cdr
Corel Draw
Uncompressed Vector Format
Proprietary to Corel
.dxf
AutoCAD Ascii Drawing Interchange Format
ASCII text stores vector data.
Used for 2D/3D graphical images
.ps or .eps
Postscript, or Encapsulated Postscript
Uncompressed Metafile
Supports, text, fonts, vectors, images
.ai
Adobe Illustrator
Metafile format
Proprietary format
Similar to .eps
.pdf
Adobe pdf document
Compressed Metafile
Supports text, fonts, images
Commonly used document format
Support hyperlinks
Supports authorized access
.pict
Macintosh Quickdraw
Used predominantly on Macintosh platforms
Can use RLE or JPEG compression
Supports Grayscale, RGB, CMYK or indexed color

11. 3.2 Digital Video Film to Analog Digital
Film displayed at 24 frames per second.
TV: 30 fps (NTSC); 25 fps (PAL, SECAM).

12. Analog Video Line by line display (1D signal) RGB to YUV
3.2 Digital Video
Analog Video
Line by line display (1D signal)
RGB to YUV
Interlaced or progressive

13. Analog Television 3.2 Digital Video Color Television Black and White
RGB To YUV
UV To C
Red
Green
Blue M O D U L A T I N Y V
YUV To RGB
C To UV EM ON F E R G B
Color Television
Black and White
Television

14. 3.2 Digital Video
Interlaced scanning

15. 3.2 Digital Video
Progressive scanning

16. 3.2 Digital Video
Types of Video Signals
Composite: chrominance (color) and luminance (brightness) information, along with synchronization and blanking pulses, all together in a single signal
S-Video (Super-Video), luminance and chrominance transmitted separately
Component: all three Y, U, V (or equivalent) components separate

17. 3.2 Digital Video
YUV Subsampling

18. 3.2 Digital Video
YUV Subsampling

19. North America, Japan, Canada
3.2 Digital Video
Analog video formats
Property
NTSC
PAL
SECAM
Frame Rate 30 25
Number of Scan Lines
525
625
Number of active lines
483
576
Aspect Ratio
4:3
Color Model
YIQ
YUV
YDbDr
Primary Area of Usage
North America, Japan, Canada
Asia
Europe

20. Digital video formats 3.2 Digital Video Format Name Lines per Frame
Pixels per Line
Frames per Second
Interlaced?
Sub sampling scheme
Image Aspect Ratio
CIF
288
352 N
4:2:0
4:3
QCIF
144
176
SQCIF 96
128
SIF-525
240 30
SIF-625 25
CCIR 601 NTSC
480
720
29.97 Y
4:2:2
CCIR 601
PAL/SECAM
576
HDTV (720p)
1280
59.94
16:9
HDTV (1080i)
1080
1920

21. HDTV 720p - 1280x720 pixels progressive
3.2 Digital Video
HDTV
720p x720 pixels progressive
1080i x1080 pixels interlaced
1080p x1080 pixels progressive
aspect ratio of digital TV is 16:9

22. 3.3 Digital Audio sampling rate quantization,
number of channels which many be one (mono), two (stereo) or multi channel (surround sound)

23. Digital audio file formats
File Suffix or logo
File Name
File Type
Features
.wav
WAV
Uncompressed PCM coded
Default standard for audio on PCs.
WAV files are coded in PCM format
G.711
ITU μ-law
Uncompressed audio
Universal support for telephone.
Packs each 16-bit sample into 8 bits, by using logarithmic table to encode with a 13-bit dynamic range.
Encoding and decoding is very fast
GSM 06.10
Global System for Mobile Communication
Lossy Compressed mobile audio.
International standard for cellular telephone technology.
Uses linear predictive coding to substantially compress the data
Compression/Decompression is slow
Freely available and thus widely used
.mp3
MPEG1 Layer3
Compressed audio file format
Uses psychoacoustics for compression.
Very good bandwidth savings and hence used for streaming, internet downloads.
.ra
Real Audio
Compressed format
Proprietary to Real Audio,
Capable of streaming, downloading
Comparable quality to mp3 at high data rates but not so at low data rates
AAC
Advanced Audio Codec
MPEG4
Superior quality to .mp3

24. Digital audio file formats
.mid
MIDI – Musical Instrument Digital Interface
Descriptive format
MIDI is a language of communication among musical instruments.
Description achieved by frequencies, decays, transients and event lists.
Sound has to be synthesized by the instrument
Dolby Digital (formerly called Dolby AC-3)
Compressed 5.1 surround sound
De facto standard of home entertainment
Distributed with DVD, HDTV systems
Provides five discrete channels – center, left, right, surround left & surround right, plus an additional 6 for LFE
DTS Surround Sound
Alternate to Dolby Digital
Distributed with DVDs, but not HDTV
Has higher data rate compared to Dolby Digital
THX Surround Sound
Designed for movie theatres (THX Ultra) as well home theaters (THX Select)
Has become the select brand for surround sound today
THX Surround Sound Extended
Compressed 6.1 or 7.1 surround sound
Jointly developed by Lucasfilm THX and Dolby Laboratories.
Also known as Dolby Digital ES
Has a surround back channel, placed behind audience achieving 3600 of sound

25. 3.4 Graphics 2-D Vector Graphics Representations 3.4 Graphics
Raster drawing

26. 3.4 Graphics
Vector Graphics

27. Polygon Representation
3.4 Graphics
Polygon Representation

28. 3.4 Graphics
Translation

29. 3.4 Graphics
Rotation

30. 3.4 Graphics
Scaling

31. Homogeneous Transformations
3.4 Graphics
Homogeneous Transformations
Translations :
Rotations :
Scaling :

32. Homework Read Text Chapter 4. Prepare Presentation Solve Exercises
1, 12